Property Indexing causes Sharing Violation

[Michael]

G'day all,

I've been loving the new Property indexing features of v1.5. There's a fair bit of grind to populate the data to start with, but having the height/width/codec/etc indexed for video files, for example, is a fantastic resource to have. (Find all those big AVC video files and convert them to HEVC to save space, for example.)

I generally index all my local and network drives with nothing in the way of exclusions (OneDrive files the exception to prevent unnecessary downloads) but recently came across a bit of an issue when using YT-DLR (a YouTube Video Downloader) that wasn't happening before. I can't isolate the problem to something changing in that software or in Everything (hadn't updated either recently), and kinda suspect it's due to a bit of an overall system slow down meaning file operations are a bit slower.

But anyway, the issue:

YT-DLR downloads videos using several steps:

1) Download Subtitles
2) Download Video Track to mp4 file
3) Download Audio Track to m4a file
4) Mux Video/Audio Tracks into .MP4 file
5) Embed the subtitles in the MP4
6) Embed any metadata in the MP4
7) Embed the thumbnail image in the MP4

Steps 5 through 7 a basically done by creating a .temp.mp4 file as the embedding is done, then delete the original .mp4 and rename the .temp.mp4 to just .mp4. And it's during these steps that the process sometimes aborts with a "process cannot access the file as it is in use by another process" error. Using the classic Sys Internals PROCMON tool I was able to establish that the "other process" was Everything.exe, and it's my strong suspicion it would be the Property indexing operation of Everything which is locking the file for a sufficiently long time that YT-DLR gives the error.

I've worked around the problem by simply excluding the temporary downloading folder from Everything altogether, which is OK, but means when doing a search I might forget about the fact that a file is in that folder and not being display.

Is there an advanced option where Everything can delay the indexing of properties for 30 second after the last modified date or something like that? Or can I just excluding files in a particular folder from having their Properties indexed, but still index the basic file information?

Many thanks.

Michael

[Michael]

Just some extra detail...

I said "YT-DLR", when of course it's "YT-DLP". Whoops.

I've also attached a clip of the PROCMON log, showing the error. You can see FFMPEG (called by YT-DLP) working on the file, then Everything starts working on it, then FFMPEG finishes, so YT-DLP starts the next step, and immediately gets an error, presumably because Everything is still working on it.

Cheers

Michael

[therube]

Is it (also) yt-dlp that does 5-7?

Have a sample link / command-line that does 1-7?
(And you say the issue isn't consistent.)

And by excluding \Grab\Temp\ you don't run into the issue.

Are height/width/codec the only Properties you're indexing?

[void]

Thank you for the issue report Michael,

I have put this on my Things to fix list.

A delay should help.

It's unfortunate that Windows Property System opens files without FILE_SHARE_WRITE (or yt-dlp is opening files without FILE_SHARE_READ)



For now, please exclude these files from your indexed properties.

[Michael]

G'day,

Rube:

Yes, steps 5-7 are all YT-DLP, but quite often it calls an external process like FFMPEG or AtomicParsley or whatever. The problem with consistency is that not all files are downloaded as the Video and Audio as separate files that then need to be muxed, and then of course not all files have subtitles, or thumbnail images, or metadata, so which steps are applied can vary a lot. And as I say, the size of these files can make a significant difference to processing time. My gut impression is that short files don't often result in an error, but big files more often do.

The Properties I'm currently indexing are Audio Format, Height, Length, Title, Video Format, and Width, and I'm doing this for "*.flac;*.mp3;*.wav;*.ogg;*.mkv;*.mp4;*.avi;*.flv;*.webm"

Void:

I think a delay would be the best "general" solution, as it's not as if I can't think of other situations where similar sorts of operations could be performed on other files anywhere in the file system. (e.g. Torrents downloading, video editing, etc.) Basically, any situation where you're working on a multimedia file, and that file is written to disc and is now valid for the indexing process to begin, but you then try to make a further change to that file (even renaming it?) could currently result in an error. If the Windows Property System demands the more exclusive access then working around that would seem to be the only option.

Cheers,

Michael

[void]

Everything 1.5.0.1379a will now wait 1 second after a file is modified before reading properties.

To customize this delay:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  property delay
• Select: property_read_on_change_delay
• Set the value to: the desired number of milliseconds to wait.
  For example: 1000
  Default: 1000
  Set to 0 to disable the delay.
• Click OK.

This delay also applies to icons, thumbnails and the preview pane.

property_read_on_change_delay

[Michael]

Hey void,

All my initial testing appears to show that this is fix is working totally as designed! No sharing violation type messages for a week or so now!

Many thanks,

Michael