XYplorer tags not updating with CTRL+F5

Discussion related to "Everything" 1.5 Alpha.
Post Reply
dougbenham
Posts: 24
Joined: Wed Mar 15, 2023 8:19 pm

XYplorer tags not updating with CTRL+F5

Post by dougbenham »

Hey void, sometimes hitting CTRL+F5 to reindex doesn't work. Using 1.5.0.1366a.

To reproduce..
  1. I have a file tagged in XYplorer with TAG_A.
  2. Index the file in Everything (with CTRL+F5) and the TAG_A is brought into the Tags field.
  3. Now I remove TAG_A from the file in XYplorer and tag it with TAG_B instead.
  4. Index the file in Everything (with CTRL+F5) and TAG_A is removed as expected.. but then TAG_B is not brought into the Tags field. It won't be brought in until Everything is File->Exit'd and opened again and the file is CTRL+F5 refreshed.
Potential error message from debug log:

Code: Select all

OpenFileById failed 87 for FRN 00780000000192ee
Possibly the file isn't properly closed. And actually if I go into Notepad++ and open my tag.dat file, make a modification & save it. Its like the file is reset and able to be read by Everything again, because CTRL+F5'ing works again.

Let me know any additional details I can give.
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

Could you please check something for me..

When you change a tag in XYplorer, does your %APPDATA%\XYplorer\tag.dat update?
Does the Date Modified field change on this file?
Everything currently caches your tag.dat and reloads it when the date modified field changes.
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

OpenFileById failed 87 for FRN 00780000000192ee
This is normal and unrelated, nothing to worry about.
dougbenham
Posts: 24
Joined: Wed Mar 15, 2023 8:19 pm

Re: XYplorer tags not updating with CTRL+F5

Post by dougbenham »

void wrote: Tue Jan 09, 2024 7:04 am Could you please check something for me..

When you change a tag in XYplorer, does your %APPDATA%\XYplorer\tag.dat update?
Does the Date Modified field change on this file?
Everything currently caches your tag.dat and reloads it when the date modified field changes.
Upon tagging a file, the date modified of tag.dat is immediately updated and the file contains the new tagging data. However, you may be right that the file is held open.

Edit:
  • Sysinternals Handle tool doesn't show tag.dat being held open by any program.
  • Worth noting that XYplorer has a file system watcher on the tag.dat file to reload in case there are any manual changes. Hopefully that doesn't conflict if you've also got a watcher on the same file.
  • Saving the tag.dat file with no modifications using Notepad++ solves the caching problem every time.
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

then TAG_B is not brought into the Tags field.
This is odd, to me this indicates Everything tries to reload the tag.dat but fails..

Could you please send me your debug log:
  • In Everything, from the Tools menu, under the Debug submenu, click Start Debug Logging...
  • Change the files tag from TAG_B to TAG_C
  • Select the file in Everything and hit Ctrl + F5
  • From the Tools menu, under the Debug submenu, click Stop Debug Logging...
  • This will open your Everything Debug Log.txt
  • Could you please upload this file here.
The debug log will show how XYplorer is saving your tags.dat and if Everything is reloading this file correctly.
dougbenham
Posts: 24
Joined: Wed Mar 15, 2023 8:19 pm

Re: XYplorer tags not updating with CTRL+F5

Post by dougbenham »

Code: Select all

CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
Debug log is uploaded.

Edit: Are you passing in a dwDesiredAccess of GENERIC_READ and a dwShareMode of (FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE)?
dougbenham
Posts: 24
Joined: Wed Mar 15, 2023 8:19 pm

Re: XYplorer tags not updating with CTRL+F5

Post by dougbenham »

My thought is that you should continue to open like this:

Code: Select all

CreateFileW ( "C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat", GENERIC_READ, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL )
And if you received an error code of 32, you should attempt with GENERIC_WRITE as well.

Code: Select all

CreateFileW ( "C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat", GENERIC_READ | GENERIC_WRITE, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL )
because
You cannot request an access mode that conflicts with the sharing mode that is specified by the dwShareMode parameter in an open request that already has an open handle.
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

Thank you for the debug logs.

The logs show XYplorer still writing to this file after you hit F5.
The logs show Everything fails to read your tags.dat with error code: 32 (sharing violation)



Everything opens the file with FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE.

XYplorer is likely saving to this file without FILE_SHARE_READ.
Everything will be unable to read your tags.dat until XYplorer finishes writing to this file.



The next Everything alpha update will no longer clear your tags.dat cache when XYplorer is still writing to this file.
However, you'll need to hit F5 after XYplorer has finished writing to this file.
You'll no longer need to exit Everything in the next alpha update, you'll just have to hit F5 again.
I'll have an update soon and post here again.

For now, the best option is to give XYplorer a few seconds to write changes to your tag.dat before hitting F5 in Everything.

I will consider a short timeout value.
Maybe a maximum of 1 second to keep retrying on error 32.
How large is your tags.dat?
dougbenham
Posts: 24
Joined: Wed Mar 15, 2023 8:19 pm

Re: XYplorer tags not updating with CTRL+F5

Post by dougbenham »

I don't think we're on the same page :D

My tag.dat is around 3MB. The file is written to very quickly by XYplorer. However, I have waited a full minute before attempting CTRL+F5 and it still fails.

Please re-read my last post. I was trying to explain that I think you have to try opening the file twice. First attempt should use GENERIC_READ. Then second attempt should use both GENERIC_READ and GENERIC_WRITE. The reason you have to try both is because of Microsoft's documentation that says "You cannot request an access mode that conflicts with the sharing mode that is specified by the dwShareMode parameter in an open request that already has an open handle."

I know the file is able to be opened and read because Notepad++ can read the file and Everything can't.

Please don't remove the tag.dat caching yet 😅
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

Everything-1.5.0.1366a-mod1 will now attempt to open your tags.dat with:

FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE
FILE_SHARE_READ|FILE_SHARE_DELETE
FILE_SHARE_READ|FILE_SHARE_WRITE
FILE_SHARE_READ

Everything-1.5.0.1366a.x64-Setup-mod1.exe
Everything-1.5.0.1366a.x86-Setup-mod1.exe
Everything-1.5.0.1366a.x64-mod1.zip
Everything-1.5.0.1366a.x86-mod1.zip

Please let me know if the issue persists with this mod.



Everything will always cache your tags.dat.
dougbenham
Posts: 24
Joined: Wed Mar 15, 2023 8:19 pm

Re: XYplorer tags not updating with CTRL+F5

Post by dougbenham »

With this modification you made it is working great :)

I see the debug log still contains quite a few failures:

Code: Select all

2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
2024-01-10 22:46:55.214: CreateFileW(): 32: Failed to open file C:\Users\dougb\AppData\Roaming\XYplorer\tag.dat
But yes it appears to be working great. Thanks for your time spent on this. It was an annoying issue to workaround.
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

Thanks for testing 1366a.

I'll have an official update soon.

These errors are normal as Everything tries all the different share modes.
NotNull
Posts: 5458
Joined: Wed May 24, 2017 9:22 pm

Re: XYplorer tags not updating with CTRL+F5

Post by NotNull »

I might have misunderstood, but this sounds to me like a workaround for an issue that needs to be reported on the XYplorer forum, so its developer can fix it?
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

It was just an issue with Everything clearing cache when failing to read the tags.dat.

No issue with XYplorer.
NotNull
Posts: 5458
Joined: Wed May 24, 2017 9:22 pm

Re: XYplorer tags not updating with CTRL+F5

Post by NotNull »

Right. Thanks for explaining!
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

Everything 1.5.0.1367a removes opening the file with different share access as this isn't necessary.

This version will continue to not clear your cache when opening your tag.dat fails.

Does the issue persist with this version?
dougbenham
Posts: 24
Joined: Wed Mar 15, 2023 8:19 pm

Re: XYplorer tags not updating with CTRL+F5

Post by dougbenham »

It works fine in 1.5.0.1367a. Can you share what the solution was?
void
Developer
Posts: 16668
Joined: Fri Oct 16, 2009 11:31 pm

Re: XYplorer tags not updating with CTRL+F5

Post by void »

Previous versions would clear Everything's tags.dat cache on Ctrl+F5

If Everything failed to read your modified tags.dat, Everything's tags.dat cache would remain cleared.
Everything would also remember the tags.dat modified timestamp and not attempt to update again.



1367a+ will only clear Everything's tags.dat cache after successfully reading the modified tags.dat.
Everything will only remember the tags.dat modified timestamp after successfully reading the modified tags.dat.
This allows Everything to keep retrying to read your tags.dat for changes if it is currently locked.
Post Reply