Missing results in specific folders

[CrendKing]

The v1.5.0.1255a works most of the time. However, it does not show any result for specific folders, which I can't tell having any outstanding properties comparing to other working folders. Let's call one of the folder C:\A\B. This folder has lots of subfolders and files, such as C:\A\B\c

1. If I put "C:\A\B", only "B" is in the result.
   Console output:

   Code: Select all

   search '"C:\A\B"' filter '' sort 1 ascending 1
   parse flags 00000000 type 00c01100
   TERM C:\A\B
   FOLDER TERM START 0000000009e61cb8 M 000000000014e8e0 N 000000000014ea00
   0000000009e61cb8 e01104 M 000000000014e8e0 N 000000000014ea00 OP 163 C:\A\B
   FILE TERM START 0000000009e61cb8 M 000000000014e8e0 N 000000000014ea00
   0000000009e61cb8 e01104 M 000000000014e8e0 N 000000000014ea00 OP 163 C:\A\B
   same results
   set high UI thread priority
   restore UI thread priority
   new results 1
   

2. If I put "C:\A\B\", 0 objects.
   Console output:

   Code: Select all

   search '"C:\A\B\"' filter '' sort 1 ascending 1
   parse flags 00000000 type 00c01100
   TERM C:\A\B\
   FOLDER TERM START 0000000009e60a58 M 000000000014e8e0 N 000000000014ea00
   0000000009e60a58 e01104 M 000000000014e8e0 N 000000000014ea00 OP 163 C:\A\B\
   FILE TERM START 0000000009e60a58 M 000000000014e8e0 N 000000000014ea00
   0000000009e60a58 e01104 M 000000000014e8e0 N 000000000014ea00 OP 163 C:\A\B\
   use old result arrays
   

3. If I put "C:\A\B\c", 0 objects.
   Console output:

   Code: Select all

   search '"C:\A\B\c"' filter '' sort 1 ascending 1
   parse flags 00000000 type 00c01100
   TERM C:\A\B\c
   FOLDER TERM START 0000000009e61e08 M 000000000014e8e0 N 000000000014ea00
   0000000009e61e08 e01104 M 000000000014e8e0 N 000000000014ea00 OP 163 C:\A\B\c
   FILE TERM START 0000000009e61e08 M 000000000014e8e0 N 000000000014ea00
   0000000009e61e08 e01104 M 000000000014e8e0 N 000000000014ea00 OP 163 C:\A\B\c
   use old result arrays
   found 0 folders with 0 threads in 0.000002 seconds
   set high UI thread priority
   found 0 files with 0 threads in 0.000002 seconds
   SET SORT 1
   set sort 1 ascending 1 is valid 1
   already sorted
   finished sort, time taken 0.000983 seconds
   update selection 0.000000 seconds
   ready
   DB_WAIT: _db_ready_proc waiting for _db_search_thread_proc...
   DB_WAIT: _db_ready_proc waited 0.000356 seconds
   new results 0
   

This is reproducible with completely default settings.

Now if I add C:\A\B to folder index, suddenly it works as expected.

[void]

Is C:\A\B a folder junction?

To confirm, from a command prompt, run the following command:

dir /al C:\A

Is B listed as a folder junction?

Everything does not follow folder junctions.


To add a NTFS folder junction to your Everything index:

• In Everything 1.5, from the Tools menu, click Options.
• Click the NTFS tab on the left.
• Right click in the NTFS volumes list and click Add....
• Click Select to the right of GUID
• Select the GUID for your <junction target> and click OK.
  For example, if the junction target is on the C: drive, select the GUID for the C: drive.
• Change the Path to: C:\A\B
• Change the Root to: <junction target>
  where <junction target> is the junction target without the root.
  For example, if the junction target is c:\some\other\folder, change the root to: some\other\folder
• Click OK.

-or-

Add the junction as a folder index (as you have already done):

• In Everything 1.5, from the Tools menu, click Options.
• Click the Folders tab on the left.
• Click Add....
• Select your C:\A\B folder and click OK.
• Click OK.

Please make sure no search options are checked under the Search menu (eg: uncheck match case, uncheck match whole words, uncheck regex)

Please make sure the Everything filter is active under the Search menu.

Please make sure these files are not excluded from Tools -> Options -> Exclude.

Please try forcing a rebuild (Not likely to help if a fresh install doesn't pickup the directory):

• In Everything, from the Tools menu, click Options.
• Click the Indexes tab on the left.
• Click Force Rebuild.
• Click OK.

[CrendKing]

Is C:\A\B a folder junction?

C:\A\B is a normal directory with same attributes and permissions as its siblings.

Please make sure no search options are checked under the Search menu (eg: uncheck match case, uncheck match whole words, uncheck regex)

Please make sure the Everything filter is active under the Search menu.

The only thing "checked" under Search menu is "Everything"

Please make sure these files are not excluded from Tools -> Options -> Exclude.

All 3 checkboxes are unchecked.

Please try forcing a rebuild (Not likely to help if a fresh install doesn't pickup the directory):

Done several times, plus hard delete the database file and let it rebuild automatically.

-----

One more thing. I enable "Index folder size", let it rebuild index, then check C:\A\B. It's size is 0. But B's creation time and modify time are all correct. So it seems Everything just failed to follow into that directory.

Also, v1.4 has no problem at all. I tried several latest versions of the v1.5 branch and all have the problem. You want me to do a binary search to pinpoint version?

Screenshot showing the difference: https://imgur.com/a/KUuXyki

[CrendKing]

Another interesting discovery. I copy the whole folder from C:\A\B to D:\A\B. The subfolders and files in D:\A\B are shown in Everything. Then I delete C:\A\B, move D:\A\B back. Suddenly C:\A\B are shown now.

However, if I simply rename C:\A\B to C:\A\C, it does not work. I wonder if these folders have some kind of hidden properties in NTFS that prevents your new indexing algorithm from picking it, like alt stream? By copying it, somehow these things are removed and allowed.

[void]

Thanks for the screenshot.

You want me to do a binary search to pinpoint version?

Thanks for the offer, I don't think this is necessary yet..

Could you please send the debug logs after forcing a rebuild:

• In Everything 1.5, from the Tools menu, under the Debug submenu, click Start Debug Logging.
• From the Tools menu, click Options.
• Click the Indexes tab on the left.
• Click Force Rebuild.
  ---please wait for Everything to do a rebuild---
• From the Tools menu, under the Debug submenu, click Stop Debug Logging.
  ---this will open your debug log in Notepad.
• Could you please send me your debug log.

Debug Logging.
Privacy

[CrendKing]

Email sent.

[void]

Thanks for the debug logs.

file count: 537288, excluded: 294187, no parent: 112737

The logs show the entire drive is indexed.
However, most files are removed as the parent folder is missing.

Could you you please confirm if the same happens in Everything 1.4 after forcing a rebuild.

• In Everything 1.4, from the Tools menu, click Options.
• Click the Indexes tab on the left.
• Click Force Rebuild.
• Click OK.

Are the same files missing? -If so, It may indicate an issue with your NTFS volume.

Please backup anything important on this drive and check the volume for errors:

• In Windows Explorer, right click your drive and click Properties.
• Click the Tools tab.
• Click Check now...
• Click Start
• If prompted to schedule a scan on the next restart, click Yes.

[CrendKing]

Could you you please confirm if the same happens in Everything 1.4 after forcing a rebuild.
Are the same files missing? -If so, It may indicate an issue with your NTFS volume.

I mentioned in the second reply that 1.4 has no problem at all. All files are shown correctly. This is a 1.5 specific issue. Also you can see that in the screenshot.

Please backup anything important on this drive and check the volume for errors:

Checked and found no problem.

Code: Select all

Windows has scanned the file system and found no problems.
No further action is required.

[void]

Thanks for checking the drive.

I mentioned in the second reply that 1.4 has no problem at all. All files are shown correctly. This is a 1.5 specific issue. Also you can see that in the screenshot.

I saw you mention 1.4 was OK.
I just wanted to confirm if the issue was present in Everything 1.4 after a forced rebuild.

Could you please send me your Everything-1.5a.ini in the same location as your Everything64.exe

I'll dig a little deeper...
The missing files/folders are found, they are just excluded for some reason...

[CrendKing]

Sent.

[void]

Thank you for the Everything.ini

I'm looking into the issue..

[NotNull]

Maybe this can help to zoom in ...

I had something similar after upgrading v1252 to 1254.
After some searches, I noticed that with searchquery "media" no results showed up. Not even the 4000+ object in the C:\windows folder that match.
However, "media\" returned all files and folders in C:\media. Furthermore: emptying the search bar - which should show all files on this system - limited resultst to only the files and folders in "c:\media".

So it looked like some search results were cached and re-used.

I reverted to v1252, had no issues there and updated to v1254 again. No issues there either. Never saw this phenomenon again.


Everything x64 is almost entirely default. Even Windows (10 Pro 1909) is completely default.
(which is terrible, but this system is dedicated to testing and toying around with Everything 1.5 for the moment)

[void]

Thanks for the info NotNull,

emptying the search bar - which should show all files on this system

Sounds like another unrelated issue, I'll look into this.



In CrendKings case, Everything is finding the files during indexing.
However, the files are also removed during indexing as they have been marked as excluded.

Files can be marked for exclusion if they match an exclude filter (none set in this case) or if a parent reference was not found.

I have not been able to re-produce the issue, I'm still looking into the issue..

[void]

Thank you for assisting with finding the issue CrendKing,

Everything 1.5.0.1256a fixes an issue with indexing NTFS volumes with out-of-sequence Master File Table fragments.

Forcing a rebuild might be necessary to pickup missing files or folders:

• In Everything, from the Tools menu, click Options.
• Click the Indexes tab on the left.
• Click Force Rebuild.
• Click OK.