NTFS indexing for symbolic links, junctions, etc.

Discussion related to "Everything" 1.5 Alpha.
Post Reply
CrendKing
Posts: 11
Joined: Sat Apr 24, 2021 2:49 am

NTFS indexing for symbolic links, junctions, etc.

Post by CrendKing »

I noticed that as of Everything 1.5.0.1273a, the NTFS indexing does not include symbolic links, junctions etc. void explained earlier that these links need to be added to manually monitored Folders. However, I install many apps through Scoop and all the persist directories are linked to the app via junction. It'd be really frustrating to add each of the tens of persist folders to Everything, and repeat that process every time a new app is installed.

Is this a design choice or technical limit to not index them? If design choice, is it possible to reconsider it in 1.5?
void
Developer
Posts: 16672
Joined: Fri Oct 16, 2009 11:31 pm

Re: NTFS indexing for symbolic links, junctions, etc.

Post by void »

There's technical reasons and performance concerns for not following folder junctions.

There are many circular referencing folder junctions on Windows by default:

For example:
C:\ProgramData\Application Data => C:\ProgramData
C:\Users\User\AppData\Local\Application Data => C:\Users\User\AppData\Local


Also, each USN event will need to check against each folder junction to see if it is a child.
For 10 folder junctions this would be OK, for 100+ you would notice a performance hit.


You will need to manually add the folder junctions to Everything 1.5:
  • (For example add the folder junction "C:\ProgramData\Application Data" which links to "C:\ProgramData")
  • In Everything, from the Tools menu, click Options.
  • Click the NTFS tab on the left.
  • Right click in the NTFS volumes list and click Add....
  • To the right of GUID, click Select....
  • Select the drive where the folder junction resides (eg: C: drive).
  • Change the Path to mount location in Everything (eg: C:\ProgramData\Application Data)
  • Change the Root to the target path without the root part (eg: ProgramData)
  • Repeat for additional folder junctions.
  • Click OK.
I will consider an option to automatically add all folder junctions.
CrendKing
Posts: 11
Joined: Sat Apr 24, 2021 2:49 am

Re: NTFS indexing for symbolic links, junctions, etc.

Post by CrendKing »

Is it feasible to build a set of current links and junctions, and never follow in if the target is in that set, to handle circular reference? Of course I don't know the performance implication, but the totally number of these shouldn't be gigantic comparing to actual directories, right? In the least, you could limit the set size to a configurable amount, disable indexing in these if actual size exceeds. User would be aware of the implication when raising that number.

Of course, if this deemed to be too complicated, you are welcomed to leave out.
void
Developer
Posts: 16672
Joined: Fri Oct 16, 2009 11:31 pm

Re: NTFS indexing for symbolic links, junctions, etc.

Post by void »

Is it feasible to build a set of current links and junctions, and never follow in if the target is in that set
I'm considering an "Add all folder junctions" item to the right click menu in Tools -> Options -> NTFS.

This would only follow junctions once.
Same as adding a folder junction manually with the method above.



The following is a valid path in Windows:
C:\ProgramData\Application Data\Application Data\Application Data\Application Data\Microsoft

If you were to index the folder junction C:\ProgramData\Application Data, Everything would not find:
C:\ProgramData\Application Data\Application Data\Application Data\Application Data\Microsoft
Only files in the following would be found:
C:\ProgramData\Application Data\Microsoft


I doubt Everything will ever follow folder junctions by default, it's just too messy.
There would be so many duplicates under C:\Users\User

Everything will require the user to specify which folder junctions to follow.



Performance is a concern as you add more folder junctions.
An example, I delete 100 files:
With no folder junctions indexed, this would be 100 lookups and removals from the Everything database.
If I am also indexing 100 folder junctions, this would turn into 10000 lookups and possible removals.

There is 47 folder junctions on a fresh Windows 10 install.
Including all these folder junctions would make Everything up-to 47 times slower at updating your indexes.
CityguyUSA
Posts: 15
Joined: Wed Apr 15, 2015 4:10 am

Re: NTFS indexing for symbolic links, junctions, etc.

Post by CityguyUSA »

I'm not sure I understand why this is desirable? Any junction would show up somewhere.
Post Reply