Filled up RAM breaks usage of Everything

[Thy Grand Voidinesss]

Weeks ago I was suppose to publish such post [entitled "Automated re-scanning issues evolved into inability to use at all"]:

For a dozens od days it seemed that every time I performed a reset of system my Everything had to re-scan - and most likely a lot [all?] of items as it was taking forever to do so [because I have ~14 millions of them]. But this was not happening if I deleted Recycle Bin folders from my volumes i.e. making sure nothing was left in them - but not by merely cleaning them but by removing those folders

At first I thought that only Everything was culprit / being affected as recently I have added some Bookmarks that forced Everything to scan a lot, suspecting that some media files after being sent to Recycle Bin were then unnecessary re-scanned - and which process was somehow causing also rescanning of similar files that were not deleted

But then I have noticed that I have a similar problem in filemanager FreeCommander and even in Windows Explorer: that if I navigate to the overall Recycle Bin system folder they both pretty much freeze at the task of indexing deleted items. And that if I get rid of Recycle Bin folders then I can re-enter them [when they get re-created by the operating system] using those very same filemanagers - but this time without a sweat. And so because of that as culprit I was suspecting very long either names or original paths of some deleted items

But now it is apparently getting even worst. And that is because at first Everything was showing on its bottom bar info about accessing volume D - a letter that is assigned only for newly connected / temporarily drives. And yes: I checked back then if some forgotten pendrive was not connected to my machine and double checked if Disk Management of Windows was not reporting some new / unknown entry. Later on instead of coming to its senses and fixing itself [like it was doing multiple times after all those multiple finished re- cannings in the past days] Everything becomes unusable, because even after a reset of the system when I open Everything it is still was plain frozen. It was only after a lot of clicking and multiple minutes that a pop-up question appears allowing me to cancel a query

So now my Everything is doing yet another re-scannng after which it will become working fine - up until I close and re-open it or reset the system

And thus I am: is there an easy way to know what exactly was triggering this and what was being re-scan?


I am using Everything 1.5.0.1383a x64 Portable on Windows 10 Enterprise 20H2 x64 [10.0.19042]

But as it turns out, i.e. after weeks of first figuring this out and then having acquired more than enough empirical experience plus replacing my not so old set of 4 x 16 GB sticks for a brand new 4 x 16 GB set, the culprit all along wat the Random Access Memory - or to be precise: the lack of it

Because if my RAM resources start hitting those 99-100% then Everything might become unresponsive, as it is unable to process data required for creation of my 1.15 GB database file. So what was happening [at least in the beginning] was me getting rid of those extra / unnecessary items in Recycle Bin - and thus making Everything free of the burden of having to index them. And also I was making mistake the same mistake of opening RAM hungry programs after entering the Windows and awaiting for Everything to finish its bidding


So if the root cause is user's machine running out of RAM then should not Everything inform user about such critical situation - instead of just freezing up? Maybe there could be a pop-up showing up whenever Everything would detect the event of the operating system hitting some percentage of RAM usage? The default value could be set to e.g. 95% but the user should be able to set it to whatever level desired or even set 2 [i.e. first warning and a critical one]

And yes: my other programs also suffer from RAM running out. But some of them pop-up informative or at least general errors - and thus either inform what is wrong or encourage me to investigate what might be wrong

[void]

I will consider showing warnings when Everything is paged to disk or there's a low memory availability.

Thank you for the suggestion.



Everything will fatal error if it cannot allocate 64K or less memory.
Everything will silently fail for larger allocations when there's not enough memory.



Everything may freeze when it is paged to disk.

To prevent Everything from being paged to disk:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  virtual_lock
• Select: virtual_lock
• Set the value to: true
  
• To the right of Show settings containing, search for:
  working
• Select: max_working_set_size
• Set the value to: 2147483647
• Select: min_working_set_size
• Set the value to: 2147483647
• Click OK.
• Restart Everything (File -> Exit)

virtual_lock
max_working_set_size
min_working_set_size

[Thy Grand Voidinesss]

I will see how Everything will behave with the above settings

But is this

2147483647

value specific for someone who [like me currently] uses 64 GB of RAM? I mea: will I have to adjust it if I change my hardware in this regard?

[void]

Set min_working_set_size and max_working_set_size to a little more than your normal Everything RAM usage.

I don't recommend setting min_working_set_size and max_working_set_size to more than half your total RAM. (32GB in your case)

Windows may still page Everything to disk in low memory situations.

[Thy Grand Voidinesss]

I have just changed it to

2222222222

[Thy Grand Voidinesss]

And am I correct to assume that the more items I will have in overall have throughout my volumes and the more of them will be pre-cached at

Tools > Options > Indexes > Properties

then the more of such minimum RAM value I should reserve within Everything?

[void]

Correct, each file/folder may consume some RAM usage for each property.

Property filters will help reduce the amount of required RAM.
For example, if you index length for *.mp3 files, only the .mp3 files will consume extra RAM for the length property.

Aim to set the min_working_set_size and max_working_set_size to around the normal RAM usage of Everything.
The reason for not setting such a high value (something like 8GB+ in your case) is to prevent Everything from using all available RAM in high memory usage situations, like reindexing, sorting, etc..

[Thy Grand Voidinesss]

But what will happen now [with this option having been set up] when Everything will reach 100% of its allocated portion of RAM: it will freeze / break [like it was doing it my description of the reported issue symptoms] - or will it just work slower but it will be prevented from freezing [because it will not let go of its allocated memory]?

[void]

Everything should run fine while it is in the foreground and there's plenty of available RAM.

Windows may start paging memory to disk when RAM usage is over max_working_set_size.
Windows likes to page memory to disk when the program is minimized or in the background or when there's low available RAM.

Everything will run slower or freeze when recalling paged memory from disk.