void wrote: ↑Tue Mar 21, 2023 9:00 amPlease note that setting the maximum size is per file, not the entire content index size.
Thank you. Might the dialogue box say this? /Maximum Size/Maximum Size of a file/ or similar.
(1) It seems that a month ago I knew how to “Enable Content Indexing” but now my mind is blank. How do I enable content indexing?Tools -> Options -> Content -> Index file content.
Thank you. I think I picked up “
enabling” by
reading the Forums. In my ignorant state I was so led to hunt for the word “Enable” in the dialogue box
(2) What is indexed? Is it the entire content of my 16,400 documents? Is that the 4,93 GB? If so that suggests about 300 KB per document, which seems high to me. I will do further analysis if required.
Any files in your index that match the specified filters, eg: *.doc on the T: drive. Only the text content is indexed. For example, if you have a word document that has a 2 MB image and a couple paragraphs of text, only the text is indexed. A minimal amount of RAM will be used to store the text.
Thanks too for this. So my “T:\ *,doc” will obtain all MSWord documents on my data partition.
I understand that {images} will be avoided.
What about VBA code? Is that considered to be “text” or is it ignored because it is part of a VBA project (or any other internal construct, such as the strings within {SEQ} field codes and many others?
Shorter question The text that gets indexed is equivalent to that that would be carried forward with a Copy/Paste via Notepad.exe? (I think the answer to this question is “yes”)
(3) At this point that 4.93 GB is stored in 16 GB of Ram, correct?
No, that is the file size, the content index will be much smaller. The indexed content size can be obtained from Tools -> Debug -> Statistics -> File data size.
“the file size”, which file?
I understand that the content index will be much smaller – each unique word appears only once, correct?
I think that “01d95b561165e21a” is your internal identifier to this index.
For reference, this morning’s search of “T:\ *.doc” says 15,5497 objects; 4.66 GB” I don’t see anything like that in the debug, and between you and me I am not in the business of decoding debug data from Everything
so I am ready to back off a bit at this time :wipes brow with relief:
Code: Select all
Database
Location: C:\Users\Chris077\AppData\Local\Everything\Everything-1.5a.db
Indexed file properties: Name, Path, Size, Date Modified, Content
Indexed folder properties: Name, Path, Size, Date Modified
Fast sorts: Name, Path, Size, Date Modified
Folder count: 74,113
File count: 484,419
Total item count: 558,532
FAT index count: 2
NTFS index count: 1
ReFS index count: 0
Network drive index count: 0
Folder index count: 0
File list index count: 0
Network index count: 0
Total index count: 3
Folder data size: 4,233,642 bytes
File data size: 165,557,761 bytes
Total data size: 169,791,403 bytes
Average folder data size: 57 bytes
Average file data size: 341 bytes
Folder index size: 2,371,616 bytes
File index size: 15,501,408 bytes
Total index size: 17,873,024 bytes
Total size: 187,664,427 bytes
Folders created: 11,037
Folders modified: 1,018,500
Folders deleted: 4,897
Folders moved: 235
Files created: 122,198
Files modified: 493,901
Files deleted: 77,666
Files moved: 20,440
Journal
Enabled: Yes
ID: 01d95b561165e21a
Size: 103,442 bytes
Max size: 1,048,576 bytes
First item ID: 1
Next item ID: 637
Item count: 636
Build
Count: 20
Total duration: 01:14
Minimum duration: 0.010010 seconds
Maximum duration: 21.450872 seconds
Average duration: 3.729881 seconds
Last duration: 1.266843 seconds
Last build date: 03-20-2023 3:32 PM
Last rebuild reason: Property structure changed.
Update
Count: 171,731
Total duration: 00:11
Minimum duration: 0.000000 seconds
Maximum duration: 0.117208 seconds
Average duration: 0.000069 seconds
Last duration: 0.007821 seconds
Last update date: 03-21-2023 9:52 AM
Load
Count: 22
Total duration: 00:03
Minimum duration: 0.073363 seconds
Maximum duration: 0.415929 seconds
Average duration: 0.137348 seconds
Last duration: 0.415929 seconds
Last load date: 03-21-2023 9:43 AM
Save
Count: 34
Total duration: 00:06
Minimum duration: 0.004985 seconds
Maximum duration: 0.553827 seconds
Average duration: 0.185798 seconds
Last duration: 0.334519 seconds
Last save date: 03-21-2023 7:58 AM
Next scheduled save date: 03-22-2023 4:00 AM
Total bytes written: 1,255,276,680
Query
Count: 568
Total duration: 00:04
Minimum duration: 0.000002 seconds
Maximum duration: 0.981268 seconds
Average duration: 0.007662 seconds
Last duration: 0.000745 seconds
Last query date: 03-21-2023 10:47 AM
Total result count: 41,130,285
Maximum result count: 1,473,811
Average result count: 72,412
Last result count: 15,497
Sort
Count: 7
Total duration: 00:00
Minimum duration: 0.000453 seconds
Maximum duration: 0.035992 seconds
Average duration: 0.007058 seconds
Last duration: 0.001832 seconds
Last sort date: 03-18-2023 3:25 PM
FAT Index
Volume name: \\?\Volume{130b3437-919d-4d7f-ac7b-923bf5507013}
Path: F:
Root:
Include only:
Drive type: Fixed
Label: 7092187927
Index number: 0
Date indexed: 03-20-2023 2:37 PM
Out of date: No
Online: No
Disk device index:
Multithreaded: Separate device thread
Folder count: 1,730
File count: 9,471
Last rescan date: 03-21-2023 9:51 AM
Last rescan successful: No
Last successful rescan date: 03-20-2023 2:37 PM
Last successful rescan duration: 00:00:01.416
Next scheduled rescan date: 03-22-2023 3:00 AM
FAT Index
Volume name: \\?\Volume{2fbb81a2-c283-11ed-83dd-141333b95f06}
Path: T:
Root:
Include only:
Drive type: Fixed
Label:
Index number: 1
Date indexed: 03-18-2023 4:15 PM
Out of date: No
Online: Yes
Disk device index:
Multithreaded: Separate device thread
Folder count: 24,919
File count: 240,394
Last rescan date: 03-21-2023 9:52 AM
Last rescan successful: Yes
Last successful rescan date: 03-21-2023 9:52 AM
Last successful rescan duration: 00:00:11.735
Next scheduled rescan date: 03-22-2023 3:00 AM
NTFS Index
Volume name: \\?\Volume{dba064f0-1ec8-4e09-a933-1c3148a49947}
Path: C:
Root:
Include only:
Drive type: Fixed
Label: Windows
Index number: 2
Date indexed: 03-18-2023 4:15 PM
Out of date: No
Online:
Disk device index: 1
Multithreaded: Separate device thread
Folder count: 47,464
File count: 234,554
USN Journal ID: 01d959c8c41f95d2
Next USN: 000000000001a068
(4) Is there a way to store the content index from RAM to HDD (for those without an SSD) which might save time tomorrow morning. More to the point, a stored index could be reclaimed as a stable platform of data for future use.
No, this defeats the purpose of content indexing in Everything. If Everything needs to access that database on disk, you might as well disable content indexing and just read the content directly from disk. There would be no noticeable performance between accessing the Everything database from disk vs. directly reading the content from disk.
For the user with a HDD performance MIGHT be an issue. For that same user with 50,000 MSWord documents, performance MIGHT be an issue.
I understand (but have not yet used Windows indexes) that for truly massive domains, “si” is preferable.
But what about the user who wants a stable testing platform? They want to set up a content index (at some production expense), and then return to that content index time and time again, perhaps over a period of several weeks while they perform analysis on content?
I have a program that converts and cleans 10,000 documents over night. In the past I would have loved a snapshot of the content so that the client and I could go back and forth on creating rules for fixing text.
I was thinking of something that is essential a binary dump to disk, and the ability to reload that dump, so that I could continue analysis day after day.
I have no need for this now, but am trying to understand Content indexing, is all.
(5) Is there but one content index in RAM, or when I have another shot in 30 minutes time, will I then have two content indexes in RAM?
Everything only has one content index. The single content index is always stored in RAM. If you rebuild your database, Everything will keep the old content index and new content index at the same time to avoid going back to disk. Once the new index is built, the old content index is freed.
Thank you; understood!
Why should I be bothered with this?
Generally, content indexing is not recommended. If you have a good SSD, reading content directly from disk is going to be very fast. If you want instant content searching for a few files, please consider content indexing. (An example might be a couple 100 MB of source code files) Another use would be to index content on volumes that are often offline.
Thank you. So a threshold region might be 100 MB 200 MB; anything below that use the plain garden-variety of “Content:”; above that region, start using Windows “si”. I don’t hold you to that range, but I think a user needs a rough guideline as to when to, and when not to.
(later) Is the performance hit likely to be gradual ("Uh-oh! Better start using "si"") or is a user possibly going to face some sort of error, such as a crash of Everything?
(later still) Should the user make use of "si" always, as being (I imagine) simpler to set up whatever it is just once, and use one consistent technique?
I don’t apologize for the length of these posts. I have gained greatly in my understanding of how this all works; even better, I (with my SSD!) have learned not to worry about “content indexing” at this time.
Many, many thanks.
Chris
Index file content