Search archives

[JosephWebber]

I find Everything invaluable.
However, I have a lot of files in archives (RAR, Zip etc) which do not appear.
Any chance of adding the ability (maybe optional) of listing the contents of archives?

[void]

This could be possible with file lists.
Unfortunately, I don't have an easy way to generate an EFU file list of all your rar/zip contents at the moment.

You might be able to do this manually with the file list editor:

• In "Everything", from the Tools menu, click File List Editor.
• Drag the contents of each rar/zip file into the File list editor window. (You might need Winrar/Winzip)
• Save the EFU file list to the desktop.
• Open the file list with Notepad
• Replace the temp folder with the correct rar/zip filename (this could be time consuming).
  For example:
  "c:\temp\rar$asd\somefile.txt" to "c:\archive\afolder\azipfile.zip\somefile.txt"
• In "Everything", from the Tools menu, click Options.
• Click the File lists tab.
• Click Add...
• Open your EFU file list from desktop.
• Click OK.

Any chance of adding the ability (maybe optional) of listing the contents of archives?

I will consider writing a tool to make an EFU file list from rar and zip contents.

[thalek]

I use the 3D rendering software Poser, and as a result, I have thousands of .zip, .7z, .rar, and .sit archives, with literally a million-plus files inside those thousands of archive files. The ability to index those archive files, especially on my network drives, would have tremendous value for me, and anyone who is in a similar position.

Supposedly, Windows 10 can index inside archive files, with the proper settings, but if true, it doesn't seem to extend to my files on the network drives.

Right now, I can use Funduc Software's Search and Replace program to search inside archive files, but it's not really designed for rapid search the way Everything is, and therefore it's a rather sluggish way of achieving this.

I'd like to request the ability to search inside my archives via Everything. The archive contents themselves don't change, so realtime monitoring of changes to archive contents wouldn't be necessary, except in the event a new archive file was added. So, perhaps a once-a-week (or user-selectable interval) update sweep, with the ability to force a manual check if I really have to. That would theoretically allow Everything to retain its usual speed for realtime monitoring of other files, while still keeping the archive inventory reasonably up to date

[void]

I have plans to add zip, rar, 7z and sit support in Everything 1.5.

However with one limitation, It will not search recursively (ie: it will not search inside zip files inside zip files). Everything will just read the directory table, no data will be decompressed.

I am considering supporting searching zip files inside zip files, but it will require plug-in support (I don't want to add zip, rar, 7z and sit decompressors directly in Everything).

[therube]

(I would think that there are "cataloger's" out there that would scan archives?
[It kind of stuck in my head that Cathy did that, but it does not.]

Even 7-zip.exe l * will walk a directory tree listing everything it finds.
Output that to a text file & use a grep to search it.

7-zip.exe l * > catalogue.out
grep -i bginfo.exe catalogue.out > foundfiles
showtx.exe foundfiles

Automate it or whatever.
Maybe even even have it only "update" only new or changed files?

Or do similar, massaging the results in a format that Everything likes, .efu, then open that .efu in Everything.)

[thalek]

For myself and myself alone, I don't have very many (fewer than 50) archives that have archives in them, so that works for me, specifically. Being able to scan archived files recursively works better, of course, but if it's not practical, I can live with that limitation. You'll have to hear from other people to know if that's more universally true, though.

I know that there are third party solutions for getting Windows to scan inside archived files, but I don't know if they allow recursion. Nor do I know at the moment if they work with Windows 10. (I should try an experiment to see if Funduc's Search and Replace offers recursion. It appears to completely decompress an archive in order to search for a specific piece of text; in my work-around for Everything, I've been putting in a file name or partial file name as the text to be searched for.)

And TheRube and I agree that the database might be well-served by scanning new archive files. (I'm not certain how scanning changed archived files would be any different than the current real-time monitoring of changed files. I got the impression that it was agreed that such real-time scanning of archive files would slow things down, and archive files, by their nature, are rarely modified. If real-time monitoring of changed archives is impractical, we could compromise on an optional programmable sweep of once a day/week/month.)

[xyz123]

I have plans to add zip, rar, 7z and sit support in Everything 1.5.

However with one limitation, It will not search recursively (ie: it will not search inside zip files inside zip files). Everything will just read the directory table, no data will be decompressed.

I am considering supporting searching zip files inside zip files, but it will require plug-in support (I don't want to add zip, rar, 7z and sit decompressors directly in Everything).

That's awesome. I'm waiting for this feature for years. I hope Everything 1.5 comes quickly That's the only thing lacking in Everything for me. I think that limitation would be ok for most people. It's better than having no archive support at all. Please don't let that limitation stop you from implementing this feature We need it and thank you so much for your continuing development.

[NotNull]

This seems like a nice challenge. I'll try to write something to auto-generate such an EFU list.

[thalek]

A bit of online research recently, looking for a plug-in or a DLL that would read the contents of archive files indicates that there may be a problem in Windows itself, when it comes specifically to network drives.

If true, this will not help our situation a lot.

[NotNull]

I'll try to write something to auto-generate such an EFU list.

Just wrote a proof of concept. It (still) has limitations, but there is also good news: you can open files that are inside an archive directly from Everything!


If you want to test it (Test! Not ready for "production" .. by far ) this is how:

[*] Change the location of 7Z.exe (the tool to inspect the ZIP's; it is part of 7-Zip) to your situation in the script:

Code: Select all

       set UNZIP=c:\Program Files\7-Zip\7z.exe

[*] Create a textfile with the names of the archives you want to include
This is my testset (named ZIPList.txt):

Code: Select all

"T:\TEST Folder\TEST Folder.zip"
"\\Cray1\media\_Parkeer\ZIP2EFU\AutoInstallSoftware.rar"
\\cray1\media\_Parkeer\ZIP2EFU\TEST Folder.zip
"\\cray1\media\_Parkeer\ZIP2EFU\Everything Experiments.zip"

[*] Drag/drop ZIPList.txt (or whatever you called your list of archives) to the script.
This will start a CMD window with output similar to this:

2017-07-30 16_30_10-C__Windows_system32_cmd.exe.png (8.62 KiB) Viewed 35198 times

The output file has the same name as the input file, but with file extension .EFU.

This is what the content of the EFU looks like (snippet):

Code: Select all

Filename,Size,Date Modified,Date Created,Attributes
"T:\TEST Folder\TEST Folder.zip\Everything.chm.zip",1200043,2017-06-16,, 
"T:\TEST Folder\TEST Folder.zip\Everything.ini",20227,2017-07-17,, 
"T:\TEST Folder\TEST Folder.zip\Kattenfoto.jpg",191644,2017-04-15,, 
"T:\TEST Folder\TEST Folder.zip\Windows PowerShell Desired State Configuration Quick Reference for Windows Management Framework 4.0.pdf",249646,2016-02-07,, 
"\\Cray1\media\_Parkeer\ZIP2EFU\AutoInstallSoftware.rar\runonceex.cmd",1587,2006-05-24,, 
"\\Cray1\media\_Parkeer\ZIP2EFU\AutoInstallSoftware.rar\Test RunOnceEx.cmd",53,2006-05-24,, 
"\\Cray1\media\_Parkeer\ZIP2EFU\AutoInstallSoftware.rar\WASDIT.TXT",736,2006-05-24,, 
"\\Cray1\media\_Parkeer\ZIP2EFU\AutoInstallSoftware.rar\apps.txt",722,2006-05-24,, 
"\\Cray1\media\_Parkeer\ZIP2EFU\AutoInstallSoftware.rar\RunOnce Import.reg",4970,2006-05-24,, 

[*] In Everything: Menu:File > Open File List and there select the just created EFU

Output looks like this:

2017-07-30 16_38_21-Everything.png (32.06 KiB) Viewed 35198 times

[*] If it is a file that can be opened with Explorer (double-clicking the ZIP, browse to the file in the ZIP and double-click that), you can double-click it in Everyting. The file will be extracted from the zip and opened in the associated program:

2017-07-30 17_15_59-Everything.png (29.06 KiB) Viewed 35197 times

EDIT: script is in the next message (or here: download/file.php?id=377 )


Remarks
- There are limitations. Actually: too much to describe here. This is just a proof of concept.
- I started out writing this in Powershell, but ran into trouble (limitations of Powershell and/or myself).
And thus rewrote it in CMD. In hindsight, it shouldn't be too hard to convert to Powershell again. Maybe later ..

[NotNull]

Testing would go a whole lot easier if I actually had attached the script

Can't add it to the original post anymore (why?), so it's in this message.

[NotNull]

A bit of online research recently, looking for a plug-in or a DLL that would read the contents of archive files indicates that there may be a problem in Windows itself, when it comes specifically to network drives.

If true, this will not help our situation a lot.

Do you know what kind of problems? The basic functions of the posted script seem to handle network locations (be it mapped (X:) or UNC (\\server\share) ) without (extra) issues.

But those are (old) DOS-paths. The first version of the script started out with Win32 paths ("Long UNC"), with paths like \\?\C:\path\to\zipfile, \\?\server\share\etc and \\?\UNC\server\share\etc. Those had problems with network locations (that's why I "stepped down" to \\server\share).
Maybe that's what you are referring to?

[GL1zdA]

Since you've mentioned making this functionality available via plug-ins, maybe it would be possible to adapt to Total Commander archive plug-ins for directory reading? This would give Everything the ability to index many archive types by reusing the plug-ins written for TC.

[Janus]

Being one of those people who have more archives around than I am willing to discuss.
I would recommend a simple read the TOC of an archive be the standard.

Then have a triggerable recursive function that does all the archives in a drive or directory tree that the individual user selects.
This way you can have the depth where the individual user needs it, while also making the smallest possible change to the main program.

That might also be the place to put a plug-in interface.
Which would allow an external program to recurse the archive. {Or network drive or online drive or tape drive or other thing that is specific to the setting}.
Then feedback in an efu or csv format to Everything for adding to the master index.

Said external program could then be called if someone does something with a file inside an archive.
This allows the external program to handle anything specific to its own needs, without any changes to Everything.
Which would allow things like a collection of offline disc or tape archives to be indexed.
If you select something offline, the external program prompts you for the needed media.
I have customers with tape, optical and offline shelved drives for storage, all three of which are still used.

Just some thoughts.


Janus.

[NotNull]

Recursive searching would be really useful for the tarballs in tar.gz files ...

[shawn8888]

Just wrote a proof of concept.

Thanks a lot for the script! It works!
I wish a tool from everything itself and it works faster.

[wemaba2426]

I have plans to add zip, rar, 7z and sit support in Everything 1.5.

Has this feature been implemented in 1.5 yet? I have searched the "Everything 1.5 Alpha" topic for "archive" and "archives" but there were 0 matches.

[void]

zip filenames only at the moment.

7z and rar are still on my TODO list.

[therube]

(7z can do a whole lot more then "just" .7z. It can even do .rar. Though not .sit .

Supported formats:
Packing / unpacking: 7z, XZ, BZIP2, GZIP, TAR, ZIP and WIM
Unpacking only: APFS, AR, ARJ, CAB, CHM, CPIO, CramFS, DMG, EXT, FAT, GPT, HFS, IHEX, ISO, LZH, LZMA, MBR, MSI, NSIS, NTFS, QCOW2, RAR, RPM, SquashFS, UDF, UEFI, VDI, VHD, VHDX, VMDK, XAR and Z.

)

[Bluscream]

[NotNull]

Welcome, Bluscream!

What is your question/ remark?

[Bluscream]

Hello NotNull, i just wanted to share that i got this working:

• https://github.com/Bluscream/script/blo ... __v0.2.cmd
• https://github.com/Bluscream/script/blo ... ziplist.py

Code: Select all

# region SETTINGS

# edit your "Compressed" filter in Everything to change which archives are included
use_everything_1_5_alpha = True
ziplist_file = 'ziplist.efu'
zipcontents_file = 'zipcontents.efu'
path_separator = os.pathsep
log_file = 'ziplist.log'

# endregion SETTINGS

[horst.epp]

2Bluscream
Both links are getting a 404 error.

[NotNull]

( i guess this repository is still in private mode)

Everything 1.5 also has an option to search for (partial) filenames in zip and rar archives:

Code: Select all

ext:zip   "c:\this folder\"   container-filenames:everything64

[Bluscream]

You're right. Sorry about that. Should be fixed now.

Anyway, i tried 1.5's search but it's painfully slow compared to getting the list of files beforehand so our method is still superior as of now

[NotNull]

There is a way to speed this up by adding this information to the index. That way Everything does not need to go to disk on every search to parse the zip files:

• Go to Menu:Tools > Options > Indexes > Properties
• Click the Add button
• In the search box, type container
• Select Container Filenames from the list
• Press OK
• Configure Include only folders: , Exclude folders: and Exclude files: to your liking
• Press OK
• Let Everything index the container filenames
  (progress bar will be shown in the statusbar of the main Everything window)
• Done.

Now searches for container-filenames: should give instant results.

[Bluscream]

I did that but even then when i enter "container-filenames:" as search, it will just spit out the actual archives, not their contents

[NotNull]

For that, add the container-filenames: column to the resultlist:

- Right-click the header of the result list, for example on Name
- container-filenames: will likely be one of the context menu entries (as it is indexed), but if not:
- Select Add columns
- Search for container
- select container-filenames:
- OK
- Hovering the container-filenames: fields will reveal the names in the zxip-file

[nemoW]

There is a way to speed this up by adding this information to the index. That way Everything does not need to go to disk on every search to parse the zip files:
<...>
Now searches for container-filenames: should give instant results.

Thanks for tip. Is there an option to search file names over the disks AND in archives?

[NotNull]

For example:

Code: Select all

"everything" | container-filenames:"everything"

| = OR
the first part will search for files/folders with everything in the name
the second part will search for files/folders with everything in the name when inside a zip/rar/7z file.


When the text to search for does not contain a space, you can omit the ""

[void]

Thanks for tip. Is there an option to search file names over the disks AND in archives?

Indexing filenames from zip archives is on my TODO list.
(so you can search them as normal filenames)



Please consider a filter to search filenames and archive filenames:

• In Everything, from the Search menu, click Add to filters....
• Change the Name to: Filename or Archive Filename
• Change the Search to: #replaceterms:#(:$param:#):,#(:<#term:|containerfilename:#term:>#):
• Click OK.

When this filter is active, Everything will search filenames and archive filenames.

Filters can be activated from the Search menu, Filter bar (View -> Filters), right clicking the status bar, filter macro or filter keyboard shortcut.

[Bluscream]

Indexing filenames from zip archives is on my TODO list.
(so you can search them as normal filenames)

This would be the ideal solution and then Everything's built-in feature would be on par with our custom solutions

[horst.epp]

I like the searching and indexing of filenames from archives
but I found that nested archives are currently not searched nor indexed
adding Properties Container Filenames.
Are there any plans to support this ?

[void]

I have put on my TODO list to index nested archive files. (eg: zip inside zip)

Thank you for the suggestion.

[pandr0id]

I have put on my TODO list to index nested archive files. (eg: zip inside zip)

Thank you for the suggestion.

Is there any chance this is somewhere in the beta options? Side note: you've absolutely redefined my relationship with Windows. I don't know if I'd be able to put up with it without your work. I'm between jobs but I am dedicated to supporting projects that enhance my life. Your work easily merits a double digit donation and you can count on it, from me, as soon as I'm back on my feet. Would like to see it open sourced to ensure the work goes on well into the future. Just the thought of it all grinding to a halt or becoming abandon-ware literally makes my heart feel heavy. THANK YOU

[void]

Thank you for your feedback and support pandr0id,

Only filenames directly in the container file for Everything 1.5. (nested containers are not supported yet)

Container filenames inside nested container files is still on my TODO list.

[pandr0id]

Not to be an ask-hole but I went cross-eyed trying to locate the option in 1.5: can you tell me how to enable? Assuming simple zip, rar, 7z archives are supported... Thanks again!

[void]

To display zip, rar and 7z filenames:

• In Everything 1.5, right click the result list column header and click Add columns...
• Search for: container
• Select Container Filenames
• Click OK.

Optionally, search for: ext:zip;rar;7z



To search for a specific filename inside a zip, rar or 7z file, search for:

ext:zip;rar;7z container-filename:"my filename.ext"

To index container filenames for instant searching:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Properties tab on the left.
• Click Add....
• Search for: container
• Select Container Filenames and click OK.
• Click OK.