File tags: yet another suggestion

Have a suggestion for "Everything"? Please post it here.
Post Reply
NotNull
Posts: 5517
Joined: Wed May 24, 2017 9:22 pm

File tags: yet another suggestion

Post by NotNull »

To be clear: this is not a suggestion to support tagging/metadata in Everything, but more a suggestion how to implement it if it gets supported.
Personally, I don't think I have much need for tagging support. But that's what I thought about Everything itself at first, too. And look at me now ;-)


After some thinking, I came to the conclusion that - under the current circumstances - it is not possible to have a universal tagging standard.
Therefore, if Everything wants to index or even read tags, it has to accept that it needs to "understand" different file formats. Office document, images, MP3's, PDF's .. they all save it in a different place, using different synatxes.

Recently I discovered Disk Explorer Professional (http://www.tjelinek.com/main.php?section=d)
Basically, it's an EFU list "on steroids" as it also indexes tags, metadata and even preview of common documents (Office, pdf, images, movies). (Side-effect is that it's database size is 8+ times that of Everything ...)
It uses plug-ins to "talk to" he different file formats. They call those Filters. There is even an SDK to develop your own Filters.

I think that is also the route Everything should take, if it wants to index tagging information.\

Here's how I see that (this week ;-) ):

- Under Options > Indexes there should be an extra entry: "Content" (but then a better name, perhaps).
- Here you can schedule the folders you want to scan for tags, including a schedule (basically the same mechanism/layout as Folder indexing)
- Further Content Indexing options include the Filters (drivers) to use.

Everything *knows* what files are changed or added. Those are the only files that need to be (re-) read for tagging info.
I don't think that Everything currently keeps track of deleted files, but that would come in handy, to prevent Everything from "ploughing" through the folder structure to find out what's new. (EDIT: when NTFS indexing is used; in case of Folder Indexing, Everything has to read the foldertree. Could scan for tags at the same time).

This tagging info are saved in one of those serverkess unstructured NoSQL database (like UnQLite; those are very good in storing key/value pairs an documents), as there are way too many different tagging keys availabe to create a proper structure for that (and those can/will change with an updated document layout)
Those NoSQL databases are really fast, btw

This database is separate from the current (filename) database and is *not* loaded in memory.
When some tags are queried from Everything, this database is read from disk. Otherwise Everything would occupy too much RAM (there are people here with 8 million and even 30 million files/folders!)
That is a little slower than reading from RAM, but beats scanning multiple terabytes of diskspace by miles.

For the result list: don't use separate headers for every possible tag-key. That's how Windows Explorer does it: adding a column for evey possible tag. Don't like that (although useful from time to time ..)
Instead, use column headers like Tag1, tag2, .. tag5 (number of columns shown configurable?).
And if you search for something like tag:2018. it will show for a matching picture in column Tag1: width = 2018 and for a matching mp3: year: 2018


That's it for now. Just a rough sketch. If I come up with more ideas/details, those will be added.

EDIT 1:
Don't think Everything should *write* tags. Thats the area of specialized software. Especially when document formats change (a new PDF layout, for example) and Everything isn't updated (yet), you might corrupt your tags, or worse: your document (I think; not sure).

EDIT 2:
(wild suggestion:) show the tags of a file in the preview window.
Last edited by NotNull on Wed Feb 28, 2018 8:25 pm, edited 5 times in total.
Stamimail
Posts: 1122
Joined: Sat Aug 31, 2013 9:05 pm

Re: File tags: yet another suggestion

Post by Stamimail »

I think the point of column headers is the ability to sort, isn't it?
Why "Tag1, tag2, .. tag5" column headers
are better than the style of status bar (when 1 selected)? it means only one header for tags:
Tags
width: 2000, height: 1000
year: 2018, artist: ArtistName, album: AlbumName

Excel screenshot?
NotNull
Posts: 5517
Joined: Wed May 24, 2017 9:22 pm

Re: File tags: yet another suggestion

Post by NotNull »

Yeah, maybe you are right. It seemed like a good idea when I thought about it (my idea). But now I can't remember why. :-S
I'll think about it a little longer ...
NotNull
Posts: 5517
Joined: Wed May 24, 2017 9:22 pm

Re: File tags: yet another suggestion

Post by NotNull »

What I wanted to prevent is that you have to choose from a lot of different tags/metadata for the column headers.
There are almost 100 different MP3's tags and over 300 EXIF tags (and that excludes manufacturer specific tags)
Imagine you have to select "year" out of a list of 500+ possible tags .... (beside EXIf and ID3 tags , there are also PDF, Office,.. tags)

But also: you don't want 300 EXIF tags crammed in one "cell" ..

You can't define the columns of the result list from your query. That's why I suggested a couple of (free format) TAG columns, so when yo search for year:<1970 artist:Pink Floyd, those tag columns will be filled automagically with year and artist for this query. Another query will give -for example- height and width in those columns.

Still not a very good idea, but so far it's the best I could come up with (I will keep thinking about it ..)
Stamimail
Posts: 1122
Joined: Sat Aug 31, 2013 9:05 pm

Re: File tags: yet another suggestion

Post by Stamimail »

You can't define the columns of the result list from your query.
This is nice idea.
Tags headers/columns will be added just for the current query, if the query contains tags.

For 2 tags in query, will be added 2 columns in result list.
For 3 tags in query, will be added 3 columns in result list.
and so on.
NotNull
Posts: 5517
Joined: Wed May 24, 2017 9:22 pm

Re: File tags: yet another suggestion

Post by NotNull »

That would be nice!
titain
Posts: 1
Joined: Mon Apr 16, 2018 7:29 am

Re: File tags: yet another suggestion

Post by titain »

https://tabbles.net/quick-intro/
It's good in File search tag,you can add tag by Drag and drop,but need to install SQL express,initializing is very slow ......

In fact,file tagging can be done by using File-List.
First, build a File-List by dropping files,then assign it in one Bookmark for search range. While clicking the bookmark(TAG),I can see or search the files with the "tag"!
But since the bookmark can only specify one file-list , file-list union or intersection (AND OR)can not be done.....

if "everything" will try to add TAG option,please try to implete "AND" "OR" function between TAGS.
Post Reply