find-dupes needs improvement!

Discussion related to "Everything" 1.5 Alpha.
Post Reply
anmac1789
Posts: 668
Joined: Mon Aug 24, 2020 1:16 pm

find-dupes needs improvement!

Post by anmac1789 »

There seems to be a weird aspect of finding dupes that doesn't quite makes sense to me. My syntax is this:

comp:"C:\Users\azeem\Downloads\xiaomi folder and file test\xiaomi mi max";"This PC\MI MAX\Internal shared storage";"Z:\run\user\1000\gvfs\mtp:host=Xiaomi_MI_MAX_c39daf56\Internal shared storage" folders: add-column:column1 column1:=regular-expression-match-1:,name:,formatfiletime($dm:)

The comp: filter is
regex:#quote:#regex-escape:<#element:<search:,;,1>>(\\.*$)#quote: | regex:#quote:#regex-escape:<#element:<search:,;,2>>(\\.*$)#quote: | regex:#quote:#regex-escape:<#element:<search:,;,3>>(\\.*$)#quote: -add-column:regexmatch1
Screenshot 2023-03-20 021453.jpg
Screenshot 2023-03-20 021453.jpg (168.97 KiB) Viewed 1866 times
Screenshot 2023-03-20 021559.jpg
Screenshot 2023-03-20 021559.jpg (76.04 KiB) Viewed 1866 times
Screenshot 2023-03-20 021636.jpg
Screenshot 2023-03-20 021636.jpg (68.11 KiB) Viewed 1866 times


After I do !find-dupes:column1

Screenshot 2023-03-20 021814.jpg
Screenshot 2023-03-20 021814.jpg (164.01 KiB) Viewed 1866 times
it only shows one instance of the duplicate when it should show all duplicates for that particular file
void
Developer
Posts: 16672
Joined: Fri Oct 16, 2009 11:31 pm

Re: find-dupes needs improvement!

Post by void »

!find-dupes:column1 will find unique items which is what you are seeing.

Please try the following search to find duplicates:
dupe:column1

Find Duplicates
anmac1789
Posts: 668
Joined: Mon Aug 24, 2020 1:16 pm

Re: find-dupes needs improvement!

Post by anmac1789 »

void wrote: Mon Mar 20, 2023 9:37 pm !find-dupes:column1 will find unique items which is what you are seeing.

Please try the following search to find duplicates:
dupe:column1

Find Duplicates
I think a better way would be if another column could calculate the number of intances of another column, so if column1 has 6 exact entries another column 2 could have 6 rows with the number "6" to identify how many dupes are found
anmac1789
Posts: 668
Joined: Mon Aug 24, 2020 1:16 pm

Re: find-dupes needs improvement!

Post by anmac1789 »

Seems like !find-dupes:column1 is broken because I have about 512,924 then waiting a few minutes the total results goes down to 512,920...when I look for a custom column duplicate, then isolate the filename and start another instance of everything and search for only this filename, then I found another duplicate that matches it's name, size, and date modified. Why is this occuring ? It seems like the duplicates are taking too long to "exit" the results. Any now just before pressing submit on this post the total number of results is ...512,918

This is also showing that in the debug console mode, there are a variety of processes occuring
void
Developer
Posts: 16672
Joined: Fri Oct 16, 2009 11:31 pm

Re: find-dupes needs improvement!

Post by void »

Formatting the filetime for each file will be really slow.

find-dupes: is not a real-time search.
You'll need to perform a requery (change the search) to get up-to-date results.
anmac1789
Posts: 668
Joined: Mon Aug 24, 2020 1:16 pm

Re: find-dupes needs improvement!

Post by anmac1789 »

void wrote: Wed Mar 22, 2023 11:29 pm Formatting the filetime for each file will be really slow.

find-dupes: is not a real-time search.
You'll need to perform a requery (change the search) to get up-to-date results.
What do you mean by that ? How can I get a real time upto date search result ?
void
Developer
Posts: 16672
Joined: Fri Oct 16, 2009 11:31 pm

Re: find-dupes needs improvement!

Post by void »

Everything will find duplicates once at the time of search.

The find duplicates command is not executed again for new, modified or deleted files/folders.
I will consider real-time support for finding duplicates.



To update your find duplicates command, change the search.
An easy way to do this is to add a space to the end of your search.

Don't expect good performance, formatting the filetime with assignable columns for half a million files will take a very long time.
anmac1789
Posts: 668
Joined: Mon Aug 24, 2020 1:16 pm

Re: find-dupes needs improvement!

Post by anmac1789 »

void wrote: Thu Mar 23, 2023 1:02 am Everything will find duplicates once at the time of search.

The find duplicates command is not executed again for new, modified or deleted files/folders.
I will consider real-time support for finding duplicates.



To update your find duplicates command, change the search.
An easy way to do this is to add a space to the end of your search.

Don't expect good performance, formatting the filetime with assignable columns for half a million files will take a very long time.
So everytime an index or folder is changed i have to re index the folder ?
void
Developer
Posts: 16672
Joined: Fri Oct 16, 2009 11:31 pm

Re: find-dupes needs improvement!

Post by void »

No, you don't need to re-index, only re-search.
anmac1789
Posts: 668
Joined: Mon Aug 24, 2020 1:16 pm

Re: find-dupes needs improvement!

Post by anmac1789 »

void wrote: Thu Mar 23, 2023 1:12 am No, you don't need to re-index, only re-search.
There seems to be an issue I don't quite understand. In the first attachment, you can see that there are no duplicates for that highlighted file even with !find-dupes:column.
Screenshot 2023-03-22 214632.jpg
Screenshot 2023-03-22 214632.jpg (188.42 KiB) Viewed 1748 times

But when searching for this specific filename in another window, there are duplicates which matches by name, size, date modified.

Screenshot 2023-03-22 220914.jpg
Screenshot 2023-03-22 220914.jpg (59.19 KiB) Viewed 1748 times
void
Developer
Posts: 16672
Joined: Fri Oct 16, 2009 11:31 pm

Re: find-dupes needs improvement!

Post by void »

!find-dupes:column1 will not find duplicates.

! = NOT

!find-dupes:column1 will find unique items. (not duplicated)

I'm probably misunderstanding something here, but it sounds like you want to search for duplicates?
Why are you using ! ?



Please try the following search to find duplicates:
find-dupes:column1
-or-
dupe:column1
anmac1789
Posts: 668
Joined: Mon Aug 24, 2020 1:16 pm

Re: find-dupes needs improvement!

Post by anmac1789 »

void wrote: Thu Mar 23, 2023 2:15 am !find-dupes:column1 will not find duplicates.

! = NOT

!find-dupes:column1 will find unique items. (not duplicated)

I'm probably misunderstanding something here, but it sounds like you want to search for duplicates?
Why are you using ! ?



Please try the following search to find duplicates:
find-dupes:column1
-or-
dupe:column1
Wait I think I found the problem, the problem was that in the first screenshot, in the search box I had a bunch of drive letters and in the 2nd screenshot I had just the filename, but if you look closer the duplicate was found in the second screenshot. So, I went back to the 1st screenshot and included the chinese phone and now, I have less results. What I am trying to do is to match duplicate files from the chinese phone and from all those USB's and external disk drives. I want to ! those duplicates and only see those files on the chinese phone that don't exist across eveywhere else
Post Reply