Wrong Modifed Date

[therube]

Wrong Modifed Date/Time

Everything shows Modified Date:
09/13/2016 08:40 AM

Actual is:
06/08/2016 08:48:24 AM

Size is correct:
1,133,824,156

Win7 x64
Everything 741b x86

If I enable display of Created & Accessed in Everything, they display correctly, both being:
09/13/2016 ‏‎08:27:57 AM

I do index Date Modified, but not the others.


And I'm finding others that are wrong too.
The ones I've caught seem to have the Created/Accessed date rather then the Modified.
(The times, I'm not coinciding to anything particularly.)

I'm checking Date/Times against Properties, Salamander (file manager) & Everything.

[void]

Thanks for the bug report, I haven't seen any incorrect dates yet...

They look like valid dates... (so it doesn't look like data corruption)
Probably unlikely, but, any third party apps setting the date modified for files?

If I enable display of Created & Accessed in Everything, they display correctly, both being:
09/13/2016 ‏‎08:27:57 AM

Everything would be using file information cache to display these dates (which is different to how the modified date is gathered).

If the creation date is 09/13/2016 ‏‎08:27:57 AM, then I would assume this file was recently recreated?
The newly created file would have the same modified date 09/13/2016 ‏‎08:27:57 AM, so some app must be setting the new date modified time which Everything is missing.. Any idea what app might be recreating this file?

When I unzip (winrar) files, I see the modified date set to the same as the creation date, and then shortly after (almost instantly) Everything will detect the modified date being changed and Everything will update correctly.

Will do some testing with tools to change the modified date of files...

[void]

Everything is picking up changes to modified dates with the power shell script:
$(Get-Item foo.txt).lastwritetime=$(Get-Date "24/11/2015 06:00 am")

I'll keep watching.

Are the files with wrong dates always large files? I wonder if its a fragmented record issue..
If possible, can you modify the files with the wrong dates and see if Everything updates correctly? -if not, try copying the file and see if the date modified times are copied.

[therube]

I was thinking...

- file size
- moving the file from 1 drive to another

But I wasn't able to purposely duplicate that.

Now the particular file in question came down through my download manager, & my dm is set to set the file's (modified) date to that of the source on the server (assuming it can).

Likewise, I have a "SETTIME.BAT" batch file that I'll use to set a files (modified) date to that of some other source files date.

Likewise, my file manager has the option to preserve the date/time of copied directories.


Now possible that Everything is picking up the modified date that split (part of) a second before it changes, but I don't recall seeing that happen with earlier versions of Everything.


(As I was looking, I did happen to notice, & as expected, that when copying a file [copying a big file & checking its Properties as the file copy progresses], the file modified date/time is set to current until the copy operation is completed, at which point the modified date/time is set to the same as source.)

always large files?

That was my initial thought, & the ones I happened to notice were - but that is also because that was my initial thought (;-)) & I was specifically looking for big files with wrong dates.

I wonder if its a fragmented record

Also possible. Download Manager has the option to use multiple threads for downloads (writes different parts of the file to disk simultaneously). Most typically I only use 1 [if the speed is sufficient, really no advantage to more], but fairly recently, at times I've set it to use 2 treads as I've hit some particularly doggy servers & sometimes that helps.

---

Circa 1995:

Code: Select all

SAMEDATE -- V1.2 Public Domain Utility
David Kirschbaum, Toad Hall
SAMEDATE -- call error, calling sequence is:

       SAMEDATE file1 file2

Action is to give file2 the same date as file1

And my rendition:

Code: Select all

@ECHO OFF

ECHO  SETTIME  sets a file's date to the "same" file, named "abc"
ECHO  a sort of more current variation of my old ancient DOS settime program
ECHO.
ECHO  therube 02/20/2015
ECHO.
ECHO  File to change the date of:  %1

FOR   %%i in (%1) do  SET   BASENAME=%%~dpiabc
ECHO  reference file (aka "abc"):  "%BASENAME%"
ECHO.
ls -l %1
ls -l "%BASENAME%"
ECHO.

PAUSE

touch %1 --reference="%BASENAME%"
ls -l %1

PAUSE
EXIT

[void]

I've been able to reproduce the issue..

After weeks of getting no where with this bug, I think I've finally figured it out...

The date modified was not updating when an alternate data stream was modified.
I noticed dates on my eml (email) files were wrong with a tool I wrote to double check dates.
Turns out my email client writes when the email was "read" and when it was "replied to" in alternate data streams.

It should be fixed in Everything 1.4.1.762b.

If you would like to try this now, please try Everything 1.4.1.762b:
http://www.voidtools.com/Everything-1.4.1.762b.x64.zip
http://www.voidtools.com/Everything-1.4.1.762b.x86.zip

Other notes: system32 files are not redirected in Everything. However, the properties window, when launched from Everything is redirected. It's not an issue with Everything x64.
https://msdn.microsoft.com/en-us/librar ... 85%29.aspx

Another note: There are not always USN Journal events immediately after data is changed in a file, so checking properties may show a different size or date to what Everything displays. Once the file handle is closed Everything will update correctly (places.sqlite with firefox is a good example).

[therube]

The date modified was not updating when an alternate data stream was modified.

Explain "modified"?
Is "modified" different from "existing"?

As in when I look at specific files where Everything shows an incorrect modified date, does that mean that the file should have an ADS too?

Checking with AlternateStreamView / Streams, I'm not seeing that to be the case. I'm not seeing any ADS on the particular files.

Or does modified mean the the "value" of the ADS was changed from what it once was?
Or maybe there was an ADS initially, but since deleted (somehow, but not by my doing)?
(Wonder if download manager has option regarding ADS? I know my file manager has an option to not copy ADS.)

In any case, on particular files I'm not seeing ADS.

[therube]

There are not always USN Journal events immediately after data is changed in a file, so checking properties may show a different size or date to what Everything displays. Once the file handle is closed Everything will update correctly (places.sqlite with firefox is a good example).

places.sqlite, while the file size may change, is generally relatively "static", starting out at a minimal size of ~10 MB & then perhaps "growing" in increments thereof.

Also data may be cached in memory or "-wal" & "-shm" files until written back to disk, so what you might "see" both on disk or otherwise may not necessarily be "current" at a particular point in time. (Exit & all changes & writes are written out, sure.)
Temporary Files Used By SQLite

[therube]

Wonder if download manager has option regarding ADS?

Not seeing anything in that respect.
Get file time from server, Yes.
Keep downloading if the sites does NOT support resume, Yes.
Reserve space on disk for entire file, No.

I know my file manager has an option to not copy ADS.

Ignore alternate data streams (ADS), Yes [generally].

[therube]

ffmpeg-latest-win32-static.7z 03-Oct-2016 23:00 13M, [downloaded through my download manager]

Everything (current time):

Code: Select all

C:\WLIB\LeechFTP\LEECHTMP\ffmpeg-latest-win32-static.7z   10/04/2016 07:56 PM

Disk:

Code: Select all

10/04/2016 12:00:12 AM

If I then rename or move the file, Everything will pick up the correct time, 12:00 AM.


Yet I do the same with ffmpeg-latest-win32-shared.7z 03-Oct-2016 23:10 11M, & that comes through correctly?

[therube]

SandboxieInstall-514.exe SHA1 b3a73a2e567de9fab2e26516cf2147413b0ece7d [downloaded through my download manager]

Everything (current date & time, +/- DST as shown on the board):

Code: Select all

C:\WLIB\LeechFTP\LEECHTMP\SandboxieInstall-514.exe   10/04/2016 08:10 PM

Disk:

Code: Select all

09/26/2016 04:41:11 PM

If I then rename or move the file, Everything will pick up the correct date & time.


Started out as, & with incorrect date/time:
C:\WLIB\LeechFTP\LEECHTMP\SandboxieInstall-514.exe


Then renamed to, with Everything correctly showing date & time:
C:\WLIB\LeechFTP\LEECHTMP\SandboxieInstall-514.exe.RENAMED

Code: Select all

Everything
Version 1.4.1.741b (x86)
Windows NT 6.1
Processors 4
IsAdmin 0
AppData 0
Service 1
cmdline "C:\DEV\Locate\Everything.exe"
WM_ACTIVATE 00000000 00000000, lastfocus 003c0024, current focus 003c0024
WM_ACTIVATE 00000001 00000000, lastfocus 003c0024, current focus 00000000
search 'sandboxie 514 exe' filter '' sort 0
SET SORT 0
found 0 folders, size 0, db search time taken: 0.011615 seconds
found 2 files, size 65540, db search time taken: 0.030359 seconds
total size 9019894, calculated in 0.000001 seconds
set sort 0 1
already sorted
finished sort, time taken 0.000442 seconds
update selection 0.000000 seconds
DB_WAIT: _db_ready_proc waiting...
DB_WAIT: _db_ready_proc waited 0.000123 seconds
new results
update m 1 92621072
_db_monitor_ntfs_update_event_proc 0
start _db_monitor_ntfs_update_thread_proc
update index E:
updated E: in 0.000442 seconds
resume ntfs monitor 1
_db_monitor_ntfs_process_usn_records_thread_proc
processed ntfs usn records in 0.000794 seconds
DB_WAIT: _db_ready_proc waiting...
DB_WAIT: _db_ready_proc waited 0.000299 seconds
WM_ACTIVATE 00000000 00000000, lastfocus 003c0024, current focus 003c0024
update m 1 92621072
_db_monitor_ntfs_update_event_proc 0
start _db_monitor_ntfs_update_thread_proc
update index E:
updated E: in 0.000377 seconds
resume ntfs monitor 1
_db_monitor_ntfs_process_usn_records_thread_proc
processed ntfs usn records in 0.002340 seconds
DB_WAIT: _db_ready_proc waiting...
DB_WAIT: _db_ready_proc waited 0.000483 seconds
update m 0 92620888
_db_monitor_ntfs_update_event_proc 0
start _db_monitor_ntfs_update_thread_proc
update index C:
updated C: in 0.000372 seconds
resume ntfs monitor 0
_db_monitor_ntfs_process_usn_records_thread_proc
processed ntfs usn records in 0.001235 seconds
DB_WAIT: _db_ready_proc waiting...
DB_WAIT: _db_ready_proc waited 0.000421 seconds
new results
update m 1 92621072
_db_monitor_ntfs_update_event_proc 0
start _db_monitor_ntfs_update_thread_proc
update index E:
updated E: in 0.000261 seconds
resume ntfs monitor 1
_db_monitor_ntfs_process_usn_records_thread_proc
processed ntfs usn records in 0.001282 seconds
DB_WAIT: _db_ready_proc waiting...
DB_WAIT: _db_ready_proc waited 0.000158 seconds
WM_ACTIVATE 00000001 00000000, lastfocus 003c0024, current focus 00000000
WM_ACTIVATE 00000000 00000000, lastfocus 003c0024, current focus 003c0024

(And as you can see, I have NOT yet tried 762b.)

[therube]

Tried it again, with -verbose, & this time the file date/time ended up correct.

So 3 times I downloaded Sandboxie, first, probably on 9-26, then twice today, & two out of the three times Everything had the wrong modified date.

http://pastebin.com/MRuk9y7A

[void]

Explain "modified"?
Is "modified" different from "existing"?

Any change to the alternate data stream, which includes:
USN_REASON_STREAM_CHANGE (new or deleted streams)
USN_REASON_NAMED_DATA_OVERWRITE (overwrite data)
USN_REASON_NAMED_DATA_EXTEND (write more data)
USN_REASON_NAMED_DATA_TRUNCATION (set new file size)

As in when I look at specific files where Everything shows an incorrect modified date, does that mean that the file should have an ADS too?

Not necessarily, Everything now also detects changes to a files security objects (which appears to be common my end) and all other file changes that may effect the files date modified. It is possible a data stream could have been created and deleted straight away, however, unlikely.

places.sqlite, while the file size may change, is generally relatively "static", starting out at a minimal size of ~10 MB & then perhaps "growing" in increments thereof.

I see the same thing, there must be data overwritten as the date modified is always changing in the file properties. The USN event is only triggered when this file is closed (the USN event usually being USN_REASON_NAMED_DATA_OVERWRITE from what I've seen). It could be because of a cache setting, or memory mapped object etc, nothing I can really do to efficiently make Everything keep the date modified up-to-date in realtime.

Wonder if download manager has option regarding ADS?


Not seeing anything in that respect.
Get file time from server, Yes.
Keep downloading if the sites does NOT support resume, Yes.
Reserve space on disk for entire file, No.

I know my file manager has an option to not copy ADS.


Ignore alternate data streams (ADS), Yes [generally].

Please try running Everything 762b in verbose debug mode, it will show you all the USN events.
Everything 761b and earlier ignored most "unknown" USN events.

Yet I do the same with ffmpeg-latest-win32-shared.7z 03-Oct-2016 23:10 11M, & that comes through correctly?

Yes, it's odd with 761b and earlier, some changes are detected, some not.
Everything 762b should detect all changes.

If I then rename or move the file, Everything will pick up the correct date & time.

A rename, copy or modifying the file contents will force Everything to rescan the up-to-date date modified.


Thanks for the debug logs, however I think its just pre-761 not processing "unknown/other" USN events.
pre-761 completely ignores these USN events, they are not even logged.

[therube]

762b & at least on [strike]one[/strike] three files (all the time I've got at the moment) & the date is still wrong? [Sorted by date & checked against disk made it easy to find a couple others.]

(Did a Force Rebuild after putting in 762b, & it looked like all existing was OK after that.)

[therube]

Above was on Win7.

I've caught it once on XP, with http://www.7-zip.org/a/7z1604.exe, but have been unable to duplicate it again with that file or any others I have downloaded this morning.


I did note that 7z1604.exe showed as content-type octet/stream, if that might have any bearing?

[void]

I've caught it once on XP, with http://www.7-zip.org/a/7z1604.exe, but have been unable to duplicate it again with that file or any others I have downloaded this morning.

If you see it again, please try running Process Explorer and searching for any open file handles. If possible, try closing them in Process Explorer and see if Everything updates.

I did note that 7z1604.exe showed as content-type octet/stream, if that might have any bearing?

This is the normal content-type for executables.