Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Curious case of three large video files being dated differently from their metadata #1195

Closed
invario opened this issue May 17, 2024 · 5 comments
Closed

Curious case of three large video files being dated differently from their metadata #1195

invario opened this issue May 17, 2024 · 5 comments
Labels
bug Something isn't working
Milestone
7.4

Comments

@invario
Copy link

invario commented May 17, 2024 •
edited
Loading

Describe the bug
I recorded (3) large HEVC HDR10+ videos via my Samsung GS23+ today. Two of the files were completely untouched. One of the files I trimmed 16 seconds from the front end using my GS23+ included Gallery video editor. All three files were then synced/upped via Foldersync to my NC instance which then showed up in Memories out of order from when the video was created.

Using the package exiftool in the bin-ext folder, I pulled the time tags:
perl exiftool -api largefilesupport=1 -time:all 20240516_*.mp4

20240516_165405.mp4 - First video that was recorded and subsequently trimmed with my video editor.

File Modification Date/Time     : 2024:05:16 18:05:28-04:00
File Access Date/Time           : 2024:05:17 09:30:07-04:00
File Inode Change Date/Time     : 2024:05:16 23:31:18-04:00
Create Date                     : 2024:05:16 20:58:58
Modify Date                     : 2024:05:16 20:58:58
Track Create Date               : 2024:05:16 22:05:27
Track Modify Date               : 2024:05:16 22:05:27
Media Create Date               : 2024:05:16 22:05:27
Media Modify Date               : 2024:05:16 22:05:27

20240516_170511.mp4 - untouched.  Second video recorded:

File Modification Date/Time     : 2024:05:16 17:07:42-04:00
File Access Date/Time           : 2024:05:17 09:30:17-04:00
File Inode Change Date/Time     : 2024:05:16 23:33:04-04:00
Create Date                     : 2024:05:16 21:07:42
Modify Date                     : 2024:05:16 21:07:42
Track Create Date               : 2024:05:16 21:07:42
Track Modify Date               : 2024:05:16 21:07:42
Media Create Date               : 2024:05:16 21:07:42
Media Modify Date               : 2024:05:16 21:07:42

20240516_172115.mp4 - untouched. Third video recorded:

File Modification Date/Time     : 2024:05:16 17:25:47-04:00
File Access Date/Time           : 2024:05:17 09:30:08-04:00
File Inode Change Date/Time     : 2024:05:16 23:36:32-04:00
Create Date                     : 2024:05:16 21:25:46
Modify Date                     : 2024:05:16 21:25:46
Track Create Date               : 2024:05:16 21:25:47
Track Modify Date               : 2024:05:16 21:25:47
Media Create Date               : 2024:05:16 21:25:47
Media Modify Date               : 2024:05:16 21:25:47

In Memories, the videos are being shown out of chronological order per this screenshot.
image

  • It makes sense that 165405 is the newest video per the Track Create Date, due to my trimming the video. However, when I view the Info for this video, it shows this:

image
Note it says 10:05PM and that is most certainly not when I trimmed the video. It appears the timezone is not being factored in as I believe I trimmed the video at approximately 6:05PM local. (-4 hours)

  • 170511 (original, untouched) shows this correctly as 5:07PM

image

  • But oddly, 172115, which is also the original and untouched shows this incorrectly as 9:25PM when it should be 7:25PM (timezone?)

image

To Reproduce
Record videos on my phone. Uploaded to NC. Browse collection in Memories.

Platform:

  • OS: TrueNAS Core 13 in FreeBSD 13.2-RELEASE-p11 jail
  • Browser: Chrome Android and Windows
  • Memories Version: 7.3.1
  • Nextcloud Version: Nextcloud Hub 8 (29.0.0)
  • PHP Version: 8.2.14

Additional context
The files are rather large, FYI.

-rw-r--r--  1 www  www  2539766902 May 16 18:05 20240516_165405.mp4
-rw-r--r--  1 www  www  1410936134 May 16 17:07 20240516_170511.mp4
-rw-r--r--  1 www  www  2530825413 May 16 17:25 20240516_172115.mp4

  • Any errors in the JS console?
    No.

  • Any errors in the Nextcloud server logs?
    No.
@invario invario added the needs triage To be triaged label May 17, 2024
@pulsejet
Copy link
Owner

  1. Video files usually don't store the timezone. If only a timestamp (e.g. modified time) is present, the server's timezone is used. You might need to set the TZ environment variable if running in docker (i.e. your host settings may not be used)
  2. CreateDate always takes precedence. I'm actually not sure why 170511 shows the correct date. The only possible explanation is that this video has the timezone embedded while the other doesn't (the other place from where the timezone is extracted is the geolocation, but neither video has a tag). Something does seem different in the two files since exiftool could extract the dimensions from one but not the other.

@pulsejet
Copy link
Owner

I just realised large file support isn't enabled by Memories. This could be the reason why it doesn't work for the bigger file.

pulsejet added a commit that referenced this issue May 17, 2024
@pulsejet
exif: enable LargeFileSupport (#1195)
54d77a7 
Signed-off-by: Varun Patil <radialapps@gmail.com>
@pulsejet pulsejet removed the needs triage To be triaged label May 17, 2024
@invario
Copy link
Author

invario commented May 17, 2024 •
edited
Loading

  1. Video files usually don't store the timezone. If only a timestamp (e.g. modified time) is present, the server's timezone is used. You might need to set the TZ environment variable if running in docker (i.e. your host settings may not be used)

I'm in a FreeBSD iocage jail, no docker here.

  1. CreateDate always takes precedence. I'm actually not sure why 170511 shows the correct date. The only possible explanation is that this video has the timezone embedded while the other doesn't (the other place from where the timezone is extracted is the geolocation, but neither video has a tag). Something does seem different in the two files since exiftool could extract the dimensions from one but not the other.

Noted. Yeah, I disable geotags on my phone too.

I just realised large file support isn't enabled by Memories. This could be the reason why it doesn't work for the bigger file.

Indeed, this might be it! I reran my commands. So for comparison, here is the output with largefilesupport:

# perl ./exiftool -api largefilesupport=1 -time:all /folder/20240516_*.mp4
======== /folder/20240516_165405.mp4
File Modification Date/Time     : 2024:05:16 18:05:28-04:00
File Access Date/Time           : 2024:05:17 16:25:20-04:00
File Inode Change Date/Time     : 2024:05:16 23:31:18-04:00
Create Date                     : 2024:05:16 20:58:58
Modify Date                     : 2024:05:16 20:58:58
Track Create Date               : 2024:05:16 22:05:27
Track Modify Date               : 2024:05:16 22:05:27
Media Create Date               : 2024:05:16 22:05:27
Media Modify Date               : 2024:05:16 22:05:27
======== /folder/20240516_170511.mp4
File Modification Date/Time     : 2024:05:16 17:07:42-04:00
File Access Date/Time           : 2024:05:17 16:25:20-04:00
File Inode Change Date/Time     : 2024:05:16 23:33:04-04:00
Create Date                     : 2024:05:16 21:07:42
Modify Date                     : 2024:05:16 21:07:42
Track Create Date               : 2024:05:16 21:07:42
Track Modify Date               : 2024:05:16 21:07:42
Media Create Date               : 2024:05:16 21:07:42
Media Modify Date               : 2024:05:16 21:07:42
======== /folder/20240516_172115.mp4
File Modification Date/Time     : 2024:05:16 17:25:47-04:00
File Access Date/Time           : 2024:05:17 16:25:20-04:00
File Inode Change Date/Time     : 2024:05:16 23:36:32-04:00
Create Date                     : 2024:05:16 21:25:46
Modify Date                     : 2024:05:16 21:25:46
Track Create Date               : 2024:05:16 21:25:47
Track Modify Date               : 2024:05:16 21:25:47
Media Create Date               : 2024:05:16 21:25:47
Media Modify Date               : 2024:05:16 21:25:47
    3 image files read

And here it is without largefilesupport:

# perl ./exiftool -time:all /folder/20240516_*.mp4
======== /folder/20240516_165405.mp4
File Modification Date/Time     : 2024:05:16 18:05:28-04:00
File Access Date/Time           : 2024:05:17 16:25:31-04:00
File Inode Change Date/Time     : 2024:05:16 23:31:18-04:00
======== /folder/20240516_170511.mp4
File Modification Date/Time     : 2024:05:16 17:07:42-04:00
File Access Date/Time           : 2024:05:17 16:25:31-04:00
File Inode Change Date/Time     : 2024:05:16 23:33:04-04:00
Create Date                     : 2024:05:16 21:07:42
Modify Date                     : 2024:05:16 21:07:42
Track Create Date               : 2024:05:16 21:07:42
Track Modify Date               : 2024:05:16 21:07:42
Media Create Date               : 2024:05:16 21:07:42
Media Modify Date               : 2024:05:16 21:07:42
======== /folder/20240516_172115.mp4
File Modification Date/Time     : 2024:05:16 17:25:47-04:00
File Access Date/Time           : 2024:05:17 16:25:31-04:00
File Inode Change Date/Time     : 2024:05:16 23:36:32-04:00
    3 image files read

This makes sense now looking at the tags extracted with and without the flag, and makes sense being that 2 of the files are 2GB and one file (the one being processed correctly) is under 2GB, which seems like the limit without largefilesupport being enabled.

@invario
Copy link
Author

invario commented May 17, 2024 •
edited
Loading

RESOLVED by implementing your fix and forcing a reindex of the folder:

Videos are now in the correct order.
image

  • 165405 date/time good.

image

  • 170511 date/time good.

image

  • 172115 date/time good.

image

Thank you!

@invario invario closed this as completed May 17, 2024
@pulsejet
Copy link
Owner

Fantastic! Thanks for the quick test.

@pulsejet pulsejet added the bug Something isn't working label May 17, 2024
@pulsejet pulsejet added this to the 7.4 milestone May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
7.4
Development

No branches or pull requests
2 participants
@pulsejet @invario