›

timestamps wrong for some photos

xray
in PhotoBackup for iOS

I have been a happy user since the issues of missing files were resolved from a previous thread. Thanks for the great app!

Recently, I noticed that some photos were uploaded with wrong timestamp. The photo was taken last year, however it showed a timestamp of several days ago around mid night on the NAS.

When I tried to manually select photos to sync from the options, I noticed that some photos (~ 20 out of 1000) are also shown with incorrect chronological order, consistent with the timestamp issue above.

I am pretty sure that this happened recently. I can confirm that the same photo was uploaded with correct timestamp before.

The only changes I can think of:

1) iOS was updated to latest 10.1.1

2) More photos were taken

3) PhotoBackup app was updated to latest 1.7

I suspect that the iOS update is more likely to be the cause? I'm happy to help debug again if necessary.


«1

Comments

  • gchen
    PhotoBackup uses the modification time of the photo as the timestamp.  The photos are also sorted by their modification times when they are presented in the manual selection mode.  So, I think that the modification times of those photos were changed for some reason, maybe by a photo editing app.

    If you have a Windows desktop, you can find out the modification times of photos easily by connecting your iphone/ipad and then browsing the device in the File Explorer.
  • xray
    edited December 2016

    Hmm..

    I checked the timestamps on a PC as you suggested. You are right, the "Created" time is as expected, but the "Modified" time is mysteriously recent. I am not using any photo editing apps on the iPhone other than the native photo app. I double checked that the photo in question was not modified in the native photo app. Really puzzling - no idea how the modification time was updated. Can you think of anything that may cause this?

    On a separate note, if PhotoBackup can preserve both modification and creation time, that'd be great. Otherwise maybe it is useful to have an option to toggle timestamp source between creation or modification time. In my case, I occasionally touch up the photo after a long time. It'd be nice to back it up on the NAS using creation time, because on the PC when I search / view a photo, the creation time is most relevant.

    Just a suggestion, not sure if you have time or if it makes sense for most users.

    Thanks!




  • xray
    edited December 2016

    PS. I checked all the iPhones in family running 10.1.1 - same issue with mysterious modification time. The one iPhone running ios 9 doesn't have the problem. Can't say for sure its iOS 10.1.1 causing this, but at least feels that way.. Any user have the same?

  • gchen
    I bet this is the same problem reported in https://discussions.apple.com/thread/7669261?start=105&tstart=0

    No, you can't preserve the creation times in the linux file system because there the creation time means the last modification time of the inode.  On mac os (and I suspect iOS too) the creation time is saved separately but there is no way to pass that to the linux file system.
  • xray
    edited December 2016
    ok. I guess currently PhotoBackup is using iOS modification time when passing it on to remote server. Would it possible to use creation time instead? Or add a toggle switch in options to choose between the two? It seems the photo modification time in iPhone changes somehow randomly - that every time I ran PhotoBackup it had to re-sync a bunch of files over because of timestamp change. It also replaced the correct timestamps of files on NAS with some random ones.  Very frustrating...  Any workaround you can think of?
  • gchen
    I just found out that my wife's iphone running iOS 10.1.1 has the same problem.  However, the changes seem to be pretty random and I can't reproduce it consistently.  Let me try to see if there is any workaround.  In the worst case I'll just add an option to use the create time as the modification time when uploading photos. 
  • xray
    edited December 2016

    Good to know it's not just me :) I was looking into it more closely - it seems that a lot of photos with updated modification times appear in the thumbnails from "Memories" and "Albums" sections within the iPhone Photos app. Those are new features for iOS 10 - could be related.

    In any case, I can imagine it would take Apple a long time to address this, if at all. Maybe the modification time change is legit in their view if those photos are later selected for album headings. Unless you can find a workaround, I guess the best can be done is to add a user option to use creation times?

  • xray
    Happy new year everyone!
    gchen - Did u get a chance to look at this further? Thanks!
  • gchen
    Sorry I haven't.  Did you try upgrading to iOS 10.2?  Some people said 10.2 didn't fix the timestamp problem (https://discussions.apple.com/thread/7669261?start=105&tstart=0) but I wonder if that is the case for PhotoBackup.

    I really want Apple to fix this problem because backing up files based on the creation times seems very dangerous to me (even if it is only provided as an option).
  • xray

    I just tried iOS 10.2, nothing changed. My feeling is this could be a different issue - people in that thread are complaining about camera roll out of order after restoring from backup. In our case, the camera roll is displayed in order on the phone. However some photos have updated modification times. Those photos seem to appear as header photos under "album" and "memories" sections in iOS 10. Apple probably say it is legitimate to update the modification times because those photos were selected for album headings. If that's the case, we may never get a "fix"...

    I agree this there's some danger of adding an option to use creation time. One scenario is if a photo has been backed up, and later edited, then it is possible that rsync thinks there's nothing new to upload based on creation time. However,  the file size would most likely change after editing, therefore triggering an upload again.

    Right now, without that option, PhotoBackup can not really function anymore. Every time it will re-upload a lot of pictures, and destroy the original timestamps long the way. It seems adding an option would do more good than harm?



  • gchen
    In my test, once a photo or a video has been marked as a favorite, its modification time will be changed.  The content of the file, including the exif metadata however, remains unchanged.  So maybe the solution is to find the modification time from the exif metadata and use that as the modification time of the file.

    I'll try that route first.
  • xray
    Yeah that's a good idea! What will happen with the Photobackup option set to backup the edited version? Will the exif metadata of the edited (rendered) photo of have an updated modification time?
  • xray
    Hi gchen - any update? I'm ready to help test if you get a beta version.. thanks!
  • gchen
    Sorry about the delay.  I haven't got a chance to work on this, as I have been busy wrapping up another project.  I should be able to release an update for this issue in two weeks.

  • xray
    Hi gchen - any chance of working on this yet? thanks!
  • gchen
    I apologize for postponing the fix.  I finally managed to have a working version -- what I did is to use the creation time, as you first suggested, but only when the option for backing up the original versions is selected.  This makes sense to me now, as the creation/modification time of the original version should remain unchanged regardless how many times the photo has been edited.

    If you want to test this fix, send email to photobackup@acrosync.com so I can add you to the beta user list.





  • xray
    Thanks! I do edit my photos quite often after taking them, using the built in editor. Would there be problems if this fix is also applied when "backup original version" is turned off?
  • gchen
    Unfortunately this fix has no effect if the 'backup original version' option is turned off.  I couldn't think of a good way to find the real modification date -- retrieving from jpeg exif data seems to be time-consuming, since it needs to read from the file contents not from the metadata.

    The new version 1.8 with this fix has already been approved by Apple and is now available from the App Store.
  • xray
    edited February 2017

    Would you consider using creation time as well when 'backup original version' option is turned off? or add another option for using creation time?

    In the context of this app, which is backing up iOS photos, I would argue that the most relevant timestamp is the creation time. Even in the case of legitimate editing afterwards, I would prefer knowing the original time when the picture was taken, rather than the modification time. Now there's a risk that rsync may skip a modified file since there's no creation timestamp change. However the file size would change for sure, making the issue very unlikely.

    It's unfortunate that iOS is messing with the modification timestamps. I couldn't use the app for my regular back up tasks now. Do you have any other ideas?


  • gchen
    I really don't want to replace the modification time with the recreation time as not uploading edited photos (with unchanged sizes) is a more serious bug.  The best approach I think is still to get the modification time from the exif data.  I'll try to go this route but please understand that my dev time on this app is limited so this may take a while.
  • xray
    OK I'll come back and check periodically. Thanks!
  • ml35
    edited March 2018
    Hi,
    Using IOS 10.2 and jailbreak. Using a test PNG photo that was a simple screenshot created using pressing power and home buttons and no other modifications on phone. No iCloud Photo Library enabled.

    Transferring using rsync from Media/DCIM/1XXAPPLE:
    - modify timestamps identical on phone and on remote linux

    Transferring using PhotoBackup
    - modify timestamps different on linux than on phone.

    *something* must be going wrong either between IOS and PhotoBackup, or between PhotoBackup and Linux. Can we have a debug version that writes some text debug log to see what it asked to IOS and what it received from IOS, as well as what did it talk to Linux?

    I can also provide remote ssh to my phone if that helps.

    On phone:
    # stat -c %y /User/Media/DCIM/109APPLE/IMG_9670.PNG
    2018-03-02 00:09:50.000000000 +0200


    On linux via rsync:
    # stat -c %y /tmp/IMG_9670.PNG
    2018-03-02 00:09:50.000000000 +0200


    On linux via PhotoBackup:
    stat -c %y /tmp/ph/IMG_9670.PNG
    2018-03-02 12:04:52.000000000 +0200

  • ml35
    > The best approach I think is still to get the modification time from the exif data.

    I'd pretty much want to preserve the file modification time and not from exif. Can that be an option so some users to choose one or another?
  • gchen
    The modification time of the photo file is just the creationDate field of the corresponding PHAsset object: https://developer.apple.com/documentation/photos/phasset.

    I don't know why creationDate is different from the last modified time, but can you take another screenshot and check if the last modified time is exactly the time the screenshot was taken?
  • ml35
    new screenshot.

    on phone

    iphone:/User/Media/DCIM/109APPLE root# stat --help|grep Time
      %w   Time of file birth, human-readable; - if unknown
      %W   Time of file birth, seconds since Epoch; 0 if unknown
      %x   Time of last access, human-readable
      %X   Time of last access, seconds since Epoch
      %y   Time of last modification, human-readable
      %Y   Time of last modification, seconds since Epoch
      %z   Time of last change, human-readable
      %Z   Time of last change, seconds since Epoch
    iphone:/User/Media/DCIM/109APPLE root# stat --printf=%w\\n%x\\n%y\\n%z\\n IMG_9716.PNG
    2018-03-06 22:08:41.000000000 +0200
    2018-03-06 22:08:41.000000000 +0200
    2018-03-06 22:08:41.000000000 +0200
    2018-03-06 22:08:41.000000000 +0200


    on linux, via `rsync -axPHAX`:

    root@srv:/tmp# stat /tmp/rsync/IMG_9716.PNG
      File: '/tmp/rsync/IMG_9716.PNG'
      Size: 130716          Blocks: 264        IO Block: 4096   regular file
    Device: 893h/2195d      Inode: 413239      Links: 1
    Access: (0644/-rw-r--r--)  Uid: (  501/ UNKNOWN)   Gid: (  501/ UNKNOWN)
    Access: 2018-03-06 23:50:07.347712888 +0200
    Modify: 2018-03-06 22:08:41.000000000 +0200
    Change: 2018-03-06 23:49:39.211762425 +0200
     Birth: -



    on linux, via PhotoBackup:

    root@srv:/tmp# stat /tmp/ph/IMG_9716.PNG
      File: '/tmp/ph/IMG_9716.PNG'
      Size: 130716          Blocks: 256        IO Block: 4096   regular file
    Device: 893h/2195d      Inode: 413092      Links: 1
    Access: (0444/-r--r--r--)  Uid: ( 1001/   user)   Gid: ( 1001/   user)
    Access: 2018-03-06 23:50:07.347712888 +0200
    Modify: 2018-03-06 22:18:18.000000000 +0200
    Change: 2018-03-06 23:48:14.775911088 +0200
     Birth: -

  • ml35
    photobackup options:
    ip of server
    username
    password
    /tmp/ph/

    select photos: about 50 files
    backup original: off
    skip: on
    create time machine: off
    port 22
    proto ssh
    key off

  • gchen
    If the backup original option is disabled, then the modificationDate field instead of creationDate is used as the modified time of the photo file.  Can you enable this option and try it again?
  • ml35
    look at the timestamps of the file in iphone. none of them are the same as the timestamps created by the app. there is no modification on the file on the iphone.
  • ml35
    look how an unedited photo looks like

    on phone
    Access: 2018-03-07 21:12:20.000000000 +0200
    Modify: 2018-03-07 21:12:20.000000000 +0200
    Change: 2018-03-07 21:12:20.000000000 +0200
     Birth: 2018-03-07 21:12:20.000000000 +0200

    on rsync
    exiftool | grep Date
    File Modification Date/Time     : 2018:03:07 21:12:20+02:00
    File Access Date/Time           : 2018:03:07 22:17:05+02:00
    File Inode Change Date/Time     : 2018:03:07 22:16:55+02:00
    Date Created                    : 2018:03:07 21:12:19

    on photobackup with backup original ON
    exiftool | grep Date
    File Modification Date/Time     : 2018:03:07 21:12:19+02:00
    File Access Date/Time           : 2018:03:07 22:17:53+02:00
    File Inode Change Date/Time     : 2018:03:07 22:17:42+02:00
    Date Created                    : 2018:03:07 21:12:19


    ------------------------------------------------------------------------------------------------

    same photo, edited (iphone saves the edited version as ~/Media/PhotoData/Mutations/DCIM/<FOLDER NAME>/<IMAGE NAME>/Adjustments/FullSizeRender.jpg)

    on phone, edited
    C:~ root# stat /User/Media/PhotoData/Mutations/DCIM/109APPLE/IMG_9731/Adjustments/FullSizeRender.jpg
      File: `/User/Media/PhotoData/Mutations/DCIM/109APPLE/IMG_9731/Adjustments/FullSizeRender.jpg'
      Size: 118264          Blocks: 232        IO Block: 4096   regular file
    Device: 1000003h/16777219d      Inode: 28574937    Links: 1
    Access: (0644/-rw-r--r--)  Uid: (  501/  mobile)   Gid: (    0/   wheel)
    Access: 2018-03-07 22:39:39.000000000 +0200
    Modify: 2018-03-07 22:39:39.000000000 +0200
    Change: 2018-03-07 22:39:39.000000000 +0200
     Birth: 2018-03-07 22:39:39.000000000 +0200

    on phone, original
    C:~ root# stat /User/Media/DCIM/109APPLE/IMG_9731.PNG
      File: `/User/Media/DCIM/109APPLE/IMG_9731.PNG'
      Size: 276082          Blocks: 544        IO Block: 4096   regular file
    Device: 1000003h/16777219d      Inode: 28572534    Links: 1
    Access: (0644/-rw-r--r--)  Uid: (  501/  mobile)   Gid: (  501/  mobile)
    Access: 2018-03-07 21:12:20.000000000 +0200
    Modify: 2018-03-07 21:12:20.000000000 +0200
    Change: 2018-03-07 21:12:20.000000000 +0200
     Birth: 2018-03-07 21:12:20.000000000 +0200

    on rsync, original
    root@gen8:~# rsync -avxPHAX -e 'ssh -o port=22222' 192.168.13.48:/User/Media/DCIM/109APPLE/IMG_9731.PNG /tmp/rsync/IMG_9731.PNG    receiving incremental file list

    sent 11 bytes  received 566 bytes  384.67 bytes/sec
    total size is 276,082  speedup is 478.48
    root@gen8:~# stat /tmp/rsync/IMG_9731.PNG
      File: '/tmp/rsync/IMG_9731.PNG'
      Size: 276082          Blocks: 552        IO Block: 4096   regular file
    Device: 893h/2195d      Inode: 409260      Links: 1
    Access: (0644/-rw-r--r--)  Uid: (  501/ UNKNOWN)   Gid: (  501/ UNKNOWN)
    Access: 2018-03-07 22:17:05.746164232 +0200
    Modify: 2018-03-07 21:12:20.000000000 +0200
    Change: 2018-03-07 22:16:55.318184851 +0200
     Birth: -

    rsync, edited
    root@gen8:~# rsync -avxPHAX -e 'ssh -o port=22222' 192.168.13.48:/User/Media/PhotoData/Mutations/DCIM/109APPLE/IMG_9731/Adjustments/FullSizeRender.jpg /tmp/rsync/FullSizeRender.jpg
    receiving incremental file list

    sent 11 bytes  received 61 bytes  48.00 bytes/sec
    total size is 118,264  speedup is 1,642.56

    root@gen8:~# stat /tmp/rsync/FullSizeRender.jpg
      File: '/tmp/rsync/FullSizeRender.jpg'
      Size: 118264          Blocks: 232        IO Block: 4096   regular file
    Device: 893h/2195d      Inode: 413248      Links: 1
    Access: (0644/-rw-r--r--)  Uid: (  501/ UNKNOWN)   Gid: (    0/    root)
    Access: 2018-03-07 22:44:24.722933087 +0200
    Modify: 2018-03-07 22:39:39.000000000 +0200
    Change: 2018-03-07 22:44:19.666943008 +0200
     Birth: -


    on photobackup, backup original ON
    root@gen8:~# exiftool /tmp/ph/IMG_9731.PNG |grep Date
    File Modification Date/Time     : 2018:03:07 21:12:19+02:00
    File Access Date/Time           : 2018:03:07 22:17:53+02:00
    File Inode Change Date/Time     : 2018:03:07 22:17:42+02:00
    Date Created                    : 2018:03:07 21:12:19



    photobackup backup original OFF
    root@gen8:~# exiftool /tmp/ph/IMG_9731.jpg |grep Date
    File Modification Date/Time     : 2018:03:07 22:39:39+02:00
    File Access Date/Time           : 2018:03:07 22:51:12+02:00
    File Inode Change Date/Time     : 2018:03:07 22:45:38+02:00
    Date/Time Original              : 2018:03:07 21:12:19
    Date Created                    : 2018:03:07 21:12:19

  • gchen
    I think PhotoBackup gets the timestamp right when Backup Original is on, right?  There maybe an offset of 1 second, but that is because we pass the --modify-window=1 option.
Sign In or Register to comment.