Some already downloaded files are downloading again on every pass...

First things first… Thanks so much @mandarons, this project is what I’ve looking for since years… I appreciate so much your efforts on this…

Describe the bug Some files are downloaded on every pass, even if the file already exists and no changes have occurred on it… Usually this issue is related to MacOS packages, which are downloaded as a file, instead of as a directory, but I have also noticed that it applies to many photos…

To Reproduce In my case, this behavior seems to be always the same, although I’m not sure you know which files it occurs with and which it doesn’t.

Expected behavior I expect not to download any file if it is already downloaded and it has not changed…

Screenshots You can see some examples on “icloud.log” where unmodified files are downloaded several times (PHOTOS sync_interval is lower than DRIVE one):

/app # more icloud.log |grep LOGO-Contacto
2022-02-26 17:02:56,650 :: INFO :: root :: sync_drive.py :: 90 :: Downloading /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm ...
2022-02-27 10:33:47,321 :: INFO :: root :: sync_drive.py :: 90 :: Downloading /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm ...
2022-02-28 04:38:14,972 :: INFO :: root :: sync_drive.py :: 90 :: Downloading /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm ...
/app # ls -la /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm 
----------    1 root     root        899758 Sep 21  2018 /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm

/app # more icloud.log |grep "Imagen 175.JPG"
2022-02-26 23:20:18,431 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-26 23:47:35,904 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 00:59:13,071 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 17:27:57,205 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 17:50:31,021 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 18:41:28,510 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-28 10:42:34,244 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-28 11:05:29,406 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-28 11:56:30,420 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
/app # ls -la "/app/icloud/photos/all/Imagen 175.JPG"
----------    1 root     root        735560 Mar 13  2011 /app/icloud/photos/all/Imagen 175.JPG

Configuration My config YAML is:

app:
  logger:
    level: "info"
    filename: "icloud.log"
  credentials:
    username: "MY-ID"
    retry_login_interval: 600
  root: "icloud"
  smtp:
drive:
  destination: "drive"
  remove_obsolete: true
  sync_interval: 3000
  filters:
    folders:
    file_extensions:
photos:
  destination: "photos"
  remove_obsolete: true
  sync_interval: 300
  filters:
    albums:
    file_sizes:
      - "original"

[mandarons]

@andressoler Thank you. I updated the main branch not to delete the zip file if extraction fails. This will keep the ball rolling. For some reason, package files created on iOS are zipped differently than package files created on Mac OS. I am seeking help on stackoverflow.com. Let’s see.

Thank you very much for your patience and co-operation, @andressoler, to solve this problem. I appreciate it.

[mandarons]

@andressoler : I am able to reproduce the issue now. For some reason, downloaded package files are always less than the one on server. I am looking into the fix now.