WebDAV Sync problems

Describe the bug

Sometimes the webdav sync seems to be broken. On some occasions floccus reported that everything was fine and synced, when in fact, nothing was fine. Trying a manual sync didn’t solve the issues either. Restarting the browser, checking network connections etc. didn’t solve the issue either.

To Reproduce

Steps to reproduce the behavior:

• Setup floccus sync with webdav backend
• Start using it
• Wait for the error (not syncing bookmarks, while telling everything is fine) to appear randomly
• OR: Wait for all bookmarks are messed up because of IDK; sync issues with a webdav backend?!
• Freak out because all bookmarks are messed up
• Remember that you have file versioning enabled, restore old version of bookmark sync, reset bookmarks, start syncing from zero

Expected behavior

Floccus should detect if there is a sync error and report it. This seems either a sync issue or the algo doesn’t properly detect if there is a sync error.

Desktop

• OS: Linux, Windows, Mac (all the same)
• Browser chrome and firefox
• Browser Version: various
• Floccus version: 4.7; but the problem existis since I started using floccus last year.
• Floccus sync method: webdav, nextcloud bookmark app couldn’t handle the amount of bookmarks

Server

(please complete the following information)

• OS: Debian 10
• Nextcloud version: [if applicable]
• Seafile CE WebDAV : 7 and 8; but this should not be relevant I think since it works with other Seafile servers running the same versions and config
• Bookmarks app version: [if applicable]

Note: Please write down the actual version numbers instead of writing ‘latest’.

Debug log

Sometimes the following error appears once one starts trying to force a sync (dangerous actions)

2021-09-13T11:13:30.839Z Starting sync process for account USER@DOMAIN@URL
2021-09-13T11:13:30.878Z onSyncStart: begin
2021-09-13T11:13:30.878Z https://URL/webdav/floccus_bookmarks/Admin/bookmarks.xbel.lock
2021-09-13T11:13:31.105Z https://URL/webdav/floccus_bookmarks/Admin/bookmarks.xbel.lock
2021-09-13T11:13:31.177Z Syncing failed with E019: HTTP status 404. Failed PUT request. Check your server configuration and log.
2021-09-13T11:13:31.180Z onSyncFail

FYI: Seafile release a patch to fix “[fix] Fix WebDAV error if a file is moved immediately after uploading”. This may be related to the problem. But still I would expect floccus to detect sync errors.

Apart from that it showed that it tried setting/getting the .lock file and nothing more.

Additional context

I hope that there is room for improvement with the webdav sync algo and to detect if there are sync issues. Maybe with some kind of checksum file stored in the server where the bookmarks file is actually stored. The lock file makes sense but sometimes it seems that floccus cannot remove it, whereas other webdav applications can just edit the file manually and it just works.

Also this error sometimes messed up all bookmarks, maybe because floccus actually thinks that everything is fine, when in fact it is not.

Thanks!

[dmotte]

I think I have found the cause of this problem.

My setup: NGINX as WebDAV server and Floccus on Chrome.

Looking at the NGINX logs, I noticed that sometimes the server response status is 304 Not Modified even if the file changed:

::1 - myuser [26/Jul/2022:15:52:01 +0200] "GET /bookmarks.xbel.lock HTTP/1.1" 404 197 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
::1 - myuser [26/Jul/2022:15:52:01 +0200] "PUT /bookmarks.xbel.lock HTTP/1.1" 201 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
::1 - myuser [26/Jul/2022:15:52:01 +0200] "GET /bookmarks.xbel HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
::1 - myuser [26/Jul/2022:15:52:01 +0200] "DELETE /bookmarks.xbel.lock HTTP/1.1" 204 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"

So I tried to edit my NGINX configuration to prevent 304 responses as a workaround. This is my location / config section:

location / {
    dav_methods PUT DELETE MKCOL COPY MOVE;
    dav_ext_methods PROPFIND OPTIONS;
    dav_access user:rw group:rw all:rw;

    client_max_body_size 0;
    create_full_put_path on;
    client_body_temp_path /tmp/;

    auth_basic "Restricted";
    auth_basic_user_file /etc/nginx/.htpasswd;

    if_modified_since off;
    add_header Last-Modified "";
    expires -1;
    add_header Cache-Control no-store;
}

Notice the last 4 lines. Now everything is working! 😃

BTW I think that this problem should be handled client-side by Floccus, maybe by setting the Pragma: no-cache and Cache-Control: no-cache headers for every GET request (see https://stackoverflow.com/a/29246795). These are the headers set by Chrome when the user presses CTRL+F5 to reload a web page. In this way the server won’t reply 304 again because we are explicitly requesting to retransmit the whole bookmarks.xbel file content.

I think that a possible fix would be adding the noted headers to the downloadFileWeb and downloadFileNative functions of the file src/lib/adapters/WebDav.ts file. Let me know what you think 😉

[DerDanilo]

Thanks for your time to check this in detail. However I don’t have the time currently to do all those tests. It may take several month until I can post more information. I saw some other posts that reported issues with webdav that looked quiet similar, maybe some of those may already point into the right direction.

Hint: To what I can say already most problems seem to have to do with the .lock file not being written/read/deleted correctly when using webdav.

I’ll post more info once I find time. Thanks for your patience in advance!