Version 4 - Looking for Testers

Version 4 🚀

Over the past few months I’ve been thinking about this problems and how to create a new architecture that can resolve them. The new version of this library is heavily inspired by rsync. rsync is a popular application that must be run over a ssh connection, it requires the rsync binary be install on both the client and the sever. rsync provides dozens of advanced arguments that can’t be supported due to protocol limitations with ftp. Version 4 attempts to emulate at a high level how rsync syncs multiple folders, however operates over the constrained ftp protocol.

The goal of version 4 is to build a slim version of rsync. The complicated bit comes from working around the limitations of ftp, v1 and v2 taught me valuable lessons around ftp. The largest problem comes from diff based deployments, without them publishes must delete all server files, then re-deploy from a blank slate. This is trivial to program but causes deployments that can last several hours…

My solution with v4 is to build a .ftp-deploy-sync-state.json file. This file is a simple nested dir listing of every file/folder in the local branch. Files are hashed and then a diffing algorithm detects the necessary file changes needed on the server to make it match the client (upload, replace, delete, folderAdd, folderRemove) - note: renames are not currently supported for simplicity but could be added at a later date.

These diffs are then converted to ftp commands and sent to the server. A large amount of code exists around error handling to make sure the script is as forgiving as possible. Once the sync is complete the .ftp-deploy-sync-state.json file is uploaded to the server so we can calculate diffs on the next deployment

Main Goals 🏆

Decouple from git

Version 3 is tightly coupled with git source control. This tight coupling allowed for extremely fast publishes because only differences are synced to the server. Git history is used to track which version is currently deployed. In hindsight this was a mistake, tightly coupling to git resulted in complex configurations for basic use cases. Untracked files were deployed every time, which negated many of the performance benefits git provided. Using git also caused hard-to-debug issues like the fetch-depth problem that still comes up in non-standard use cases. Publishing a folder dynamically generated from build scripts becomes difficult and cumbersome.

Version 4 will no longer use git in any way. It will still run via github actions and be able to publish git tracked files. Because git is no longer used as the diffing solution .git-ftp-include and .git-ftp-ignore are no longer needed, the script will publish all files/folders (I will provide glob filter options).

Improved Defaults

.git and node_modules files should be ignored by default, allowing users to opt-in to their deployment instead.

Improved Error Handling

Errors were cryptic. Because this action is based around third parties adding error messages is difficult and requires third party PRs and a long delay in order to see those changes in this down-stream library. Error messages should tell users exactly what went wrong and how to fix it.

Local Testing

Testing actions today is difficult, often requiring devs to change a value and commit it without testing the change locally. Using docker makes testing actions a large amount of work. Allowing users to run this command from powershell/terminal with shorten the feedback loop

Minimize downtime

Maintaining this code myself gives me the opportunity to sync changes in a way that minimizes downtime. For example, new files can be uploaded, then replace updated files, and finally delete old files.

Removed features 🗑️

sftp support has been removed. In hindsight this never should have been added. Users with sftp access should be using rsync over ssh. I will create a separate github action for rsync deployments.

The important arguments have been migrated (--dry-run) but most git-ftp commands have been removed (--silent, --all, --lock, --key, --branch, --pubkey, --insecure, --cacert)

---

Open Issues Resolved in Version 4 ✅

• Removing git-based file tracking #101, #99, #88, #82, #78, #76, #68, #56, #52, #49, #40
• Local testing - #71
• Better defaults - #66, #57
• Better error handling - #48, #19
• Detect when hosts require ftp/ftps/ftpes and output helper text - related issues #87
• Other issues - #83
• better documentation/warning on escaping spaces/quotes - #80

Testers 🐞

I’ve been working on version 4 over the past several weeks. I feel comfortable enough at this point to bring on anyone in the community that would be willing to help test these changes. If you would like to help out please let me know (with a comment or direct message).

I’m also looking for someone with a host that requires secure ftp (ftps or ftpes) as my current hosting company does not require secure connections. If someone can provide me with a limited access ftp account to play around with, or knows a host that has this restriction so I can buy a account it would help.

[mikedelgaudio]

Just tried it out with my provider Bluehost (cPanel) using FTPS but I don’t believe they enforce FTPS. It did successfully upload but I discovered two issues trying to upload a compiled Angular project however, do you have any specific contribution guidelines?

[ghost]

My hoster requires ftps:// and I am willing to test version 4