Dallinger deploy from zip file: 5pt

The Dallinger CLI provides two pairs of functions that are theoretically rather powerful: hibernate/awaken and export/load. The documentation is as follows:

hibernate

Temporarily scales down the specified app to save money. All dynos are removed and so are many of the add-ons. Hibernating apps are non-functional. It is likely that the app will not be entirely free while hibernating. To restore the app use awaken. A required --app <app> parameter specifies the experiment by its id.

awaken

Restore a hibernating app. A required --app <app> parameter specifies the experiment by its id.

export

Download the database and partial server logs to a zipped folder within the data directory of the experimental folder. Databases are stored in CSV format. A required --app <app> parameter specifies the experiment by its id. Use the optional --local flag if exporting a local experiment data. An optional --no-scrub flag will stop the scrubbing of personally identifiable information in the export. The scrubbing of PII is enabled by default.

load

Import database state from an exported zip file and leave the server running until stopping the process with <control>-c. A required --app <app> parameter specifies the experiment by its id. An optional --verbose flag prints more detailed logs to the command line. Use the optional --replay flag to start the experiment locally in replay mode after loading the data into the local database.

These functions currently work for the current use cases:

• Finalising an experiment and exporting the resulting data (export).
• Freezing a Heroku-deployed experiment so that you can reinstantiate it later for debugging (hibernate/awaken).
• Loading an exported experiment from its exported zip file so you can explore the networks as if you were a participant (export/load).

However, they do not work for the following use case: taking a snapshot of an experiment, shutting down the deployment, and subsequently continuing the experiment by redeploying from the snapshot. As far as we can see, the main issue standing in the way is that dallinger awaken does not create a new recruiter, so no new participants will be recruited.

Clearly there are two ways that we could redeploy an experiment to collect more data:

• Redeploy by scaling up a hibernated Heroku app (similar to the awaken function)
• Redeploy from an exported zip file (similar to the load function).

In my opinion we should focus on the second route (zip file). I see no application except for debugging where the first route would be preferable. The advantages of the second route are that a) it’s easy to backup multiple zip files as snapshots, and b) it’s easy to edit the zip files.

Following this logic, we would therefore propose the following:

Add an optional argument to dallinger deploy that allows the user to specify a zip file to load the experiment from. This zip file should take the format as created by dallinger export. Applying this option means that the zip file will be used to populate the database, similar to dallinger load.

In order to prevent the same participant from taking the experiment twice, we should tell the experimenter to have a config.txt like this:

group_name = myexp
assign_qualifications = true
qualification_blacklist = myexp

If we’ve thought through this correctly, this should allows us to redeploy experiments from arbitrary snapshots, as desired.

[pmcharrison]

Thanks @jessesnyder , my bad, your advice fixed it!

[pmcharrison]

I did struggle to login to the dashboard after dallinger load, so it’s maybe worth checking that the dashboard still works after you implement this updated dallinger deploy (not sure whether you want to generate a new password or keep the old one)