Progress: Migration from Parse to Gists

This issue will outline the steps I am going to take to move away from Parse.

Context

Parse is discontinuing its service on January 28th. As explained in #142, I am planning to use gists as storage instead. The linked issue explains the advantages that gists have.

astexplorer server

Using the GitHub API requires to send authentication tokens (either received via OAuth or otherwise) or similar sensitive information. For me, the easiest way to get this to work is to run a server for astexplorer. Having our own server has other advantages as well. For example, we could add support for parsers that are not implemented in JavaScript.
I will be paying for the server myself at first and will look for other financing options in the future.

How will code snippets be saved as gists?

At the beginning, all snippets will be saved as secret gists owned by the astexplorer user, that I created specifically for this. A gist will have up to three files:

• astexplorer.json: Containing configuration and settings, such as the parser that is used.
• code.js: The code that is parsed.
• transform.js: If a transform/tool is selected, this file contains the code of the transform.

Example: https://gist.github.com/astexplorer/28ae2bb52426fa497264674356d0341a

A snippet can then be accessed via a URL similar to

https://astexplorer.net/#/gist/28ae2bb52426fa497264674356d0341a/407c5e81320b3bf6a07117e191d4a45936d1f772

The first part is the hash of the gist, the second part the hash of the revision. I know the URLs are long and ugly. Maybe we can eventually offer to create short URLs to specific revisions.

Note: Everybody that has a URL to such as snippet will be able to add a new revision, just like it works now.

Can I save the snippets to my GitHub account?

Eventually yes. The first priority is to ensure that astexplorer continues to work. Being able to authenticate will be added later. The advantage of saving snippets to your account is that only you can add new revisions.

What happens to the existing code snippets?

Existing URLs and snippets will continue to work. After considering various options, I think the easiest solution is to export the existing data and simply store the exported files on the server and serve them from there.
Maybe the snippets will be migrated to gists eventually, but that’s not a priority.

Note: Existing snippets will be loaded “read-only”. Changes can only be made by forking the snippet (in which case it will be saved as gist). The UI will point that out.

Next steps

Phase 1

• Deploy server with gist API
• Deploy new site that loads existing snippets read only and stores new snippets in gists

Phase 2

• Export final Parse dataset
• Deploy new server version that can serve parse snippets from the exported files
• Deploy new site that loads parse snippets from server

Phase 3

• Add authentication via GitHub, allow snippets to be stored to authenticated user

---

Please let me know if you have any questions or suggestions!

[fkling]

Parse snippets are now served from the astexplorer server 🎉 .

Unfortunately it wasn’t possible to simply export a final dataset. I had to spin up a mongoDB and use the migration service Parse offered. The data transfer was really quick though. No new snippets have been created in Parse since the astexplorer update rolled out.

I don’t know when I’m going to implement the user authentication part (if someone wants to take stab at it, go for it!), probably not within the next two weeks. Last week was pretty exhausting, so I’m going to take it slow for a bit.

However, I will start writing down some ideas about what could be next for astexplorer.

Thank you for all the encouraging words and for using astexplorer!

[fkling]

@giuseppeg: I see. It’s not possible to create gists associated with your user account atm anyway. And even if it is possible, you don’t have to use that option. You can always create gists that are owned by the astexplorer user.

@skratchdot: That was the plan 😃 I was also considering adding versioning too, e.g. /#/v1/gist/, but not sure if that’s too much… what do you think?

Regarding donations: We will do that eventually. I don’t expect costs to be high for now. I first want to get everything working properly.