[Proposal] Move to CommonMark via Markdown-it instead of Showdown

“Markdown is the heart of Ghost, and as such we intend to make working with it one of our core competencies.” — Ghost Wiki: “Future of Markdown”

Ghost wanted a spec. CommonMark is it.

If markup is one of Ghost’s core competencies, then that markup must be rigorously specified. Markdown is notoriously ambiguous and poorly defined. Without a well-defined spec, other tools can’t reliably integrate with Ghost. The Ghost wiki explicitly calls out the need for a spec, stating that “the goal is to create something like: https://github.com/mustache/spec”

Fortunately, in the year since that page was last edited, the CommonMark specification has emerged, matured, and stabilized. Much like HTML5, CommonMark made compatibility with existing documents an explicit goal, and wrote its spec with an awareness of common behaviors in existing Markdown parsers. The resulting spec represents three years of work and fits Ghost’s needs. it would be foolish to duplicate that effort or further fragment basic Markdown syntax.

Embracing CommonMark gives Ghost a well-defined foundation with a rigorous, automated test suite. Ghost-specific extensions could be concisely defined in relation to CommonMark’s baseline.

Markdown-it is the right library.

The two major JavaScript implementations of CommonMark are the reference implementation and Markdown-it. Markdown-it is the better choice for Ghost:

• It’s been actively maintained by a pair of developers for over a year.
• It can optionally be configured for strict CommonMark compliance.
• It has a stable, plugin-based API for syntax extensions.

Plus, Markdown-it is fast and already ships with plugins for things like GFM-style tables and strikethrough, among others.

This solves real problems.

In addition to resolving ambiguities in Ghost’s markup, switching to CommonMark via Markdown-it would make it easy to address:

• Smart typography (em-dashes, curly quotes, etc.): #534, #1795, #4649, UV#7196701
• Table support: #1670, #4777, UV#7312035
• Dialect confusion / ambiguity / extensions: #1642, #2675, #5077, UV#7675026
• Specific requests for CommonMark: #3959, UV#7196681

This would also go a long way in alleviating problems with maintainability:

• All of the problems in #2381, “Make the showdown extensions make sense”
• “Ultimately this is a problem with the showdown library rather than with Ghost. […] Showdown is unfortunately quite buggy.” — @ErisDS on #2675
• “This is one of the many features that would be an absolute nightmare to crowbar into the existing showdown markdown conversion system.” — @ErisDS on #4649.

Thanks to Markdown-it’s API, it’s easy to create small, clean, and independent plugins for Ghost as needed, especially compared to maintaining an entire fork of Showdown.

Ghost can get there incrementally.

Everything is in place for a graceful transition:

• Rendered HTML gets stored in the database, so existing posts are safe by default.
• Showdown and Markdown-it can both exist in package.json at the same time.
• Markdown-it can live behind a Labs setting for as long as needed.

If Ghost commits to moving to CommonMark + extensions, it would have to figure out what to do when users edit posts that predate the CommonMark transition. By order of increasing effort, Ghost could:

1. Show a generic warning on old posts.
2. Produce a rich diff by running old posts through both Markdown-it and Showdown, or just comparing Markdown-it’s output to the HTML in the database.
3. Offer a “quirks mode” that sends posts through Showdown instead of Markdown-it.
4. Offer a “quirks mode” that uses custom Markdown-it extensions to mimic the idiosyncrasies of Showdown.

In all cases, the end goal would be to eventually drop support for Showdown’s quirks. Ghost Foundation employees could run the entire Ghost(Pro) corpus through both Markdown-it and Showdown to measure and mitigate any potential breakage. Similarly, someone could run the CommonMark test suite through Showdown to help understand fundamental differences between the two engines.

Again, since rendered HTML is in the database, there would be no risk to existing, published posts.

TL;DR:

I really want curly quotes.

[7k50]

Markdown table support would be fantastic.

[sandorfr]

Thanks for the clarification @kevinansfield 😃