Document side-effects of renderers' initialisation

Hello, this is possibly an issue concerning the doc and not the code.

• Parsing outside of the renderer’s context manager:

d = Document('a <b> c')
with HTMLRenderer() as r:
    print(r.render(d))  # <p>a &lt;b&gt; c</p>

• Parsing inside of the renderer’s context manager:

with HTMLRenderer() as r:
    d = Document('a <b> c')
    print(r.render(d))  # <p>a <b> c</p>

Not sure where the difference in output comes from. CommonMark asks for the second output though, which seems to be what is performed in mistletoe.markdown and by the mistletoe command line.

---

$ python -V
Python 3.7.0
$ pip freeze
mistletoe==0.7.1

[Rogdham]

I’m actually considering getting rid of this behavior, and thus removing the necessity of using context managers entirely. Would that be a better solution?

I guess so.

For what it’s worth, here is what I was expecting when I first encountered missletoe last week:

• Specify which tokens to be generated when doing the parsing phase: given that it seems to be done by Document, I would expect to write something like Document(md, extra_tokens=[GithubWiki])
• Then, the returned value could be rendered by different renderers without needing to parse the markdown text again
• At that point, it would not be needed to tell the renderer about extra_tokens: I would expect the renderer to just throw an exception if it doesn’t know about some token type

So something like this:

md = 'Hello, *world*!'
tokens = Document(md, extra_tokens=[GithubWiki])
print(ASTRenderer().render(tokens))
print(HTMLRenderer().render(tokens))

In other words, I would expect the parsing and the rendering phases to be independent. Since the types of tokens has to be specified in the parsing phase, it seems odds to me to specify it in the rendering phase instead.

Just my two cents. Obviously, you do what you want with the project, and what I mentioned above is likely to change the API and break backward compatibility.

I’ll also allow HTMLBlock and HTMLSpan to be tokenized by default in ASTRenderer.

I don’t really have an opinion on the matter, except that it should probably be documented if that’s not the case.

That make me think of the following: it may be a good thing to be able to specify some tokens not to be generated. For example, if I want to do CommonMark and nothing more, how would I disable tables and footnotes? I’m not sure if that’s really an important feature though.

[pbodnar]

Thanks for your suggestions. 😃

So I’ve tried to document the side effects as originally requested. See f5ea6d66f7084921fd403030f7faebe312935c8d. I hope this will suffice for now.

Regarding separating parsing and rendering phases, I hope we will get to this one day, but let’s do this in a separate ticket.