Make punctuation collapsing localisable

Punctuation in input data may collide with punctuation in style definitions. The citeprocs handle these punctuation clusters by applying hard-coded rules. While these rules have been chosen on a good basis and probably suffice for all English language documents and most others in Latin script, some style guides or non-English standards might require divergent collapsing.

I brought this up on the citeproc-js issue tracker for a German language style which requires suppression of subsequent , after titles ending in ! or ?, with the intention of a CSLm-extension: https://github.com/Juris-M/citeproc-js/issues/154. @denismaier suggested to discuss this here and also came up with possible solutions to make punctuation collapsing localisable on a per-style basis:

<punct-handling>
  <punct>
    <input value="?,"/>
    <output value="?"/>
  </punct>
</punct-handling>

Perhaps this is closer to how it looks in citeproc-js:

<punct-handling>
  <punct value="?">
    <next value="!" result="?!"/>
    <next value="." result="?"/>
    <next value=":" result="?"/>
    <next value="," result="?,"/>
    <next value=";" result="?;"/>
  </punct>
</punct-handling>

We can take the combinations from citeproc-js as a basis for here if we can agree upon a syntax. I think CSL 1.1 is a good target for this. But perhaps we can even add it to 1.0.2 as this just makes the current behaviour explicit and configurable.

[bdarcus]

Not commenting on the details, but do keep in mind that XML elements are ordered.

[bwiernik]

@cormacrelf the motivation is that some locales use different punctuation symbols entirely (e.g., Persian has its own semicolon symbol). Currently, supporting that requires a completely separate style, rather than just being able to localize the semicolon symbol.

It’s a fairly similar situation as in French where a colon is localized to narrow-no-break-space-colon.