Wagtail Draftail editor moves ending whitespace at the boundary of overlapping or composite HTML tags outside the tag

Issue Summary

Italicized words inside a boldfaced string is one instance. But the following is a general issue. For example, what I want is this: “you may not believe what…”. Wagtail changes <b>may </b><b><i>not</i></b> to <b>may</b> <b><i>not</i></b> while saving (update: it does it while fetching and not saving) the page to the database. The spaceless templatetag (from Django) removes spaces between the tags, and I rely heavily on it. So, in the end there is no space between may and not. The first time such text is added to the editor, the published text looks good. But this issue creeps in if I edit the page and republish it.

Steps to Reproduce

1. Install a fresh wagtail site.
2. Have this in the home app’s models.py file:

from django.db import models
from wagtail.core.models import Page
from wagtail.core.fields import RichTextField
from wagtail.admin.edit_handlers import FieldPanel

class HomePage(Page):
    body = RichTextField()

    content_panels = Page.content_panels + [
        FieldPanel('body', classname="full"),
    ]

Run makemigrations and migrate.

1. In base.html of the root app, surround the html with:

{% spaceless %}
...
{% endspaceless %}

1. In home app’s home_page.html, have this:

{% block content %}
<div>
    {{page.body|richtext}}
</div>
{% endblock content %}

1. Now run the app using runserver, and go to Wagtail admin interface. Add a new page to the Home app, and something like this in the rich text editor: You may not believe what is happening! Publish the page and observe the text - it will look good.

2. Edit the page again, do not make any changes, and publish again. The string will change, and there will be no space between may and not.

• I have confirmed that this issue can be reproduced as described on a fresh Wagtail project: yes

Technical details

• Python version: 3.8.5
• Django version: 3.1.5
• Wagtail version: 2.11.3
• Browser version: Chrome 87

[manisar2]

For anyone interested in this issue, I have worked around by stopping using {% spaceless %}. It should be possible to use it around non-rich-text elements, but I chose to simply not use it anywhere for the time being.

I’m now using django.middleware.gzip.GZipMiddleware which is supported by all major browsers. It does not strip spaces (it is lossless), but otherwise provides very good compression ratio.

[thibaudcolas]

Spending a bit more time with this, I see that there does seem to be a conversion bug as you describe, even when the markup initially comes from Wagtail.

Authoring this in the rich text editor:

**bold _bolditalic_**

Draftail serializes that as what I’d expect:

{
  "blocks": [
    {
      "text": "bold bolditalic",
      "inlineStyleRanges": [
        { "offset": 0, "length": 15, "style": "BOLD" },
        { "offset": 5, "length": 10, "style": "ITALIC" }
      ],
    }
  ],
  "entityMap": {}
}

The contentstate -> HTML conversion produces an unexpected sequence of tags (see also https://github.com/springload/draftjs_exporter/issues/136), but the whitespace is still within the bold formatting:

<p><b>bold </b><b><i>bolditalic</i></b></p>

Then re-editing this in the CMS, HTML -> ContentState moves the whitespace outside of the bold style:

{
    "blocks": [
        {
            "text": "bold bolditalic",
            "inlineStyleRanges": [
                {
                    "offset": 0,
                    // This should be 5.
                    "length": 4,
                    "style": "BOLD"
                },
                {
                    "offset": 5,
                    "length": 10,
                    "style": "BOLD"
                },
                {
                    "offset": 5,
                    "length": 10,
                    "style": "ITALIC"
                }
            ],
        }
    ],
    "entityMap": {}
}

Saving this again, we get the incorrect but expected:

<p><b>bold</b> <b><i>bolditalic</i></b></p>

I see this also happens for juxtaposed links (so I’d assume all inline entities or styles) – again the trailing whitespace moves out of the first tag and goes in-between.

Link content conversions

After authoring [link ](/)[link](/) in the editor:

{
  "blocks": [
    {
      "text": "link link",
      "entityRanges": [
        { "offset": 0, "length": 5, "key": 0 },
        { "offset": 5, "length": 4, "key": 1 }
      ],
    }
  ],
  "entityMap": {
    "0": {
      "type": "LINK",
      "data": { "url": "/", "id": 60, "parentId": 1 }
    },
    "1": {
      "type": "LINK",
      "data": { "url": "/", "id": 60, "parentId": 1 }
    }
  }
}

Then contentstate -> HTML rendered on the page as:

<p><a href="/">link </a><a href="/">link</a></p>

Then HTML -> contentstate back into the CMS UI:

{
    "blocks": [
      {
        "text": "link link",
        "entityRanges": [
          // Should be length 5.
          { "offset": 0, "length": 4, "key": 0 },
          { "offset": 5, "length": 4, "key": 1 }
        ],
      }
    ],
    "entityMap": {
      "0": {
        "type": "LINK",
        "mutability": "MUTABLE",
        "data": { "id": 60, "url": "/", "parentId": 1 }
      },
      "1": {
        "type": "LINK",
        "mutability": "MUTABLE",
        "data": { "id": 60, "url": "/", "parentId": 1 }
      }
    }
  }

Then re-rendering this to the page:

<p><a href="/">link</a> <a href="/">link</a></p>

---

Thinking of what this causes in practice – I think the most likely issue would be if people use inline formatting where whitespace is styled, for example with a strikethrough, underline, or background like s, mark, ins, del, u:

Strikes back vs Strikes back vs Strikes back

<del>Removed content</del> vs <del>Removed </del><del>content</del> vs <del>Removed</del> <del>content</del>

This would also be an issue for links but it’s not like people are likely to want to link over whitespace anyway.

---

Back to spaceless though – even if we were to fix this it still won’t change the fact that spaceless breaks the very reasonable <p><a href="/">link</a> <a href="/">link</a></p>. So yes I think going for a smarter compressor is mandatory if you want to run this over inline tags. And even then, those compressors only know about elements’ inline vs block formatting as defined by the HTML spec (assuming your site’s stylesheets don’t switch block-level elements to inline).