Tokenizing fails when font property line height value is set from custom property

Tokenizer fails to tokenize font property correctly if line-height is using a custom property. Example of CSS that is not tokenized correctly:

html { font: 1em/var(--line-height); }

Tokenizer seems to think that 1em/var is a function.

I created a failing test for this here: https://github.com/lauriii/postcss/commit/87085589fa6ed303f0f2052f424214a54a804b72

This trickles down to https://github.com/shellscape/postcss-values-parser. I also created a failing test for that https://github.com/lauriii/postcss-values-parser/commit/d0bbbbcae47afee98f2d367e3fd0e525d0480cfe.

This finally trickles down to https://github.com/postcss/postcss-custom-properties. I did also create a failing test for that: https://github.com/lauriii/postcss-custom-properties/commit/6de537ec9c25c87f3f5b41be0dd0f353a9f83e28.

[shellscape]

@ai no worries, I don’t believe the issue is in PostCSS’s tokenizer 😄

The tokenizer in postcss-values-parser was always very difficult for me to maintain before I started leveraging PostCSS tokenizer - but please let me specify by saying that it was only difficult for me personally. I would never say that it would be 1000x worse for anyone else 👍

[ai]

It’s actually quite an efficient and reliable method and makes maintenance 1000x easier.

I am supporting the tokenizer in potscss-scss. It is not 1000x worse (check out special patch comments in SCSS tokenizer).

If you can’t send non-breaking changes PR, I can accept it.

I understand that you want “ready for everything tokenizer”, but I just don’t believe that it is possible to create it (and I can change my mind if you will show me some).