tokenizer treats commands as expandable strings

The tokenizer seems to treat commands (and some other tokens) as expandable strings. Why is this? It makes misleading colorization in PSReadLine.

function crazy$function {echo test} # `$function` appears to be a variable, but is not

crazy$function # again, `$function` appears to be a variable that will expand, but it is not

Another one that occurs is if you start a command with a dash, the tokenizer marks it as a parameter, but it should mark it as a command.

[bpayette]

By design, a command name can contain any characters that can be in a file name since, rather obviously, files are commands. And, since it’s also perfectly valid to have a file foo$bar.ps1, such names must be supported as commands. So functionally, everything is fine and cannot change. But it would be nice to fix the highlighting . The token is correct in the AST. Can you not use that?

[bpayette]

@msftrncs I think @iSazonov means that having a function name likefoo$bar is an edge-case.