[config] no param naming validation

I have noticed that one could make a typo in the config file and DeepSpeed silently ignores it. e.g. I did:

    "gradient_clipp1ng": 1

and got no errors.

Not sure if it should validate the value as well, as float is expected, but I guess int is fine as it’s just converted to float.

This lack of validation could lead to user errors which could be hard to notice.

Nothing urgent, but at some point it’d be good to tell the user if they added a config entry which is invalid.

Thank you.

[stas00]

I was going to suggest YAML, but I thought that something better must have been invented since then. YAML is from around 2001 if I remember correctly.

I have never used https://hjson.github.io/, but it does look like the most straightforward solution as it was created to overcome limitations of json for human use. May be that’s all is needed.

I don’t know how your user/developer community works, perhaps throwing the net wider would give practical feedback from users who may have used hjson already and can tell how good it is in practice.

[jeffra]

Thanks @stas00, JSON validation I think is on our radar and definitely something we should add in the short term. Also agree JSON isn’t the most user-friendly of formats. We’ll have to look into alternatives. I have heard good things about hjson. It allows commas/comments but I don’t think supports numbers in the way you describe unfortunately. YAML might also be a reasonable option but also comes with difficulties haha

It appears YAML supports numbers like 5_000_000_000 and comments. We’ll have to look into this further.