question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Clarify dimensions issues in generate_dims_coords

See original GitHub issue

In data/base.py in generate_dims_coords() this warning may be emitted:

if len([dim for dim in dims if dim not in default_dims]) > len(shape):
        warnings.warn(
            "More dims ({dims_len}) given than exists ({shape_len}). "
            "Passed array should have shape (chains, draws, *shape)".format(
                dims_len=len(dims), shape_len=len(shape)
            ),
            SyntaxWarning,
        )

This is problematic because

  1. It’s only a warning, so we can’t get into the debugger to see what’s wrong
  2. The warning information is insufficient. At the very least, the var_name argument should be added to the warning string, but I think the dims should be supplied as well.

One more thing if you are looking at this function.
The docstring says

default_dims : list[str]
        Dimensions that do not apply to the variable's shape

In what way are dimensions relevant if they don’t apply to the variable’s shape?

Later it says:

   -------
    list[str]
        Default dims
    dict[str] -> list[str]
        Default coords

This seems to contradict the above definition of “default,” since surely this function isn’t going to return the irrelevant dimensions!

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
rpgoldmancommented, Aug 13, 2019

I think the warning issue was solved in #707.

Yes, that is correct. But, again, shouldn’t this be an error, instead of a warning? Maybe a ValueError? I guess since we can muscle our way through, but give uglier output, a warning is OK?

The docstring clarification is still pending though (@rpgoldman please confirm).

Yes, confirmed.

I agree that the function argument names are not very intuitive and the docstring is not 100% clear, we can discuss here on different names and explanation and then modify the docstring and or arguments.

If it helps, I will try to explain the role of default_dims. This parameter is present in order to take into account that the MCMC results of a variable with shape (3, 7) will have shape (n_chains, n_draws, 3, 7). Thus, in general, the user does not need to explicitly say that the 2 first dimensions are chain, draw, it is assumed for all InferenceData groups except observed_data. It should be noted that generate_dims_coords goal is to get the dimensions and coordinates of the variable, not of its MCMC results.

In my opinion, the best option would be to edit (again, sorry) the warning message, clarify the docstring and optionally, change the name of default_dims. I do not know how should the dosctring be modified to be both clear and concise though. With respect to the warning, now it is:

    warnings.warn(
        (
            "In variable {var_name}, there are "
            + "more dims ({dims_len}) given than exist ({shape_len}). "
            + "Passed array should have shape (chains, draws, *shape)"
        ).format(var_name=var_name, dims_len=len(dims), shape_len=len(shape)),
        SyntaxWarning,
    )

However, the proper message should not have chains, draws hardcoded (it makes no sense in the case of observed_data group). It should use something like ", ".join(default_dims) which will be chain, draw, in most cases, or nothing, in the case of observed_data.

I have pushed #793 for this that we can discuss. It includes a fix to the warning message (maybe we could work out together a test case to make sure I’m handing None properly, etc?).

0reactions
rpgoldmancommented, Jan 6, 2020

I think we can call it fixed in the warning above. We can potentially return to it later.

Read more comments on GitHub >

github_iconTop Results From Across the Web

The behavior of excluded dimensions needs clarified. #2478
The behavior of excluded dimensions needs clarified. #2478. Open. drcrallen opened this issue on Feb 16, 2016 · 1 comment.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found