question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Normalize input to `list` properly in `normalize_args`

See original GitHub issue

Currently, if a user wants to partition along column “x”, and since columns are unique, thinks it is appropriate to provide a set as the value to the argument e.g. partition_on={"x"}; this would be “normalized” to [{"x"}] ( a list of sets), and then an error will be raised at some other part of the code where this input is unexpected. The expected behavior of the normalization would be to normalize sets, tuples or similar iterables to a list such as ["x"] (a list of strings).

I do not consider this a bug as in the docs it states partition_on should be provided as a list.

Normalization function: kartothek.io_components.utils.normalize_arg

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
fjettercommented, Jun 1, 2019

Sets can not be properly normalized since they don’t preserve order. For ‘partition_on’ the order is quite important, i.e. a set would be an incompatible input

0reactions
lr4dcommented, Jun 4, 2019

Hint: the following could be a start to parse (0,1) to [0,1] instead of the current [(0,1)].

def _make_list(x):
    if isinstance(x, (str, bytes)):
        return [x]
    return list(x)
    elif old_value is None:
        new_value = _ARGS_TO_TYPE[arg_name]()
Read more comments on GitHub >

github_iconTop Results From Across the Web

Normalizing a list of numbers in Python
Use : norm = [float(i)/sum(raw) for i in raw]. to normalize against the sum to ensure that the sum is always 1.0 (or...
Read more >
How to Normalize data in R [3 easy methods]
In such cases, the easiest way to get values into proper scale is to scale ... rm(list = ls()) data = c(1200,34567,3456,12,3456,0985,1211) ...
Read more >
Text Normalization. Why, what and how. | by Tiago Duque
Therefore, take the list of normalization steps presented in this article as not hard rules, but instead as guidelines for doing text Normalization....
Read more >
Normalize Data: Component Reference - Azure Machine ...
The goal of normalization is to change the values of numeric columns in the dataset to use a common scale, without distorting differences...
Read more >
Use the CIM to normalize data at search time
Tagging all of the authentication related events appropriately makes it possible for your dashboards to pull data from the correct events ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found