Compatibility issues with Python 3.9.7 and altair 4.1.0?

I have some scripts I re-run weekly and they have been working fine for over a year now. That is until, I had to set up my computer completely new (new operating system: Mac OS Big Sur 11.6) which also prompted a series of installing newer versions (Upgrading Python from 3.7.2 to 3.9.7 and upgrading altair from 3.2.0 to 4.1.0). Since then my scripts don’t work anymore and produce errors I can’t figure out.

First, things that work:

• running altair sample code, eg from the Example Gallery (both for charts as well as for maps)
• creating my own charts that are not using alt.layer()

Things that don’t work:

1. any kind of map that isn’t sample code but self made

2. charts (eg line chart with multiple lines) with alt.layer()

Details on errors:

1. For the maps I get a AttributeError: 'list' object has no attribute 'get' error, that is thrown at alt.mark_geoshape() and then moves on to defining the projection. The error message references altair/utils, the vegalite core and api as well as Python’s jsonschema validator – so I’m not sure whether it is an altair or a Python problem

2. For the charts I get a TypeError: Object of type set is not JSON serializable error, although I’m not aware of any set I’m using/creating

Any pointers as to what is going on and / or how to fix it would be much appreciated. After three days of trying and googling, I’m out of ideas 😉

Complete error messages

1. For maps:

AttributeError                            Traceback (most recent call last)
/var/folders/fk/_qbrt9ys18b6s2yb_kqblv1xt9_xg3/T/ipykernel_36594/216999948.py in <module>
      8 center_values=[25,10]
      9 
---> 10 base = alt.Chart(countries).mark_geoshape(
     11     fill='#D8DDE2',
     12     stroke='white'

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/vegalite/v4/api.py in project(self, type, center, clipAngle, clipExtent, coefficient, distance, fraction, lobes, parallel, precision, radius, ratio, reflectX, reflectY, rotate, scale, spacing, tilt, translate, **kwds)
    664 
    665         """
--> 666         projection = core.Projection(
    667             center=center,
    668             clipAngle=clipAngle,

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/vegalite/v4/schema/core.py in __init__(self, center, clipAngle, clipExtent, coefficient, distance, extent, fit, fraction, lobes, parallel, parallels, pointRadius, precision, radius, ratio, reflectX, reflectY, rotate, scale, size, spacing, tilt, translate, type, **kwds)
  12124                  size=Undefined, spacing=Undefined, tilt=Undefined, translate=Undefined, type=Undefined,
  12125                  **kwds):
> 12126         super(Projection, self).__init__(center=center, clipAngle=clipAngle, clipExtent=clipExtent,
  12127                                          coefficient=coefficient, distance=distance, extent=extent,
  12128                                          fit=fit, fraction=fraction, lobes=lobes, parallel=parallel,

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/utils/schemapi.py in __init__(self, *args, **kwds)
    174 
    175         if DEBUG_MODE and self._class_is_valid_at_instantiation:
--> 176             self.to_dict(validate=True)
    177 
    178     def copy(self, deep=True, ignore=()):

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/utils/schemapi.py in to_dict(self, validate, ignore, context)
    335         if validate:
    336             try:
--> 337                 self.validate(result)
    338             except jsonschema.ValidationError as err:
    339                 raise SchemaValidationError(self, err)

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/utils/schemapi.py in validate(cls, instance, schema)
    440             schema = cls._schema
    441         resolver = jsonschema.RefResolver.from_schema(cls._rootschema or cls._schema)
--> 442         return jsonschema.validate(instance, schema, resolver=resolver)
    443 
    444     @classmethod

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in validate(instance, schema, cls, *args, **kwargs)
    963     cls.check_schema(schema)
    964     validator = cls(schema, *args, **kwargs)
--> 965     error = exceptions.best_match(validator.iter_errors(instance))
    966     if error is not None:
    967         raise error

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/exceptions.py in best_match(errors, key)
    352     """
    353     errors = iter(errors)
--> 354     best = next(errors, None)
    355     if best is None:
    356         return

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in iter_errors(self, instance, _schema)
    222 
    223                     errors = validator(self, v, instance, _schema) or ()
--> 224                     for error in errors:
    225                         # set details if not already set by the called fn
    226                         error._set(

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/_validators.py in ref(validator, ref, instance, schema)
    293 
    294         try:
--> 295             yield from validator.descend(instance, resolved)
    296         finally:
    297             validator.resolver.pop_scope()

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in descend(self, instance, schema, path, schema_path)
    238 
    239         def descend(self, instance, schema, path=None, schema_path=None):
--> 240             for error in self.evolve(schema=schema).iter_errors(instance):
    241                 if path is not None:
    242                     error.path.appendleft(path)

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in iter_errors(self, instance, _schema)
    222 
    223                     errors = validator(self, v, instance, _schema) or ()
--> 224                     for error in errors:
    225                         # set details if not already set by the called fn
    226                         error._set(

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/_validators.py in properties(validator, properties, instance, schema)
    327     for property, subschema in properties.items():
    328         if property in instance:
--> 329             yield from validator.descend(
    330                 instance[property],
    331                 subschema,

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in descend(self, instance, schema, path, schema_path)
    238 
    239         def descend(self, instance, schema, path=None, schema_path=None):
--> 240             for error in self.evolve(schema=schema).iter_errors(instance):
    241                 if path is not None:
    242                     error.path.appendleft(path)

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in iter_errors(self, instance, _schema)
    222 
    223                     errors = validator(self, v, instance, _schema) or ()
--> 224                     for error in errors:
    225                         # set details if not already set by the called fn
    226                         error._set(

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/_validators.py in ref(validator, ref, instance, schema)
    293 
    294         try:
--> 295             yield from validator.descend(instance, resolved)
    296         finally:
    297             validator.resolver.pop_scope()

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in descend(self, instance, schema, path, schema_path)
    238 
    239         def descend(self, instance, schema, path=None, schema_path=None):
--> 240             for error in self.evolve(schema=schema).iter_errors(instance):
    241                 if path is not None:
    242                     error.path.appendleft(path)

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in iter_errors(self, instance, _schema)
    222 
    223                     errors = validator(self, v, instance, _schema) or ()
--> 224                     for error in errors:
    225                         # set details if not already set by the called fn
    226                         error._set(

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/_validators.py in items(validator, items, instance, schema)
     74     else:
     75         for index in range(prefix, total):
---> 76             yield from validator.descend(
     77                 instance=instance[index],
     78                 schema=items,

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in descend(self, instance, schema, path, schema_path)
    238 
    239         def descend(self, instance, schema, path=None, schema_path=None):
--> 240             for error in self.evolve(schema=schema).iter_errors(instance):
    241                 if path is not None:
    242                     error.path.appendleft(path)

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in iter_errors(self, instance, _schema)
    212                 return
    213 
--> 214             scope = id_of(_schema)
    215             if scope:
    216                 self.resolver.push_scope(scope)

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/jsonschema/validators.py in _id_of(schema)
     88     if schema is True or schema is False:
     89         return ""
---> 90     return schema.get("$id", "")
     91 
     92 

AttributeError: 'list' object has no attribute 'get'

1. For charts:

TypeError                                 Traceback (most recent call last)
~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/IPython/core/formatters.py in __call__(self, obj, include, exclude)
    968 
    969             if method is not None:
--> 970                 return method(include=include, exclude=exclude)
    971             return None
    972         else:

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/vegalite/v4/api.py in _repr_mimebundle_(self, include, exclude)
   1652             return {}
   1653         else:
-> 1654             return renderers.get()(dct)
   1655 
   1656     def display(self, renderer=Undefined, theme=Undefined, actions=Undefined, **kwargs):

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/utils/display.py in __call__(self, spec, **metadata)
    178         kwargs = self.kwargs.copy()
    179         kwargs.update(metadata)
--> 180         return spec_to_mimebundle(
    181             spec, format="html", output_div=self.output_div, **kwargs
    182         )

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/utils/mimebundle.py in spec_to_mimebundle(spec, format, mode, vega_version, vegaembed_version, vegalite_version, **kwargs)
     60         return altair_saver.render(spec, format, mode=mode, **kwargs)
     61     if format == "html":
---> 62         html = spec_to_html(
     63             spec,
     64             mode=mode,

~/.virtualenvs/DW_Data-QfWUG2cH/lib/python3.9/site-packages/altair/utils/html.py in spec_to_html(spec, mode, vega_version, vegaembed_version, vegalite_version, base_url, output_div, embed_options, json_kwds, fullhtml, requirejs, template)
    219 
    220     return template.render(
--> 221         spec=json.dumps(spec, **json_kwds),
    222         embed_options=json.dumps(embed_options),
    223         mode=mode,

/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    229         cls is None and indent is None and separators is None and
    230         default is None and not sort_keys and not kw):
--> 231         return _default_encoder.encode(obj)
    232     if cls is None:
    233         cls = JSONEncoder

/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py in default(self, o)
    177 
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181 

TypeError: Object of type set is not JSON serializable

[jakevdp]

Thanks for the report. This is due to the most recent release of jsonschema; see #2496. There is a fix in progress in #2503, until then you can pip install jsonschema==3.2 to fix the error.

[gcgruen]

Thank you for teaching me about the Minimal reproducible example! That seems to be a great way to debug code, as now the original problem does not exist anymore, but I found new ones that are probably more related to my way of coding rather than to any bugs, I assume. So I’m closing this for now; in case I run into a dead end will submit a new issue with including a minimal reproducible example. Thanks again for your help and patience!