Multiple group constraints fail with different constraint types
See original GitHub issueProblem description
When there is more than one group constraint, and the group constraints have different constraint types, Calliope fails
Error message from test
calliope/test/common/util.py:30: in build_test_model
scenario=scenario
calliope/core/model.py:70: in __init__
self._init_from_model_run(model_run, debug_data)
calliope/core/model.py:92: in _init_from_model_run
self._model_data_original = build_model_data(model_run)
calliope/core/preprocess/model_data.py:71: in build_model_data
data.merge(xr.Dataset.from_dict(data_dict), inplace=True)
../../miniconda3/envs/calliope_dev/lib/python3.6/site-packages/xarray/core/dataset.py:2571: in merge
join=join)
../../miniconda3/envs/calliope_dev/lib/python3.6/site-packages/xarray/core/merge.py:561: in dataset_merge_method
return merge_core(objs, compat, join, priority_arg=priority_arg)
../../miniconda3/envs/calliope_dev/lib/python3.6/site-packages/xarray/core/merge.py:439: in merge_core
aligned = deep_align(coerced, join=join, copy=False, indexes=indexes)
../../miniconda3/envs/calliope_dev/lib/python3.6/site-packages/xarray/core/alignment.py:216: in deep_align
exclude=exclude)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
objects = (<xarray.Dataset>
Dimensions: (carrier_tiers: 2, carriers: 1, constraint...e_max (constraint_groups, carriers) float64 0.4
group_demand_share_min (constraint_groups, carriers) float64 0.6)
kwargs = {}, join = 'outer', copy = False, indexes = {}, exclude = frozenset()
all_indexes = defaultdict(<class 'list'>, {'carrier_tiers': [Index(['out', 'in'], dtype='object', name='carrier_tiers')], 'carriers'...21:00:00', '2005-02-01 22:00:00',
'2005-02-01 23:00:00'],
dtype='object', name='timesteps', length=768)]})
unlabeled_dim_sizes = defaultdict(<class 'set'>, {'carriers': {1}, 'constraint_groups': {1}, 'costs': {1}, 'loc_techs': {6}, 'loc_techs_finite_resource': {2}, 'loc_techs_investment_cost': {4}, 'techs': {3}})
dim = 'constraint_groups', joiner = functools.partial(<built-in function reduce>, <built-in function or_>)
joined_indexes = {'carriers': Index(['electricity'], dtype='object', name='carriers'), 'constraint_groups': Index(['demand_share_max', 'demand_share_min'], dtype='object', name='constraint_groups')}
unlabeled_sizes = {1}
def align(*objects, **kwargs):
"""align(*objects, join='inner', copy=True, indexes=None,
exclude=frozenset())
Given any number of Dataset and/or DataArray objects, returns new
objects with aligned indexes and dimension sizes.
Array from the aligned objects are suitable as input to mathematical
operators, because along each dimension they have the same index and size.
Missing values (if ``join != 'inner'``) are filled with NaN.
Parameters
----------
*objects : Dataset or DataArray
Objects to align.
join : {'outer', 'inner', 'left', 'right', 'exact'}, optional
Method for joining the indexes of the passed objects along each
dimension:
- 'outer': use the union of object indexes
- 'inner': use the intersection of object indexes
- 'left': use indexes from the first object with each dimension
- 'right': use indexes from the last object with each dimension
- 'exact': instead of aligning, raise `ValueError` when indexes to be
aligned are not equal
copy : bool, optional
If ``copy=True``, data in the return values is always copied. If
``copy=False`` and reindexing is unnecessary, or can be performed with
only slice operations, then the output may share memory with the input.
In either case, new xarray objects are always returned.
exclude : sequence of str, optional
Dimensions that must be excluded from alignment
indexes : dict-like, optional
Any indexes explicitly provided with the `indexes` argument should be
used in preference to the aligned indexes.
Returns
-------
aligned : same as *objects
Tuple of objects with aligned coordinates.
Raises
------
ValueError
If any dimensions without labels on the arguments have different sizes,
or a different size than the size of the aligned dimension labels.
"""
join = kwargs.pop('join', 'inner')
copy = kwargs.pop('copy', True)
indexes = kwargs.pop('indexes', None)
exclude = kwargs.pop('exclude', _DEFAULT_EXCLUDE)
if indexes is None:
indexes = {}
if kwargs:
raise TypeError('align() got unexpected keyword arguments: %s'
% list(kwargs))
if not indexes and len(objects) == 1:
# fast path for the trivial case
obj, = objects
return (obj.copy(deep=copy),)
all_indexes = defaultdict(list)
unlabeled_dim_sizes = defaultdict(set)
for obj in objects:
for dim in obj.dims:
if dim not in exclude:
try:
index = obj.indexes[dim]
except KeyError:
unlabeled_dim_sizes[dim].add(obj.sizes[dim])
else:
all_indexes[dim].append(index)
# We don't reindex over dimensions with all equal indexes for two reasons:
# - It's faster for the usual case (already aligned objects).
# - It ensures it's possible to do operations that don't require alignment
# on indexes with duplicate values (which cannot be reindexed with
# pandas). This is useful, e.g., for overwriting such duplicate indexes.
joiner = _get_joiner(join)
joined_indexes = {}
for dim, matching_indexes in iteritems(all_indexes):
if dim in indexes:
index = utils.safe_cast_to_index(indexes[dim])
if (any(not index.equals(other) for other in matching_indexes) or
dim in unlabeled_dim_sizes):
joined_indexes[dim] = index
else:
if (any(not matching_indexes[0].equals(other)
for other in matching_indexes[1:]) or
dim in unlabeled_dim_sizes):
if join == 'exact':
raise ValueError(
'indexes along dimension {!r} are not equal'
.format(dim))
index = joiner(matching_indexes)
joined_indexes[dim] = index
else:
index = matching_indexes[0]
if dim in unlabeled_dim_sizes:
unlabeled_sizes = unlabeled_dim_sizes[dim]
labeled_size = index.size
if len(unlabeled_sizes | {labeled_size}) > 1:
raise ValueError(
'arguments without labels along dimension %r cannot be '
'aligned because they have different dimension size(s) %r '
'than the size of the aligned dimension labels: %r'
> % (dim, unlabeled_sizes, labeled_size))
E ValueError: arguments without labels along dimension 'constraint_groups' cannot be aligned because they have different dimension size(s) {1} than the size of the aligned dimension labels: 2
../../miniconda3/envs/calliope_dev/lib/python3.6/site-packages/xarray/core/alignment.py:145: ValueError
Steps to reproduce the problem
Build a model with group constraints using different constraint types. For example:
group_constraints:
demand_share_max:
techs: ["cheap_supply"]
demand_share_max:
electricity: 0.4
demand_share_min:
techs: ["expensive_supply"]
demand_share_min:
electricity: 0.6
Full model
model:
name: Test model for group constraint mechanism
timeseries_data_path: 'timeseries_data'
run:
mode: plan
solver: glpk
techs:
cheap_supply:
essentials:
name: Cheap supply tech
carrier: electricity
parent: supply
constraints:
lifetime: 25
energy_cap_max: .inf
costs:
monetary:
interest_rate: 0.1
energy_cap: 10
expensive_supply:
essentials:
name: Expensive supply
carrier: electricity
parent: supply
constraints:
lifetime: 25
energy_cap_max: .inf
costs:
monetary:
interest_rate: 0.1
energy_cap: 20
electricity_demand:
essentials:
name: Electricity demand
carrier: electricity
parent: demand
constraints:
resource: file=demand_elec.csv
locations:
0,1:
techs:
cheap_supply:
expensive_supply:
electricity_demand:
overrides:
more_than_one_group_constraint:
group_constraints.demand_share_max:
techs: ["cheap_supply"]
demand_share_max:
electricity: 0.4
group_constraints.demand_share_min:
techs: ["expensive_supply"]
demand_share_min:
electricity: 0.6
Calliope version
0.6.4-dev
Issue Analytics
- State:
- Created 5 years ago
- Comments:8 (8 by maintainers)
Top Results From Across the Web
Means, Groups, and Constraints in SEM - GitHub Pages
This is a multiple-group analysis in which we have a covariance matrix for each country in three separate samples (each with N=1000).
Read more >What Are the Different Types of Database Constraints?
DEFAULT constraints can involve only one column, but they can be defined at either level.
Read more >Constraints on type parameters - C# Programming Guide
Learn about constraints on type parameters. Constraints tell the compiler what capabilities a type argument must have.
Read more >Documentation: 15: 5.4. Constraints - PostgreSQL
A foreign key constraint specifies that the values in a column (or a group of columns) must match the values appearing in some...
Read more >Grouping Javax Validation Constraints - Baeldung
By default, the constraint groups are not evaluated in any particular order. But we may have use cases where some groups should be...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I’ll rewrite of the way the data is store in
model_data
today, then (if it works) you can update your PRs to expect the new structure?Yes, I can confirm that my example from above works as expected now. Thanks!