Remove slash from all data variable names

We’re currently naming variables like call/genotype, variant/id, sample/id, etc., but I think we should switch to call_genotype, variant_id, sample_id, etc.

The disadvantages of using the slashes are:

• Xarray stores these as separate Zarr groups which means you can’t load an sgkit dataset with single command. You have to instead do something like this: ds = xr.merge([xr.open_zarr(path, group=g) for g in ['call', 'variant', 'sample']). There is no clear advantage to having the variables split up on disk by this grouping. If they were instead grouped by something more meaningful like contig, the partitioning would make more sense but creating directories based on similar variables does not.
• Assigning variables requires a kwargs splat rather than using the simpler ds.assign(call_genotype=...) syntax, e.g. ds.assign(**{'call/genotype': ...})
• I’ve found that for some datasets, you can’t pass custom Zarr encodings to Xarry when variables have ‘/’ in the name – the bug has been hard to reproduce on a small dataset so I’m not sure why yet.
• You cannot autocomplete variable names on a dataset instance

The only disadvantage I can see to not using the ‘/’ is that it offers a convenient delimiter for extracting the group name for a set of variables like “variant” or “call”. I don’t think that’s difficult to live without and using underscore case is more common in other pydata projects anyhow.

@alimanfoo or @tomwhite do you have any objections to this?

[eric-czech]

Yes and I don’t want to use them: https://github.com/pystatgen/sgkit/issues/17

[alimanfoo]

Hi Eric, no objection, sounds like several things will fit with xarray more naturally if we avoid slash so happy to go with your suggestion.

On Sat, 1 Aug 2020, 15:22 Eric Czech, [e-mail] wrote:

We’re currently naming variables like call/genotype, variant/id, sample/id, etc., but I think we should switch to call_genotype, variant_id, sample_id, etc.

The disadvantages of using the slashes are:

• Xarray stores these as separate Zarr groups which means you can’t load an sgkit dataset with single command. You have to instead do something like this: ds = xr.merge([xr.open_zarr(path, group=g) for g in [‘call’, ‘variant’, ‘sample’]). There is no clear advantage to have the variables split up on disk by this grouping. If they were instead grouped by something more meaningful like contig, the partitioning would make more sense but creating directories based on similar variables does not.
• Assigning variables requires a kwargs splat rather than using the simpler ds.assign(call_genotype=…) syntax, e.g. ds.assign(**{‘call/genotype’: …})
• I’ve found that for some datasets, you can’t pass custom Zarr encodings to Xarry when variables have ‘/’ in the name – the bug has been hard to reproduce on a small dataset so I’m not sure why yet.
• You cannot autocomplete variable names on a dataset instance

The only disadvantage I can see to not using the ‘/’ is that it offers a convenient delimiter for extracting the group name for a set of variables like “variant” or “call”. I don’t think that’s difficult to live without and using underscore case is more common in other pydata projects anyhow.

@alimanfoo https://github.com/alimanfoo or @tomwhite https://github.com/tomwhite do you have any objections to this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pystatgen/sgkit/issues/81, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFLYQR2SL5NMCUEVY37SY3R6QQKTANCNFSM4PRZBAQQ .