Model support for Citus Data

With Microsoft’s recent acquisition of Citus Data and their rollout of Hyperscale (Citus) for Azure Database for PostgreSQL, we should expect more Azure users to want Citus integration for their EF Core PostgreSQL database models.

Adding an extension package (Npgsql.Citus?) would allow users to opt into Citus support and configure their distributed tables, reference tables, local tables, and set access methods using LINQ model configuration extension methods.

I believe this could be accomplished using custom entity annotations in migrations that generate the required SQL to configure Citus.

Any suggestions for this would be greatly appreciated!

[kirkbrauer]

@roji Thank you for getting back about this issue!

I am a bit newer to Citus data as we are currently investigating using it on our project. That’s why I preemptively created this issue in case we decide to go ahead with that in the future. However, I have been reviewing the docs, so I think I can propose a draft API and maybe somebody with a bit more experience can review it and suggest additions.

The API I’m currently envisioning looks like this:

// Distribute the 'Products' table by 'ShopId'
context.Entity<Product>().DistributeBy(x => x.ShopId);

// Distrubute the 'Variants' table by 'ShopId' and co-locate with 'Products'
context.Entity<Variant>().DistributeBy(x => x.ShopId).ColocateWith(context.Products);

// Make 'States' a reference table located on all nodes
context.Entity<State>().AsReferenceTable();

// Store 'Events' as append-only columnar data
context.Entity<Event>().HasAccessMethod(CitusAccessMethod.Columnar);

I’m open to suggestions on the naming conventions for these extension methods, for example, it may make more sense to follow the EF Core naming conventions:

context.Entity<Product>().HasDistributionProperty(x => x.ShopId);

[kirkbrauer]

I would, but first I would like to take a look at all the other methods in Citus Data, there are some specific configurations that we may need to be aware of via metadata so we can produce the correct migrations between them.

A good example of this would be un-distributing a table and verifying that there are no foreign keys that would need to be cascaded as part of the un-distribution process. The same applies for making a table to a reference table depending on if it’s already distributed on a single shard or not.