Flexible mapping to CLR types and members (Custom O/C Mapping)

This is a grouping of related issues. Feel free to vote (👍) for this issue to indicate that this is an area that you think we should spend time on, but consider also voting for individual issues for things you consider especially important.

---

This an “epic” issue for the theme of improvements to mapping of different CLR constructs to standard database concepts. Specific pieces of work will be tracked by linked issues.

Done in 7.0

• Improve Exception message: Cannot bind to constructor parameters #19383
• Support more types of value generation with converters #11597

Backlog

• Implement value conversions that spread out over multiple columns (value objects) #13947
• Use C# structs or classes as value objects #9906
• Expand tuples to multiple columns #14661
• Support value conversions from two properties into one column #20582
• Allow HasConversion/ValueConverters to convert nulls #13850
• Support different SQL translation when store representation has changed due to value conversion #10434
• Enum Flags and HasConversion<string>() #20094
• Allow different types for FK/PK if a value converters to compatible provider types are used #22542
• Unable to map an enum FK to an integer PK #24568
• Cannot create composite key from class #16450
• Update type mapping/converters to make translation data available for query generation #15979
• Mixing Database Functions and Value Conversions #16617
• Value Converters: Enable passing in some context #12205
• Support value conversions for Spatial datatypes and UDTs #23352
• Support UTF8 end-to-end #14066
• Allow related entities to be passed to constructor of aggregate root #12078
• Support inheritance for owned types #9630
• Custom collection patterns #752
• Collections of primitive types #30731
• Use correct value comparers for collection properties with value conversion #17471
• Move collection comparers to Core #25364
• Richer collection support #2919
• Support IEnumerable navigation properties with Add/Remove/Contains methods/delegates #752
• Support fixed-size collections #24497
• Ordered Collection Navigation Properties (aka Indexed Collections) #9067
• Allow IImmutableList<T> for collection navigational properties with backing fields? #21176
• Indexer Navigations #13729
• Custom property access patterns #2968
• Support saving no-setter properties (for example expression bodied properties) #2319
• Support immutable entity types #11457
• Allow properties to use a different backing field on a derived type #24006
• Expand property bag definition #22009
• Flexible construction through factory methods and DI #3342
• Add API to configure construction of entity type instances #10789
• Support Maybe patterns for navigations #22504
• Allow mapping to interfaces with polymorphic associations #757
• Defining aliases for enum values #20202
• Allow entity type instance to implement/inherit from type in model #13344
• Support IDictionary navigation property for one-to-many relationship #21262
• Support Polymorphic Relationships #7623
• Support non-many-to-many skip navigations #21673
• Model “implicit” many-to-many scenarios via readonly navigations #20337

---

Although we have had support for POCOs through a few major versions, using EF for persistence still places some undesired constraints on the design choices developers can make on their domain objects.

For example, EF Core still requires that mapped CLR types can be instantiated through constructors without parameters (this is only true for navigation properties now), that they contain both property getters and setters for each mapped scalar and reference property, and that all mapped collection navigation properties are exposed as properties that are of a type that implements ICollection<T>.

Note that POCO proxy functionality introduces additional constraints, e.g. lazy loading requires that property getters are virtual and change tracking proxies (not yet supported in EF Core) require that all properties are virtual at the same time that properties mapped to collections are effectively declared as ICollection<T>.

EF Core also requires each entity type in the model to be mapped to a distinct CLR type, which makes some dynamic model scenarios (#2282) harder.

Moreover, scalar properties on objects have to be of a small set of recognized types in order to be mapped (in EF Core the set of types supported nativly by the provider).

Richer constructs such as collections of scalars or collections of complex types, ordered collections, inheritance in complex types, collection manipulation methods, factory methods, and immutable objects are not supported.

This issue tracks the removal of those constrains as a whole and serves as a parent issue for some individual features that we will track independently:

An old hope

One of many interesting design strategies of EF as an O/RMs was that every EF model defined an abstract data model that is separate from the object model. When using EF in regular O/RM scenarios there are in fact three layered models involved: the storage/database model (often referred to as the S-space model), the conceptual/abstract data model (often referred to as the C-space) and the actual object model (also known as the O-space model). EF also makes use of two mapping specifications: the translation of data access operations between the C-space model and the S-space model is the most well-known and where most of the mapping capabilities of EF focus. The mapping between the C-space and the O-space model is less widely known and in fact since only trivial 1:1 mappings are supported.

There are many historical motivations for this underlying design, including the desire to provide a complete “weakly typed“ programming interface (often referred to as a “value layer”) that could be used in scenarios outside the traditional O/RM space, such as runtime metadata driven user interfaces and tools, database reporting, etc. EntityClient was intended to be such weakly typed API, but it had considerable restrictions (e.g. it was read-only) and some usability issues (e.g. it was designed after an ADO.NET provider without a public DbDataAdapter and with a quite complicated connection string format) on the first version of EF. Since its popularity never took off, the investments necessary to unleash the “value layer” were never completed. In the meanwhile, EF role in solving the traditional O/RM scenario of mapping strongly typed objects to database tables and functions became more and more relevant since EF4, so the lion share of the investments have gone into making EF easier to use in those scenarios, which in many cases meant burying the separation between the abstract data model and the object model even deeper under layers of new API.

In practice the object model can be considered for most intents and purposes a straight realization of what is in C-space. The most commonly used APIs on EF take advantage of this similarity and just conflates C-space and O-space seamlessly, abstracting away the fact that there is a layer of indirection. In other words, although this differentiation between C-space and O-space is real and very explicit in the implementation, it is purposely not evident in most of the EF APIs and usage patterns. A clear example of this is Code First. To a great degree, the usability of the Code First API relies on users being able to ignore the existence of an abstract data model.

While sharing the same meta-model for conceptual models has made it easier to expose EF models through things like OData services, the majority of EF users has never needed to know that there is a difference between the conceptual and object models of their applications.

Regardless of the past motivations for creating this extra layer of indirection, there is potential in the decoupling of the conceptual level (at which most of EF’s mapping, query and update pipeline operates) and the objects level. The basic idea is that by breaking the current status quo of 1:1 O-C mapping, it should be possible to extend EF to support much richer object mapping without having to touch at all the guts of the system.

[marchy]

Guys this should be prioritized. The whole point of an ORM is to Map to Objects… and EF has traditionally stood out particularly due to its ability to map to POCOs / domain representations of objects.

This means removing the friction of relational mapping and the ability to hide the complexity. Today there is still quite a LOT of friction in this and the reality is quite a ways from the promise/vision.

I know the team’s been really focused on re-creating EF Core for modern computing paradigms (cloud, lightweight client, mobile – still missing, though really who cares – Realm has you beat)… but I feel it’s really time to get back to new feature growth as opposed to catch-up/parity with EF6.

EF’s biggest benefits are that of productivity by removing the object-relational friction / impedance mismatch and custom mapping would allow the framework to much better deliver on its promise.

Would love to see the team bite off this big, ambitious chunk and put the framework on an exciting path against the alternatives!

#keepupthegreatwork

[hidegh]

@jnm2 a clean code will keep cost of quality low (less bugs, easier to maintain, easier to extend). considering prices for a developer vs. price for better hardware… btw. if you need performance, there’s nothing wrong to have NHibernate for the command and views and EF (even database-first) with async for the read model.