textX + PyEcore

Note: In this issue, are reported discussions and experiments around the association of textX and PyEcore. Progress can be found here and it currently requires the PyEcore version from the develop branch. (follows discussion from igordejanovic/textX#41 and igordejanovic/textX#42)

TextX is a meta-language, inspired by Xtext which allows you to easily generate a parser and an metamodel/model-based AST for a textual language. The metamodel based AST created by textX is not yet compatible with the Ecore API. This POC tries to bring PyEcore, so Ecore, to textX in order to be able to handle an Ecore compatible model and metamodel.

This addition could open a way of interoperability between Java-EMF and Python-EMF (PyEcore) through the xmi/ecore file format. Moreover, using pyecoregen, it could enable a full Python T2M (Text to Model), and M2T (Model to Text) experience.

The goal of this POC is to see if PyEcore can be integrated inside textX following three scenarios:

1. from a textX grammar, an Ecore metamodel is automatically generated, and the generated parser + metamodel can be directly used to build the AST.
2. using existing dynamic Ecore metamodel (built in memory) and a textX grammar, textX uses the defined dynamic Ecore metamodel to build the AST.
3. using a first compilation phase: a textX grammar is used for generating the Python Ecore metamodel code (through the textX cli). Then the generated Python code is used by textX to build the AST.

This latter scenario allows the user to easily add behavior to the metaclasses instances.

The scenario 2 had been already validated. This implied minor modifications of how textX create instances and init them (no need to init instances anymore as PyEcore does it), and a biggest modifications on how the cross-references are resolved.

The scenario 1 and 3 relies on the same principle: the Ecore metamodel must be generated on-the-fly from the textX grammar. Consequently, the isses and challenges are not the same as in scenario 2.

Here what’s have been validated so far and the future features to add:

• Replace textX metaclasses creation by EClass instances
• Add EAttribute/EReference support for EClass
• Reference eOpposite support for static and dynamic Ecore metamodel (not automatically generated at the moment)
• Deal with EClass inheritance
• Deal with abstract classes
• EPackage creation for a grammar/metamodel
• Detection of rules that define EDataType
• Creation of EDataType
• Detection of rules that define EEnum (currently not bullet-proof)
• Creation/Filling of EEnum
• Basic data types support (should be modified)
• Management of split grammar (defined in many files)
• Management of multiple namespace/EPackage
• Detect ‘match’ and ‘non-match’ rule call in the same rule definition as semantic errors
• Better support for multiple grammars relative paths
• Find a common way of dealing with basic EDataType and textX builtin basic types
• Deal with split grammar/metamodel generation (generations and imports)
• Add a cli option to generate the PyEcore metamodel using pyecoregen
• Add special parameter to pass an existing EPackage as user-classes
• Add a kind of “switch” for PyEcore activation in textX
• Add better multiple metamodel generation support (currently, some imports are missing)
• Add documentation! (obviously)

As a complement of this list, preferably, the PyEcore integration is optional and can be activated using a parameter in textX. Also, another idea would be to add a dedicated keyword to introduce eOpposite from grammar.

The generation of the metamodel Python code using the cli also implies the question of re-generation and perservation of manually inserted modifications. This issue is more general and could be addressed in pyecoregen or pymultigen.

[ipa-hsd]

@aranega any updates on this? Would be great to move my EMF and Xtext based projects to textX and pyecore!

[fibersel]

@aranega Hi, I want to parse DSL language into Ecore model.How can I do this using textX and pyecore?