New persistence strategy for better cross-version compatibility

Note - This issue originally called for the use of Json, which was determined be unfeasible due to the lack of a good full-featured Json serializer that is compatible with Unity.

Describe your idea

In TMPE’s current persistence strategy, the introduction of a new type into the persistent data will break backward compatibility. Although old versions are not supported, players reverting from alpha to stable should not lose savegame data needlessly.

The same issue also affects forward compatibility in ways that restrict future development choices. Ideally, the save data would not be bound to specific types. A move away from strongly typed serialization would give everyone more flexibility moving forward.

Requirements

1. This is an opt-in change. All existing serialization will remain untouched until there is a reason to do it the new way, as determined feature by feature.
2. A manager-based persistence mechanism in which SerializableDataExtension enumerates a collection of managers and calls LoadData() and SaveData() methods basically like the ones we have today.
3. SerializableDataExtension will save the data from each manager separately, to isolate them and minimize the risks associated with changes to persistent data.
4. A manager will have a means of identifying other managers upon which its data depends. This will determine the enumeration order during load and save.
5. Linq to XML will be used for persistence.

TrafficLightSimulationManager will be the first converted to the new persistence strategy, since displaced lane support was the breaking change that prompted this issue. Its conversion to the new persistence strategy will be delivered separately from and prior to displaced lane support.

Tasks

• Persistence framework
• Add a DOM to the serialized data and make a mechanism for opt-in transition away from the old serialization strategy (SerializableDataExtension)
• Update timed traffic lights to be saved into the DOM (half-baked persistable model)
• Update junction restrictions to be saved into the DOM (fully persistable model)
• Compress XML data, with fallback to uncompressed if it fails (especially on some Linux distros)
• Test versioning in a relevant feature branch such as https://github.com/Elesbaan70/TMPE/tree/lane-grouping
• Final code cleanup
• Wiki articles for future development
• Mark #1559 ready for review

[Elesbaan70]

If anyone wants to follow the progress in code, it is here: https://github.com/Elesbaan70/TMPE/tree/xml-persistence

[krzychu124]

@krzychu124 finally got around to mentioning that there’s a compression library distributed with CS. Wish I’d known that earlier! LOL

I thought you’ve tried “obvious” solution and it hadn’t worked😂 I haven’t tried using it for other things than collecting files from disk to create a zip file.