re-evaluate how syntax/software support is modeled

This issue is to continue discussion from #493 with @hawkeye116477 , @DandelionSprout , and whoever else.

FilterLists is currently designed so that each FilterList is linked to a single Syntax. Each Syntax is supported by multiple Software.

However, we might need to expand the complexity of that model a bit more. Some ideas summarized below:

0. Keep on Keeping on, but Stricter:

Continue using the current model, but be very strict about saying that a Software supports a Syntax. The example that came up is that we think AdGuard supports nearly all of uBlock Origin Static, however it does not support Scriptlet Injection. In this case, we would say that AdGuard does not support uBlock Origin Static. Only Software that can support any kind of rule defined in a Syntax should be linked to that Syntax.

1. Most Precise:

The most precise solution would be to drop the Syntax model altogether in favor of each FilterList having its own set of Software that is known to support it. That incurs a lot of data maintenance overhead, and I am really hoping to keep Syntax around in some form to lighten the load of having to maintain all those individual incompatibilities.

2. Boolean Partial Syntax Support:

I proposed adding a boolean flag to each SoftwareSyntax that represents if the Syntax is fully supported by the Software (all rules except comments/metadata should be applied by the Software) or only partially supported (some of the rules are ignored by the Software, while some are applied). We could then indicate in the UI somehow whether the Software fully or partially supports the FilterList.

3. More Granular Syntaxes:

Another option could be to chop a Syntax like uBlock Origin Static into multiple Syntaxes. So, instead of having uBlock Origin Static, we would instead have: uBlock Origin Static Network Filtering, uBlock Origin Static Extended Filtering, uBlock Origin Scriptlet Injection, … This solution would also require changing the relationship from a FilterList to a Syntax from one-to-many to many-to-many so that each FilterList can implement multiple Syntaxes.

Surfacing software support is one of the primary pieces of value that FilterLists provides, in my view, so getting this right is pretty key. Please provide your further thoughts/suggestions.

[collinbarrett]

new data model is fairly well fleshed out here. each list can now have many syntaxes so we can be a bit more granular.

I still need to document the changes. I hope to deploy in the next week or so. I don’t plan any major UI changes just yet other than to make sure nothing existing breaks with the new db/api launch.

it will also resolve:

• #503 (each entry in FilterListViewUrl.json can have a property called “SegmentNumber”. if a list is split across several “segments”, this allows multiple “viewUrls” to reflect that.
• #1231 / #696 there will be no limit on the number of mirrors the database can support. a new property call “primariness” in FilterListViewUrl.json indicates how primary each viewUrl is for the list. a value of “1” is the primary source. a secondary mirror has a value of “2”. etc.
• #697 a new property called “OnionUrl” gives each list the opportunity to have a tor url. (do we need more than one? do we need support for an OnionUrl for "viewUrl"s? not sure software supports subscribing to OnionUrls?)

once this new data model is deployed, I plan to shift some real effort towards #372 . it’s long overdue. I hate that people (mostly @DandelionSprout ) want to help maintain this project but they have to fiddle with json files. it’s very sub-optimal.

[ameshkov]

I also think that options 2 and 3 are better than the other two.

Option 3, however, requires quite a bit of work and cannot be done in a short time. At the same time, it will be really handy to filters maintainers and us developers, but it might be overly complicated and confusing for regular users.

Anyway, if you decide to proceed with option 3, please let me know, I’ll find some time to help.