Backwards incompatibility in v3.3.2 alternative handling

I have a Lektor site that deploys fine on Lector 3.3.1; upgrading to 3.3.2 causes multiple failures due to what appears to be an issue with default alternative lookup.

Every page on the site has a contents.lr, and en is set as the primary language. Partial translations for the site exist in many languages (ar, es, pt, zh, fr, it, pl). However, whenever a translation for a specific language isn’t present, it looks for contents+en.lr - a file that doesn’t exist - and the build fails as a result.

Build Log excerpt:

2022-04-06T01:44:17.2474088Z Started build
2022-04-06T01:44:17.2640283Z U ar_AR/index.html
2022-04-06T01:44:17.5287436Z U cs_CZ/index.html
2022-04-06T01:44:17.5820897Z   E cs_CZ/index.html (FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/beeware.github.io/beeware.github.io/content/contents+en.lr')
2022-04-06T01:44:17.5924754Z U da_DK/index.html
2022-04-06T01:44:17.5983828Z   E da_DK/index.html (FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/beeware.github.io/beeware.github.io/content/contents+en.lr')
2022-04-06T01:44:17.6085304Z U de_DE/index.html
2022-04-06T01:44:17.7004800Z U index.html
2022-04-06T01:44:17.7845988Z U es/index.html
2022-04-06T01:44:17.8958702Z U fr_FR/index.html
2022-04-06T01:44:17.9884367Z U it_IT/index.html
2022-04-06T01:44:17.9939498Z   E it_IT/index.html (FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/beeware.github.io/beeware.github.io/content/contents+en.lr')
2022-04-06T01:44:18.0039511Z U ko_KR/index.html
2022-04-06T01:44:18.0093995Z   E ko_KR/index.html (FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/beeware.github.io/beeware.github.io/content/contents+en.lr')
2022-04-06T01:44:18.0193713Z U pl_PL/index.html
2022-04-06T01:44:18.1121850Z U pt_BR/index.html
2022-04-06T01:44:18.1980698Z U tr_TR/index.html
2022-04-06T01:44:18.2898397Z U zh_TW/index.html
2022-04-06T01:44:18.4029289Z U ar_AR/b/index.html
2022-04-06T01:44:18.4506113Z U ar_AR/batavia/index.html
2022-04-06T01:44:18.4663084Z   E ar_AR/batavia/index.html (FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/beeware.github.io/beeware.github.io/content/batavia/contents+en.lr')
2022-04-06T01:44:18.4701624Z U ar_AR/cricket/index.html
2022-04-06T01:44:18.4838880Z   E ar_AR/cricket/index.html (FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/beeware.github.io/beeware.github.io/content/cricket/contents+en.lr')
2022-04-06T01:44:18.4877632Z U ar_AR/colosseum/index.html
2022-04-06T01:44:18.5103121Z   E ar_AR/colosseum/index.html (FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/beeware.github.io/beeware.github.io/content/colosseum/contents+en.lr')
2022-04-06T01:44:18.5141530Z U ar_AR/signup/index.html
...

Full log can be found in the CI builds for this pull request.

As best as I can work out, the bug is related to https://github.com/lektor/lektor/pull/958 - however, the commentary on that and related tickets doesn’t suggest there is any possible backwards incompatibility or migration required (in fact, the commentary on https://github.com/lektor/lektor/issues/965 seems to specifically dismiss the possibility of backwards incompatibilities).

[dairiki]

Lektor could merge the two layers into one before returning .contents but then again, as you said, it is not used by Lektor and could be deprecated (I dont use it myself) … in line with the slogan “and lead us not into temptation”.

I guess that would be a fix. It’s slightly involved since Page.contents returns a FileContents instance which has a bunch of methods and properties we’d have to support in order to be backwards compatible. And it’s still not fully backwards compatible (though I think it is in the Beeware case, where all of there overlay files include a full copy of all the fallback fields.) What if some really only wanted the data from the overlay file (rather than the merged contents)?

(We could do that to fix the issue and deprecate it.)

And for source_filename, how about returning the list of sources source_filenames?

That already exists in the iter_source_filenames method. 😃

[relikd]

If one uses the Lektor admin web UI to construct content for a non-primary alt, generally the alt-specific contents file will only contain those fields which have been specifically edited in the admin UI. The values for the rest of the fields will still come from the fallback contents.lr file.

Came to say the same thing. Adding myself here as an “edge case” 😄 (btw. I am very glad that Lektor is using this layered view on source files. Back in 3.2.0 (or earlier) it duplicated all the fields from the fallback. I am happy that this is not the case anymore. Should this close #711?).

Lektor could merge the two layers into one before returning .contents but then again, as you said, it is not used by Lektor and could be deprecated (I dont use it myself) … in line with the slogan “and lead us not into temptation”.

And for source_filename, how about returning the list of sources source_filenames?