A <mapref> with a @keys attribute is processed incorrectly

Expected Behavior

If <mapref> elements have the @keys attribute defined:

then publishing should yield the same results as if @keys was not defined.

Actual Behavior

The content is published incorrectly in the preprocess2 pipeline, as described below. (preprocess is also affected when force-unique is used - see comments below this description.)

During the mapref preprocessing stage, the presence of the @keys attribute causes a <keydef> element to be created in addition to a <submap> element for the referenced map contents:

During the branch-filter preprocessing stage, the profiled “Product” key values are correctly pruned to reflect the <ditavalref>:

During the keyref preprocessing stage, the <keydef> element is incorrectly processed to insert a redundant (and unfiltered) <submap> of “Product” values:

When preprocess2 is used, because topics are processed after maps, the unfiltered “Product” key values are incorrectly used to resolve the key reference in the topic file, causing the ***ERROR*** value to be used in the output.

(Note that an extra preprocessing-only topic reference to preface.dita is also created, but it does not affect output in this testcase.)

Possible Solution

Somehow the second insertion of the <submap> (due to the <keydef>) must be prevented. I don’t know enough about the keyref stage to know what element structures should/should not be created.

Steps to Reproduce

1. Unarchive the following testcase: ditaot_mapref_keys_bug.zip
2. Run the pdf2 transformation: dita -i map.ditamap -f pdf2

Environment

• DITA-OT version: 3.7
• Operating system and version: Ubuntu 20.04
• How did you run DITA-OT? dita command
• Transformation types: pdf2

[chrispy-snps]

@SoonBoonChin - thank you for sharing your findings! I am always hesitant to change the default ordering of preprocessing steps for our production processing, but I think it’s good information that the cause of the bug is some kind of order dependency.

@raducoravu - interesting! I took a closer look. In the map-keyref figure in the issue description, the questionable submap copy of preface.ditamap (shown in red) has processing-role="resource-only", which causes the duplicate topics. The fix suppresses these duplicates, which is a nice practical benefit until the @keys processing itself can be fixed.

I also confirmed that even with the #3815/#4003 fix, the preprocess2 bug for incorrect key resolution still occurs, as it is an independent bug caused by the duplicate <submap>.

[raducoravu]

@SoonBoonChin you can open a pull request yourself, anybody can open pull requests, you do not need to be a project admin for this: https://www.dita-ot.org/contributing#coding

We are not using the “preprocess2” stage by default with the PDF (CSS based on XSL-FO based) and WebHelp outputs generated from the DITA OT bundled with Oxygen. And the “preprocess” stage no longer has this problem after the fix I made here: https://github.com/dita-ot/dita-ot/pull/4003 So I’m not directly interested in further investigating a possible fix for this.

Also I believe the possible fix where you do keyref processing before branch filtering has the potential to generate side effects, even if all the auto tests will work.