if parallel=true, some files are not rendered

Expected Behavior

all files are rendered

Actual Behavior

some files are skipped with error “folder could not be created”. seemingly randomly in documents, but consistently between builds

Steps to Reproduce

render a document with a lot of topics in the same subfolder (?)

Copy of the error message, log file or stack trace

[xslt] Failed to transform document: Failed to transform document: Failed to create directory

Environment

• DITA-OT version: 3.7.1, 3.6.1
• Operating system and version: Linux, Windows
• How did you run DITA-OT? dita command
• Transformation type: HTML5

[chrispy-snps]

I found that for N topics, if I edit “do.sh” to put sqrt(N) topics in each directory:

# change this to the number of topics and topics-in-group you want
./make_topics.pl 1000 33

then the probability for collision is maximized:

% do.sh
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/826_to_858
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/397_to_429
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/727_to_759
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/397_to_429
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/793_to_825
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/826_to_858
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/1_to_33
993
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/826_to_858
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/397_to_429
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/397_to_429
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/364_to_396
996
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/67_to_99
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/826_to_858
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/826_to_858
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/397_to_429
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/463_to_495
     [xslt] Failed to transform document: Failed to transform document: Failed to create directory /chrispy/dita/out/397_to_429
994

thus making a more effective testcase. And with this configuration, even my 12-core machine starts giving occasional failures too.

@raducoravu - with your suggested fix, the issue is completely resolved on both machines:

% do.sh
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
...continues...

@jelovirt - Radu’s suggested fix is a one-word addition to line 238 at

https://github.com/dita-ot/dita-ot/blob/hotfix/3.7.2/src/main/java/org/dita/dost/store/StreamStore.java#L238

Do you want me to make a pull request, or is this simple enough to fix with a bare commit?

[raducoravu]

In general in our Oxygen user’s guide at least the parallel processing has not brought any significant time decrease so I have not used it much. The store-type=memory seems to help quite a lot on Windows though.