Add more detailed steps to update the dictionaries

Add software requirements and steps to update the dictionaries. This should help potential extra maintainers

I managed to get WiktionarySplitter.sh to start processing with the following steps. (I’m dumping this here for now in case it helps someone)

Using an Ubuntu 18.04 VM
# clone Dictionary
# clone DictionaryPC
$ apt install openjdk-11-jdk

$ ./compile.sh 
ICU4J needs to be installed
--> apt search ICU4J
--> apt install libicu4j-49-java

$ ./compile.sh 
Junit needs to be installed
---> sudo apt install junit

$ ./compile.sh 
Xerces needs to be installed
--> sudo apt install libxerces2-java


$ ./compile.sh 
commons-lang needs to be installed
---> apt install libcommons-lang3-java

$ ./compile.sh 
commons-compress needs to be installed
--> libcommons-compress-java

$ ./compile.sh 
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: ../Dictionary/Util/src/com/hughes/util/CachingList.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

---> change compile.sh to have:
javac -g -Xlint:deprecation -Xlint:unchecked ../Dictionary/Util/src/com/hughes/util/*.java ../Dictionary/Util/src/com/hughes/util/raf/*.java ../Dictionary/src/com/hughes/android/dictionary/DictionaryInfo.java ../Dictionary/src/com/hughes/android/dictionary/engine/*.java ../Dictionary/src/com/hughes/android/dictionary/C.java src/com/hughes/util/*.java src/com/hughes/android/dictionary/*.java src/com/hughes/android/dictionary/*/*.java src/com/hughes/android/dictionary/*/*/*.java -classpath "$ICU4J:$JUNIT:$XERCES:$COMMONS:$COMMONS_COMPRESS"

$ ./compile.sh 
../Dictionary/Util/src/com/hughes/util/CachingList.java:32: warning: [unchecked] unchecked cast
        chunked = useChunked ? (ChunkedList<T>)list : null;
                                               ^
  required: ChunkedList<T>
  found:    List<T>
  where T is a type-variable:
    T extends Object declared in class CachingList
src/com/hughes/util/MapUtil.java:39: warning: [deprecation] newInstance() in Class has been deprecated
                map.put(key, valueClass.newInstance());
                                       ^
  where T is a type-variable:
    T extends Object declared in class Class
src/com/hughes/android/dictionary/parser/wiktionary/WholeSectionToHtmlParser.java:399: warning: [deprecation] StringEscapeUtils in org.apache.commons.lang3 has been deprecated
        final String htmlEscaped = StringEscapeUtils.escapeHtml3(plainText);
                                   ^
3 warnings


# Note: there are 2048m of RAM in the VM used to test this
$ ./WiktionarySplitter.sh 
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.base/java.io.PipedInputStream.initPipe(PipedInputStream.java:161)
	at java.base/java.io.PipedInputStream.<init>(PipedInputStream.java:125)
	at com.hughes.android.dictionary.engine.WriteBuffer.<init>(WriteBuffer.java:28)
	at com.hughes.android.dictionary.engine.WiktionarySplitter.go(WiktionarySplitter.java:89)
	at com.hughes.android.dictionary.engine.WiktionarySplitter.main(WiktionarySplitter.java:60)
---> add '-Xmx2048m' to java command in ./WiktionarySplitter.sh
$ ./WiktionarySplitter.sh 
./WiktionarySplitter.sh: line 15:  7924 Killed                  "$JAVA" -Xverify:none -Xmx2048m -classpath src:../Util/src/:../Dictionary/src/:"$ICU4J":"$XERCES":"$COMMONS_COMPRESS" com.hughes.android.dictionary.engine.WiktionarySplitter "$@"
---> increase amount of ram in the VM to 4096m
---> add '-Xmx3072m' to java command in ./WiktionarySplitter.sh
$ ./WiktionarySplitter.sh

–> hitting issue https://github.com/rdoeffinger/Dictionary/issues/81

[rdoeffinger]

Here’s the Java code compiled to a native Linux binary. The latest git code is changed to use that binary if it exists. I have note tested compatibility with older Linux distributions etc. pp. or tested the scripts all that well, but if anyone wants to help, I think that’s likely the best approach available for better usability. I can’t generate Windows binary so far, though in theory it should be possible. But it should be DictionaryPC.zip possible run under WSL on Linux.

[kenden]

It would be helpful to indicate the requirements for development, maybe in the README.md, or in a new CONTRIBUTING.md.

Have things like:

Hardware requirements:

• 8 GB RAM minimum (maybe 5 is enough?)
• XX GB disk minimum

Software requirements:

• OS: Linux:

  • tested to work: Ubuntu XXX(16.04, 18.04…)

• Packages:

  • For Ubuntu 18.04

apt install 
  openjdk-11-jdk \
  libicu4j-49-java \
  junit \
  libxerces2-java \
  libcommons-lang3-java
  libcommons-compress-java