question-mark

Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error on Importing a list of xml-files

See original GitHub issue

Issue Description

Good morning,

I have the task to import a long list of around 9000 xml files, all in the size range of 1 to 150MB.

I have encountered some errors while importing, and I can’t find a solution for them.

The most common error is the following:

[10:13:07 ERROR] Aborting import due to errors. [10:13:07 ERROR] Caused by: java.lang.NoClassDefFoundError: org/citydb/core/operation/common/cache/BranchCacheTable [10:13:07 ERROR] Caused by: java.lang.ClassNotFoundException: org.citydb.core.operation.common.cache.BranchCacheTable [10:13:07 WARN] Database import aborted. Exception in thread "main" java.lang.NoClassDefFoundError: picocli/CommandLine$IExitCodeGenerator at picocli.CommandLine$RunAll.recursivelyExtractExitCodeGenerators(CommandLine.java:2441) at picocli.CommandLine$RunAll.extractExitCodeGenerators(CommandLine.java:2438) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2180) at picocli.CommandLine$RunAll.execute(CommandLine.java:2391) at org.citydb.cli.ImpExpCli.process(ImpExpCli.java:305) at org.citydb.cli.ImpExpCli.start(ImpExpCli.java:157) at org.citydb.cli.ImpExpCli.main(ImpExpCli.java:149) Caused by: java.lang.ClassNotFoundException: picocli.CommandLine$IExitCodeGenerator at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 7 more

The error occurs mostly on the bigger files of the list (110MB+), but if I import them as a single file, everything runs fine. Apparently this error comes up when the Importer/Exporter is already running some time on other files, and then finally gets somehow overwhelmed from those bigger files. It is hard to investigate in this, as the errors only occur when the Importer is running for some time and on a bigger list, but the BranchCacheTable Keyword, has always been in the description of the error.

The error also happend while using the GUI, where it looked like this: [01:27:05 ERROR] Aborting import due to errors. [01:27:05 ERROR] Caused by: java.lang.NoClassDefFoundError: org/citydb/core/operation/common/cache/BranchCacheTable [01:27:05 ERROR] Caused by: java.lang.ClassNotFoundException: org.citydb.core.operation.common.cache.BranchCacheTable [01:27:05 WARN] Database import aborted. Exception in thread "AWT-EventQueue-0" java.lang.NoClassDefFoundError: com/formdev/flatlaf/MnemonicHandler$1 at com.formdev.flatlaf.MnemonicHandler.showMnemonics(MnemonicHandler.java:205) at com.formdev.flatlaf.MnemonicHandler.processKeyEventOnWindows(MnemonicHandler.java:125) at com.formdev.flatlaf.MnemonicHandler.postProcessKeyEvent(MnemonicHandler.java:82) at java.awt.DefaultKeyboardFocusManager.dispatchKeyEvent(Unknown Source) at java.awt.DefaultKeyboardFocusManager.preDispatchKeyEvent(Unknown Source) at java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(Unknown Source) at java.awt.DefaultKeyboardFocusManager.dispatchEvent(Unknown Source) at java.awt.Component.dispatchEventImpl(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Window.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.awt.EventQueue.access$500(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue$4.run(Unknown Source) at java.awt.EventQueue$4.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue.dispatchEvent(Unknown Source) at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.run(Unknown Source) Caused by: java.lang.ClassNotFoundException: com.formdev.flatlaf.MnemonicHandler$1 at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 29 more

Both error logs look somehow different on the paths, but happened on the same files and around the “expected” time (in the later part of a list, where bigger files were located.

Unfortunately I cannot provide the files, that are producing this error due to privacy reasons. But as the error is not occurring when I import the files one by one (and also the validation function returns that the files are valid), I hope to find a solution to this without the files.

If you need further information, I’ll respond as soon as possible. Thanks! I’m curious about your answer!

Little remark: It is somehow a little bit annoying, that the Import/Exporter does not follow the order of the files in the folder, that is selected. For me it seems like the order in which the Importer works through the list is somehow random, which makes it impossible to continue an import, that was interrupted by an error. If the Import had an option to import in an size-order or alphabetical-order, it would be easy to find the list of files, that are already imported or otherwise that are still remaining.

My setup: Importer/Exporter 5.1.0 used in CLI mode OS: Windows Server

Issue Analytics

  • State:closed
  • Created 10 months ago
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
clausnagelcommented, Mar 14, 2022

I guess your Out of Shared Memory error is thrown by your PostgreSQL database? You might need to increase the max_locks_per_transaction paramemter in your PostgreSQL configuration (see, for example, here). Alternatively, you can reduce the number of parallel workers and connections that are used by the Importer/Exporter. See the online manual and a similar discussion here.

1reaction
clausnagelcommented, Mar 9, 2022

Your issue looks strange because your JVM cannot find classes like BranchCacheTable which are contained in the JARs of the Importer/Exporter. These JARs are located in the lib folder inside the installation folder of the Importer/Exporter and the filenames start with impexp-*.jar.

Cache tables like BranchCacheTable store temporary information when importing/exporting datasets. The Importer/Exporter first holds this temporary information in main memory but writes it to the database (or a local cache) as soon as a predefined limit is reached to avoid out-of-memory situations. That’s why your issue only occurs after some time because only then these limits are reached. And obviously the limits are not reached when you only import single files.

You can set these limits under “Preferences -> Import -> Resources”. Check the “Cache settings” paragraph in this chapter of the online user manual. So, by reducing the numbers for the geometry and feature cache (e.g., to 20 or 50), you can force the Importer/Exporter to write the temporary information much earlier. And thus, your issue will occur much earlier. Just an idea to speed up your tests.

To resume an import operation, read this.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Import XML data - Microsoft Support
In the Import XML dialog box, locate and select the XML data file (.xml) you want to import, ... The following table lists...
Read more >
Getting Error while importing data from XML file - ServiceNow
Hello, I want create a data source for importing XML files. I have configured all but the file path is not working ....
Read more >
Error importing XML files - Software Support
Hi, I am currently migration a system between servers, I have created a new install of OJS 2.4.8. From my understanding the origin...
Read more >
Importing XML Data - LCUserManualV2
This message is displayed when there are fields in your spreadsheet (or data types in your XML file) that are not formatted correctly....
Read more >
Error when importing List type into xml file using Android Data ...
Error when importing List type into xml file using Android Data binding ; data> ; import type="android.view.View" /> ; import type="java.util.List" ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found