Error running astminer

Hello,

I downloaded JDK 14, gradle 6.3 on Windows 11 and succesfully built astminer at build/astminer/astminer.jar. I tried to run Python parser to generate code2vec paths but I got error:

$ ./cli.sh python.yaml Docker image not found, will use build/shadow/astminer.jar Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.UnsupportedClassVersionError: astminer/Main Kt has been compiled by a more recent version of the Java Runtime (class file ve rsion 55.0), this version of the Java Runtime only recognizes class file version s up to 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.security.SecureClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.access$100(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)

@SpirinEgor gave me Python .ymal file configuration:

# input directory (path to project)
inputDir: src/test/resources/
# output directory
outputDir: output

# parse Python files with ANTLR parser
parser:
  name: antlr
  languages: [py]

filters:
  - name: by tree size  # exclude the trees that have > 1000 nodes
    maxTreeSize: 1000

# use file names as labels
# this selects the file level granularity
label:
  name: function name

# extract from each tree paths with length 9 and width 2
# save paths in code2vec format
storage:
  name: code2vec
  maxPathLength: 9
  maxPathWidth: 2

[SpirinEgor]

44 rows mean 44 methods in python files under the passed inputDir path. It seems that you ran astminer on test data. You can traverse all .py files in /src/test/resources and count appear functions (all expressions started with def). There are exactly 44 functions.

Of course, the more files you provide the more lines in result files. I may advise you not to open such files with GUI editor. If you want to check a specific line, it’s better to use a combination of bash tools such as head and tail.

Also, since model training requires validation and test holdouts along with train data, astminer may create path contexts for each holdout based on the passed dataset. For this, split data in inputDir into three folders: train, val, and test. astminer will detect this and provide 3 path context files in the end.

Since your problem is solved, I close this issue. If you still have problems with running astminer don’t hesitate to reopen it. If you have any other problems or questions, feel free to open new issues.

[SpirinEgor]

This file contains python counters, and therefore it is not possible to save them from Kotlin. You can download preprocessed data, there are links in the readme file in the code2vec repository. It also contains this dict file so you can open it and see what is inside.

It’s better to open the issue in the code2vec repository and ask authors how to collect those counts. You should use train.c2s to collect them.