API cleanup

The API has become unweildy.

Apologies to anyone landing here, this bug is mostly a set of “note to self” items that will hopefully be implemented in the near future, with a major planned revamp of the API.

TODO:

• Create a new branch of both the code and the wiki
• Must get rid of all methods that take a Class<?> reference, and always classload by name, since this will minimize the number of cases of classes accidentally being loaded by the context classloader. This can prevent matched classes from being assignable to their superclass etc., or can result in a TypeNotPresentException. Also put a note in the docs that class names should not be derived from class references if a custom classloader will be created (either implicitly or explicitly, e.g. to handle Spring-Boot jars).
  • Also get rid of all MatchProcessors and MatchProcessorExceptions
• Now that temporary files are kept around, also run FileMatchProcessors on the ScanResult, not at scan time
• Add methods for getting list of visible modules in the system, to complement methods for getting classpath
• Expose methods for parsing Java type signatures / type descriptors and modifiers, since these are now exposed in JSON
• Create a single configuration call that takes an enum varargs parameter
• Remove all the direct-vs-indirect/meta methods (directly annotated by, etc.), make indirect the default (since this is almost always the right thing to do), and make direct an optional parameter.
• Unify field type scanning and static final field initializer scanning with FieldInfo
• Time all the scanning options (FieldInfo, MethodInfo etc.), and see if they should be enabled by default
• Remove deprecated methods
• Make strict mode the default when listing all classes in a package, and maybe get rid of the distinction between strict mode and non-strict mode (just use a sensible default for each match type)
• Get rid of MatchProcessors? Most of their code is now duplicated by ScanResult#classNamesToClassRefs(classNames) (double check there is no special handling in the code that calls any of the MatchProcessors)
  • Remove MatchProcessorException
• Unify handling of whitelisting and blacklisting, so that each resource type (jars, directories, modules, packages) is treated the same way, using the same data structures. Should be able to override system blacklist (e.g. blacklisting system modules by default) by specifically whitelisting a resource. If no whitelist is given, everything is whitelisted (except in the case of system resources, when the system blacklist is enabled – in that case, if the whitelist is empty, it doesn’t undo the default system blacklist by blacklisting everything). If any whitelist is given, only the whitelisted resources are scanned. Resources are only scanned if they are not in the blacklist. Specific whitelisted items may be in a blacklisted context, e.g. a whitelisted class in a blacklisted package.
• Make the API more fluent? e.g. ScanResult#<T>forClass(Class<T> class).getSubClasses()), where if the parameter to forClass() is a String, the result never calls the classloader (just returns strings or lists of strings), but if the parameter is a Class<T> reference, the result is a Class type (and the result is properly cast, wherever possible).
• Don’t expose classNameToClassInfo anymore, just make the user use ScanResult#forClass(), but add #forAllClasses() or something that returns a list of all classes.
• Also have ScanResult#forAnnotation(a)#getAnnotatedClasses() etc. – i.e. have some sort of class QueryClass, with subclasses QueryInterface, QueryAnnotation etc. (Be careful of the ability to use annotations as interfaces.) The methods of these query interfaces would return ClassInfo or Class<T> objects.
• Will need to rework all the unit tests, if the API is changed in this way.
• Make "!" and "!!" into config options, rather than scan spec parameters
• Make verbose() a config option
• Remove ability to use custom classloaders and custom classpaths? (Probably not, there was at least one user building their own dynamic classloader)
• Remove non-callback async scanning; use some sort of standard generic 1-arg functional interface types for the callback functions?
• Get rid of the duplication between the methods of FastClasspathScanner, ScanResult and ClassInfo. Each method should only be defined once, in one place.
• Move all the classpath-related functions (e.g. get classpath elements) into a new ClasspathInfo class.
• Remove deprecated methods
• Rather than return a List<String> of class names, return a ClassNameList that extends List<String> and is able to do its own classloading.
• Maybe get rid of jar whitelisting/blacklisting? Or at least give big caveats in the logs and the docs
• Get rid of classpath overriding? (But some users will still want to scan a custom path without doing classloading…) – so the ClassNameList objects for custom classes should refuse to load classes from an overridden classpath (this is currently the default, as of a recent commit).
• Don’t require classfile scanning – if only non-classfile matches are needed, then skip it entirely
• Don’t return external classes by default, only return classes in whitelisted packages.
• Don’t rely on initial uppercase letters to distinguish between packages and classes in the scan spec – have separate methods .whitelistPackage(...), .whitelistClass(...) etc.
• A ScanResult should contain only classes that matched filter criteria. The ScanResult should then generate a .dot file containing only matching classes. (#188)
• In all the *Info classes, the method for getting the name should just be getName(), not getClassName(), getMethodName(), etc. – check the rest of the API for consistency too.
• Is there any reason to ever return more than one context classloader to the user? The current API returns an array.

[lukehutch]

FCS should only scan the bare minimum for what is actually going to be used in the ScanResult. However, having to configure FCS before scanning, such that the required features are enabled and nothing more, would be a pain. Lazy configuration would be much better. Consequently, split up ScanResult into a separate result for each scan type. Maybe something like:

new FastClasspathScanner()
    .findClasses()  // Creates and returns a ClassMatch instance
        .inPackage("com.xyz").withSuperclass("com.xyz.Widget")
            .callMatchProcessor(c -> System.out.println(c.getClassName()))
    .and()  // Jumps fluid API back to FastClasspathScanner instance
    .findFiles()  // Creates and returns a FileMatch instance
        .withExtension("jpg")
            .callMatchProcessor(filename, imgBytes -> loadImage(imgBytes))
    .scan();

ClassMatch, FileMatch etc. all have a .scan() method, which calls FastClasspathScanner#scan(), and scan() will now have void type. The individual results of the scan will be obtained by calling ClassMatch#classNames(), FileMatch#fileNames() etc., which will throw an exception if #scan() has not yet been called (you have to add all the scan types before calling .scan() on any of them):

FastClasspathScanner fcs = new FastClasspathScanner();
ClassMatch cm = fcs.findClasses().inPackage("com.xyz").withSuperclass("com.xyz.Widget");
FileMatch = fcs.findFiles().withExtension("jpg");
fcs.scan();
List<String> classNames = cm.classNames();
List<String> imgFilenames = fm.fileNames();

[lukehutch]

All the items in this list have been implemented, and will be released in version 4 in the next couple of days.