Allow `create_config_file` to work with an external package

Description

[EDIT] : see https://github.com/astropy/astropy/issues/12960#issuecomment-1072537490 before reading further.

I was unable to use create_config_file with configuration options defined in a package outside of astropy. Maybe there is a trick that I didn’t figure out, but I think there is a fundamental problem in generate_config() where it is getting the configuration options based on a hardwired ConfigNamespace which always points to the built-in astropy class instead of a user-defined version as documented here: https://docs.astropy.org/en/stable/config/index.html#customizing-config-location-in-affiliated-packages.

        # Parse the subclasses, ordered by their module name
        subclasses = ConfigNamespace.__subclasses__()  # <<== symbol is defined in same configuration.py file.
        processed = set()

I’d propose to make create_config_file and generate_config be class methods of ConfigNamespace so that the above would become:

        # Parse the subclasses, ordered by their module name
        subclasses = cls.__subclasses__()
        processed = set()

Before doing this I’d like input from a package maintainer that this is on the right track and reasonable.

[taldcroft]

After quite a circuitous path I have a better understanding of this now and the root cause of problems. Basically all the business about needing class method(s) was a red herring.

The real problem is that the code to find all the subclasses of the astropy ConfigNamespace stops at one subclass deep with ConfigNamespace.__subclasses(). But the example given in the docs has the user first create a subclass where the rootname class attribute is set and then subsequent creation of class Conf(ConfigNamespace): # from the package-specific subclass means that the package conf object is 2-subclasses deep and gets lost.

Instead I put in this code:

diff --git a/astropy/config/configuration.py b/astropy/config/configuration.py
index 3ebf6c67b0..6979136ca2 100644
--- a/astropy/config/configuration.py
+++ b/astropy/config/configuration.py
@@ -643,8 +643,13 @@ def generate_config(pkgname='astropy', filename=None, verbose=False):
             # assume it's a file object, or io.StringIO
             fp = filename
 
-        # Parse the subclasses, ordered by their module name
-        subclasses = ConfigNamespace.__subclasses__()
+        # Parse the subclasses, ordered by their module name. From
+        # https://stackoverflow.com/questions/3862310
+        def all_subclasses(cls):
+            return set(cls.__subclasses__()).union(
+                [s for c in cls.__subclasses__() for s in all_subclasses(c)])
+
+        subclasses = all_subclasses(ConfigNamespace)
         processed = set()

So that is just a bug and should be fixed, but I don’t have time right now to get to it.

Another mistake is in the docs. They recommend this pattern:

import astropy.config as astropyconfig

class ConfigNamespace(astropyconfig.ConfigNamespace):
    rootname = 'packagename'

class ConfigItem(astropyconfig.ConfigItem):
    rootname = 'packagename'

But it turns out the ConfigNamespace class never uses that rootname attribute in any way, there is no need to make that package-specific subclass in the first place! If you don’t do that (i.e. class Conf(astropy.config.ConfigNamespace):, then the subclasses() bug above doesn’t apply and everything just works with the current release.

Phew. So someone should fix the docs and that subclasses problem… but in the meantime I need to get back to just using astropy config instead of fixing it.

[taldcroft]

OK, I was planning a PR.