Introduce discriminative NMF

Describe the workflow you want to enable

With the discriminative NMF, we can achieve matrix factorization, guided by (side) label information, so that the lower dimension representation might fit our needs better. An example would be to find contact networks within a relationship graph, (e.g. A knows B, B knows C, C knows D, A knows C etc.), we can then supply labels (such as A and B love horror films, C and D don’t like horror films) to guide the network identification. This will help us in both identifying contact networks, as well as which features are significant in determining these networks.

Describe your proposed solution

paper here

In this paper, we propose a novel constrained nonnegative matrix factorization algorithm, 
called the graph regularized discriminative non-negative matrix factorization (GDNMF), 
to incorporate into the NMF model both intrinsic geometrical structure and discriminative information 
which have been essentially ignored in prior works. Specifically, both the graph Laplacian and 
supervised label information are jointly utilized to learn the projection matrix in the new model. 
Further we provide the corresponding multiplicative update solutions for the optimization framework, 
together with the convergence proof.

Describe alternatives you’ve considered, if relevant

There are many variants of NMF and this seems to be the most solid one I have found.

Additional context

No response

[jasonyz-ucb]

Certainly, thanks for the questions.

Could you explain what are the practical benefits for those 2 methods?

The general theme here is to add discriminative feature to NMF, i.e. incorporating (side) labels into the factorization process. Taking dimensionality reduction as an example, it is often the case where it is hard to determine what lower dimensions will be achieved by NMF, while adding side labels, such as gender to a face recognition dataset, disease to a gene expression profile, will enable us to guide NMF to reduce the high dimension matrix towards the desired lower dimensions.

Do you use those methods to solve a particular problem?

This actually is where this comes from. I need such methods to analyze a gene expression profile of a group of subjects, to understand the network/cluster structure of the genes. NMF is fairly well adopted in gene expression network analysis, but no one has tried with discriminative variants, which should potentially bring significant improvement due to its ability to utilize critical side information. And on a larger scale, this still serves as an NMF method, so can be applied in (most) NMF applications.

In your experience what kind of problems can be effectively be tackled using those that could not be achieved using what’s already present in scikit-learn?

The problem mentioned above, i.e. I have some side labels (or features in a way) on the subjects, but there is no good way to use those in the network structure discovery problem.

Hope this makes sense and thanks very much indeed for your time. And I certainly understand your caution and outstanding work in maintaining this library. I will do my best to maintain my bits if it gets merged in as I do want to make sure it usable and useful for others:)

[jasonyz-ucb]

@ogrisel Sorry for another reminder, any updates?