Why is the HSIC not minimized but maximized?
See original GitHub issueThank you for such a great work! When reading the paper and code, I have the following questions.
According to the definition of HSIC, it measures the level of independence and HSIC(U, V) = 0
indicates that U and V are independent. A larger HSIC value indicates that U and V are dependent to some extent.
So, to debias the representation of network f by using a biased network g, shouldn’t we minimize HSIC(f, g)?
Besides, the for loop in line 62 seems redundant because the g_dim
will be over-written by the last loop, right?
Looking forward to your reply. Thanks!
Issue Analytics
- State:
- Created 3 years ago
- Comments:5
Top Results From Across the Web
Cannot click taskbar or window minimize, maximize, and close ...
Since the last week the computer has gone into a mode where the taskbar and window interaction icons cannot be clicked.
Read more >Feature Selection via Dependence Maximization
Like the mutual in- formation, HSIC is a nonparametric dependence measure, which takes into account all modes of dependence between the variables (not...
Read more >Eliminating accidental deviations to minimize ... - PLOS
Eliminating accidental deviations to minimize generalization error and maximize replicability: Applications in connectomics and genomics.
Read more >Self-Supervised Learning with Kernel Dependence ... - arXiv
SSL-HSIC maximizes dependence between representations of transformations of an image and the image identity, while minimizing the kernelized ...
Read more >Self-Supervised Learning with an Information Maximization ...
Correlative information maximization among alternative latent ... It's true in computer vision, but not true in speech or natural language ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@Cogito2012 In theory, you have to alternatively update to correctly solve a minimax problem, which aims to find a saddle point:![](https://upload.wikimedia.org/wikipedia/commons/thumb/1/1e/Saddle_point.svg/1200px-Saddle_point.svg.png)
If you jointly optimize min and max problems at the same time, it will not guarantee a correct saddle point. I even cannot sure that such optimization will be converged to a certain point.
Note that our conceptual objective function is
which have to solve alternative updates for f and g, respectively.
Such minimax problem is popular in many machine learning methods such as
Let me rephrase my concern. Can we simplify the alternative updating as minimizing the objective?
where
\fixed{f}
or\fixed{g}
means the network parameters off
org
will not be updated by corresponding HSIC term.If this is also working, then this kind of ReBias variant may be even easier to use.