Representation Learning¶
Correlation Alignment for Deep Domain Adaptation (Deep CORAL)¶
-
class
dglib.generalization.coral.CorrelationAlignmentLoss[source]¶ The Correlation Alignment Loss in Deep CORAL: Correlation Alignment for Deep Domain Adaptation (ECCV 2016).
Given source features \(f_S\) and target features \(f_T\), the covariance matrices are given by
\[C_S = \frac{1}{n_S-1}(f_S^Tf_S-\frac{1}{n_S}(\textbf{1}^Tf_S)^T(\textbf{1}^Tf_S))\]\[C_T = \frac{1}{n_T-1}(f_T^Tf_T-\frac{1}{n_T}(\textbf{1}^Tf_T)^T(\textbf{1}^Tf_T))\]where \(\textbf{1}\) denotes a column vector with all elements equal to 1, \(n_S, n_T\) denotes number of source and target samples, respectively. We use \(d\) to denote feature dimension, use \({\Vert\cdot\Vert}^2_F\) to denote the squared matrix Frobenius norm. The correlation alignment loss is given by
\[l_{CORAL} = \frac{1}{4d^2}\Vert C_S-C_T \Vert^2_F\]- Inputs:
f_s (tensor): feature representations on source domain, \(f^s\)
f_t (tensor): feature representations on target domain, \(f^t\)
- Shape:
f_s, f_t: \((N, d)\) where d means the dimension of input features, \(N=n_S=n_T\) is mini-batch size.
Outputs: scalar.
Note
Under domain generalization setting, we can’t access to samples from target domain during training. Therefore, two different domains from source domains are treated as source domain and target domain respectively to calculate Correlation Alignment Loss.
