Adaptive Feature Norm (AFN)¶
-
class
dalib.adaptation.afn.AdaptiveFeatureNorm(delta)[source]¶ The Stepwise Adaptive Feature Norm loss (ICCV 2019)
Instead of using restrictive scalar R to match the corresponding feature norm, Stepwise Adaptive Feature Norm is used in order to learn task-specific features with large norms in a progressive manner. Given feature representations \(f\) on source or target domain, the definition of Stepwise Adaptive Feature Norm loss is
\[\begin{split}norm\_loss = \mathbb{E}_{i}(\Vert f_i \Vert_2.detach() + delta - \Vert f_i \Vert_2)^2\\\end{split}\]- Parameters
delta (float) – positive residual scalar to control the feature norm enlargement.
- Inputs:
f (tensor): feature representations on source or target domain.
- Shape:
f: \((N, F)\) where F means the dimension of input features.
Outputs: scalar.
Examples:
>>> adaptive_feature_norm = AdaptiveFeatureNorm(delta=1) >>> f_s = torch.randn(32, 1000) >>> f_t = torch.randn(32, 1000) >>> norm_loss = adaptive_feature_norm(f_s) + adaptive_feature_norm(f_t)
-
class
dalib.adaptation.afn.Block(in_features, bottleneck_dim=1000, dropout_p=0.5)[source]¶ Basic building block for Image Classifier with structure: FC-BN-ReLU-Dropout. We use \(L_2\) preserved dropout layers. Given mask probability \(p\), input \(x_k\), generated mask \(a_k\), vanilla dropout layers calculate
\[\begin{split}\hat{x}_k = a_k\frac{1}{1-p}x_k\\\end{split}\]While in \(L_2\) preserved dropout layers
\[\begin{split}\hat{x}_k = a_k\frac{1}{\sqrt{1-p}}x_k\\\end{split}\]
-
class
dalib.adaptation.afn.ImageClassifier(backbone, num_classes, num_blocks=1, bottleneck_dim=1000, dropout_p=0.5, **kwargs)[source]¶ ImageClassifier for AFN.
- Parameters
backbone (torch.nn.Module) – Any backbone to extract 2-d features from data
num_classes (int) – Number of classes
num_blocks (int, optional) – Number of basic blocks. Default: 1
bottleneck_dim (int, optional) – Feature dimension of the bottleneck layer. Default: 1000
dropout_p (float, optional) – dropout probability. Default: 0.5
