Regressive Domain Adaptation (RegDA)

class dalib.adaptation.keypoint_detection.regda.PseudoLabelGenerator(num_keypoints, height=64, width=64, sigma=2)

Generate ground truth heatmap and ground false heatmap from a prediction.

Parameters

• num_keypoints (int) – Number of keypoints

• height (int) – height of the heatmap. Default: 64

• width (int) – width of the heatmap. Default: 64

• sigma (int) – sigma parameter when generate the heatmap. Default: 2

Inputs:

• y: predicted heatmap

Outputs:

• ground_truth: heatmap conforming to Gaussian distribution

• ground_false: ground false heatmap

Shape:

• y: \((minibatch, K, H, W)\) where K means the number of keypoints, H and W is the height and width of the heatmap respectively.

• ground_truth: \((minibatch, K, H, W)\)

• ground_false: \((minibatch, K, H, W)\)

class dalib.adaptation.keypoint_detection.regda.RegressionDisparity(pseudo_label_generator, criterion)

Regression Disparity proposed by Regressive Domain Adaptation for Unsupervised Keypoint Detection (CVPR 2021).

Parameters

• pseudo_label_generator (PseudoLabelGenerator) – generate ground truth heatmap and ground false heatmap from a prediction.

• criterion (torch.nn.Module) – the loss function to calculate distance between two predictions.

Inputs:

• y: output by the main head

• y_adv: output by the adversarial head

• weight (optional): instance weights

• mode (str): whether minimize the disparity or maximize the disparity. Choices includes min, max. Default: min.

Shape:

• y: \((minibatch, K, H, W)\) where K means the number of keypoints, H and W is the height and width of the heatmap respectively.

• y_adv: \((minibatch, K, H, W)\)

• weight: \((minibatch, K)\).

• Output: depends on the criterion.

Examples:

>>> num_keypoints = 5
>>> batch_size = 10
>>> H = W = 64
>>> pseudo_label_generator = PseudoLabelGenerator(num_keypoints)
>>> from common.vision.models.keypoint_detection.loss import JointsKLLoss
>>> loss = RegressionDisparity(pseudo_label_generator, JointsKLLoss())
>>> # output from source domain and target domain
>>> y_s, y_t = torch.randn(batch_size, num_keypoints, H, W), torch.randn(batch_size, num_keypoints, H, W)
>>> # adversarial output from source domain and target domain
>>> y_s_adv, y_t_adv = torch.randn(batch_size, num_keypoints, H, W), torch.randn(batch_size, num_keypoints, H, W)
>>> # minimize regression disparity on source domain
>>> output = loss(y_s, y_s_adv, mode='min')
>>> # maximize regression disparity on target domain
>>> output = loss(y_t, y_t_adv, mode='max')

class dalib.adaptation.keypoint_detection.regda.PoseResNet(backbone, upsampling, feature_dim, num_keypoints, gl=None, finetune=True, num_head_layers=2)

Pose ResNet for RegDA has one backbone, one upsampling, while two regression heads.

Parameters

• backbone (torch.nn.Module) – Backbone to extract 2-d features from data

• upsampling (torch.nn.Module) – Layer to upsample image feature to heatmap size

• feature_dim (int) – The dimension of the features from upsampling layer.

• num_keypoints (int) – Number of keypoints

• gl (WarmStartGradientLayer) –

• finetune (bool, optional) – Whether use 10x smaller learning rate in the backbone. Default: True

• num_head_layers (int) – Number of head layers. Default: 2

Inputs:

• x (tensor): input data

Outputs:

• outputs: logits outputs by the main regressor

• outputs_adv: logits outputs by the adversarial regressor

Shapes:

• x: \((minibatch, *)\), same shape as the input of the backbone.

• outputs, outputs_adv: \((minibatch, K, H, W)\), where K means the number of keypoints.

Note

Remember to call function step() after function forward() during training phase! For instance,

>>> # x is inputs, model is an PoseResNet
>>> outputs, outputs_adv = model(x)
>>> model.step()