Image Classification¶
We provide benchmarks of different domain adaptation algorithms on Digits, Office-31 , Office-Home, VisDA-2017 and DomainNet. Those domain adaptation algorithms includes:
Note
Originmeans the accuracy reported by the original paper.Avgis the accuracy reported by Trasnfer-Learn.Source Onlyrefers to the model trained with data from the source domain.Oraclerefers to the model trained with data from the target domain.
Note
We found that the accuracies of adversarial methods (including DANN, ADDA, CDAN, MCD, BSP and MDD) are not stable even after the random seed is fixed, thus we repeat running adversarial methods on Office-31 and VisDA-2017 for three times and report their average accuracy.
Note
ADDA with gradient reverse layer is frequently benchmarked in the literature. Therefore, we implement this baseline and use ADDAgrl to denote it below.
Digits accuracy¶
Methods |
SVHN2MNIST |
MNIST2USPS |
USPS2MNIST |
Source Only |
74.1 |
82.1 |
74.6 |
DANN |
90.8 |
91.7 |
95.2 |
DAN |
82.1 |
86.0 |
89.5 |
JAN |
90.3 |
84.0 |
86.8 |
ADDA |
93.3 |
94.5 |
98.3 |
CDAN |
93.8 |
96.0 |
97.7 |
MCD |
90.6 |
94.1 |
97.6 |
AFN |
88.2 |
88.6 |
97.2 |
BSP+DANN |
84.2 |
95.7 |
97.8 |
MDD |
88.4 |
94.8 |
97.8 |
MCC |
76.6 |
95.1 |
94.6 |
Office-31 accuracy on ResNet-50¶
Methods |
Origin |
Avg |
A → W |
D → W |
W → D |
A → D |
D → A |
W → A |
Source Only |
76.1 |
79.5 |
75.8 |
95.5 |
99.0 |
79.3 |
63.6 |
63.8 |
DANN |
82.2 |
86.1 |
91.4 |
97.9 |
100.0 |
83.6 |
73.3 |
70.4 |
DAN |
80.4 |
83.7 |
84.2 |
98.4 |
100.0 |
87.3 |
66.9 |
65.2 |
JAN |
84.3 |
87.0 |
93.7 |
98.4 |
100.0 |
89.4 |
69.2 |
71.0 |
ADDA |
/ |
86.5 |
91.2 |
98.5 |
100.0 |
84.3 |
73.7 |
71.2 |
ADDAgrl |
/ |
87.3 |
94.6 |
97.5 |
99.7 |
90.0 |
69.6 |
72.5 |
CDAN |
87.7 |
87.7 |
93.8 |
98.5 |
100.0 |
89.9 |
73.4 |
70.4 |
MCD |
/ |
85.4 |
90.4 |
98.5 |
100.0 |
87.3 |
68.3 |
67.6 |
AFN |
85.7 |
88.6 |
94.0 |
98.9 |
100.0 |
94.4 |
72.9 |
71.1 |
BSP+DANN |
87.7 |
87.8 |
92.7 |
97.9 |
100.0 |
88.2 |
74.1 |
73.8 |
MDD |
88.9 |
89.6 |
95.6 |
98.6 |
100.0 |
94.4 |
76.6 |
72.2 |
MCC |
89.4 |
89.6 |
94.1 |
98.4 |
99.8 |
95.6 |
75.5 |
74.2 |
Office-Home accuracy on ResNet-50¶
Methods |
Origin |
Avg |
Ar → Cl |
Ar → Pr |
Ar → Rw |
Cl → Ar |
Cl → Pr |
Cl → Rw |
Pr → Ar |
Pr → Cl |
Pr → Rw |
Rw → Ar |
Rw → Cl |
Rw → Pr |
Source Only |
46.1 |
58.4 |
41.1 |
65.9 |
73.7 |
53.1 |
60.1 |
63.3 |
52.2 |
36.7 |
71.8 |
64.8 |
42.6 |
75.2 |
DANN |
57.6 |
65.2 |
53.8 |
62.6 |
74.0 |
55.8 |
67.3 |
67.3 |
55.8 |
55.1 |
77.9 |
71.1 |
60.7 |
81.1 |
DAN |
56.3 |
61.4 |
45.6 |
67.7 |
73.9 |
57.7 |
63.8 |
66.0 |
54.9 |
40.0 |
74.5 |
66.2 |
49.1 |
77.9 |
JAN |
58.3 |
65.9 |
50.8 |
71.9 |
76.5 |
60.6 |
68.3 |
68.7 |
60.5 |
49.6 |
76.9 |
71.0 |
55.9 |
80.5 |
ADDA |
/ |
62.5 |
47.4 |
63.9 |
72.6 |
53.1 |
62.6 |
64.3 |
56.0 |
49.1 |
76.3 |
68.1 |
56.5 |
80.3 |
ADDAgrl |
/ |
65.6 |
52.6 |
62.9 |
74.0 |
59.7 |
68.0 |
68.8 |
61.4 |
52.5 |
77.6 |
71.1 |
58.6 |
80.2 |
CDAN |
65.8 |
68.8 |
55.2 |
72.4 |
77.6 |
62.0 |
69.7 |
70.9 |
62.4 |
54.3 |
80.5 |
75.5 |
61.0 |
83.8 |
MCD |
/ |
67.8 |
51.7 |
72.2 |
78.2 |
63.7 |
69.5 |
70.8 |
61.5 |
52.8 |
78.0 |
74.5 |
58.4 |
81.8 |
AFN |
67.3 |
68.2 |
53.2 |
72.7 |
76.8 |
65.0 |
71.3 |
72.3 |
65.0 |
51.4 |
77.9 |
72.3 |
57.8 |
82.4 |
BSP+DANN |
64.9 |
67.6 |
54.7 |
67.7 |
76.2 |
61.0 |
69.4 |
70.9 |
60.9 |
55.2 |
80.2 |
73.4 |
60.3 |
81.2 |
MDD |
68.1 |
69.7 |
56.2 |
75.4 |
79.6 |
63.5 |
72.1 |
73.8 |
62.5 |
54.8 |
79.9 |
73.5 |
60.9 |
84.5 |
MCC |
/ |
72.4 |
58.4 |
79.6 |
83.0 |
67.5 |
77.0 |
78.5 |
66.6 |
54.8 |
81.8 |
74.4 |
61.4 |
85.6 |
VisDA-2017 accuracy ResNet-101¶
Note
Originmeans the accuracy reported by the original paper.Meanrefers to the accuracy average overclassesAvgrefers to accuracy average oversamples.
Methods |
Origin |
Mean |
plane |
bcycl |
bus |
car |
horse |
knife |
mcycl |
person |
plant |
sktbrd |
train |
truck |
Avg |
Source Only |
52.4 |
51.7 |
63.6 |
35.3 |
50.6 |
78.2 |
74.6 |
18.7 |
82.1 |
16.0 |
84.2 |
35.5 |
77.4 |
4.7 |
56.9 |
DANN |
57.4 |
79.5 |
93.5 |
74.3 |
83.4 |
50.7 |
87.2 |
90.2 |
89.9 |
76.1 |
88.1 |
91.4 |
89.7 |
39.8 |
74.9 |
DAN |
61.1 |
66.4 |
89.2 |
37.2 |
77.7 |
61.8 |
81.7 |
64.3 |
90.6 |
61.4 |
79.9 |
37.7 |
88.1 |
27.4 |
67.2 |
JAN |
/ |
73.4 |
96.3 |
66.0 |
82.0 |
44.1 |
86.4 |
70.3 |
87.9 |
74.6 |
83.0 |
64.6 |
84.5 |
41.3 |
70.3 |
ADDA |
/ |
79.3 |
93.6 |
70.8 |
83.2 |
63.5 |
90.6 |
93.2 |
89.0 |
75.3 |
88.4 |
79.3 |
87.4 |
37.2 |
76.4 |
ADDAgrl |
/ |
77.5 |
95.6 |
70.8 |
84.4 |
54.0 |
87.8 |
75.8 |
88.4 |
69.3 |
84.1 |
86.2 |
85.0 |
48.0 |
74.3 |
CDAN |
/ |
80.1 |
94.0 |
69.2 |
78.9 |
57.0 |
89.8 |
94.9 |
91.9 |
80.3 |
86.8 |
84.9 |
85.0 |
48.5 |
76.5 |
MCD |
71.9 |
77.7 |
87.8 |
75.7 |
84.2 |
78.1 |
91.6 |
95.3 |
88.1 |
78.3 |
83.4 |
64.5 |
84.8 |
20.9 |
76.7 |
AFN |
76.1 |
75.0 |
95.6 |
56.2 |
81.3 |
69.8 |
93.0 |
81.0 |
93.4 |
74.1 |
91.7 |
55.0 |
90.6 |
18.1 |
74.4 |
BSP+DANN |
75.9 |
80.5 |
95.7 |
75.6 |
82.8 |
54.5 |
89.2 |
96.5 |
91.3 |
72.2 |
88.9 |
88.7 |
88.0 |
43.4 |
76.2 |
MDD |
/ |
82.0 |
88.3 |
62.8 |
85.2 |
69.9 |
91.9 |
95.1 |
94.4 |
81.2 |
93.8 |
89.8 |
84.1 |
47.9 |
79.8 |
MCC |
78.8 |
83.6 |
95.3 |
85.8 |
77.1 |
68.0 |
93.9 |
92.9 |
84.5 |
79.5 |
93.6 |
93.7 |
85.3 |
53.8 |
80.4 |
DomainNet accuracy on ResNet-101¶
Methods |
c->p |
c->r |
c->s |
p->c |
p->r |
p->s |
r->c |
r->p |
r->s |
s->c |
s->p |
s->r |
Avg |
Source Only |
32.7 |
50.6 |
39.4 |
41.1 |
56.8 |
35.0 |
48.6 |
48.8 |
36.1 |
49.0 |
34.8 |
46.1 |
43.3 |
DANN |
37.9 |
54.3 |
44.4 |
41.7 |
55.6 |
36.8 |
50.7 |
50.8 |
40.1 |
55.0 |
45.0 |
54.5 |
47.2 |
DAN |
38.8 |
55.2 |
43.9 |
45.9 |
59.0 |
40.8 |
50.8 |
49.8 |
38.9 |
56.1 |
45.9 |
55.5 |
48.4 |
JAN |
40.5 |
56.7 |
45.1 |
47.2 |
59.9 |
43.0 |
54.2 |
52.6 |
41.9 |
56.6 |
46.2 |
55.5 |
50.0 |
ADDA |
38.4 |
54.1 |
44.1 |
43.5 |
56.7 |
39.2 |
52.8 |
51.3 |
40.9 |
55.0 |
45.4 |
54.5 |
48.0 |
CDAN |
40.4 |
56.8 |
46.1 |
45.1 |
58.4 |
40.5 |
55.6 |
53.6 |
43.0 |
57.2 |
46.4 |
55.7 |
49.9 |
MCD |
37.5 |
52.9 |
44.0 |
44.6 |
54.5 |
41.6 |
52.0 |
51.5 |
39.7 |
55.5 |
44.6 |
52.0 |
47.5 |
MDD |
42.9 |
59.5 |
47.5 |
48.6 |
59.4 |
42.6 |
58.3 |
53.7 |
46.2 |
58.7 |
46.5 |
57.7 |
51.8 |
MCC |
37.7 |
55.7 |
42.6 |
45.4 |
59.8 |
39.9 |
54.4 |
53.1 |
37.0 |
58.1 |
46.3 |
56.2 |
48.9 |
Oracle DomainNet accuracy on ResNet-101¶
Oracle |
clp |
inf |
pnt |
real |
skt |
Avg |
/ |
78.2 |
40.7 |
71.6 |
83.8 |
70.6 |
69.0 |
