=============================== Image Classification =============================== We provide benchmarks of different domain adaptation algorithms on `Digits`_, `Office-31`_ , `Office-Home`_, `VisDA-2017`_ and `DomainNet`_. Those domain adaptation algorithms includes: - :ref:`DANN` - :ref:`DAN` - :ref:`JAN` - :ref:`ADDA` - :ref:`CDAN` - :ref:`MCD` - :ref:`AFN` - :ref:`BSP` - :ref:`MDD` - :ref:`MCC` .. note:: - ``Origin`` means the accuracy reported by the original paper. - ``Avg`` is the accuracy reported by Trasnfer-Learn. - ``Source Only`` refers to the model trained with data from the source domain. - ``Oracle`` refers to the model trained with data from the target domain. .. note:: We found that the accuracies of adversarial methods (including DANN, ADDA, CDAN, MCD, BSP and MDD) are not stable even after the random seed is fixed, thus we repeat running adversarial methods on *Office-31* and *VisDA-2017* for three times and report their average accuracy. .. note:: `ADDA` with gradient reverse layer is frequently benchmarked in the literature. Therefore, we implement this baseline and use `ADDAgrl` to denote it below. .. _Digits: -------------------------------- Digits accuracy -------------------------------- =========== =========== =========== =========== Methods SVHN2MNIST MNIST2USPS USPS2MNIST Source Only 74.1 82.1 74.6 DANN 90.8 91.7 95.2 DAN 82.1 86.0 89.5 JAN 90.3 84.0 86.8 ADDA 93.3 94.5 98.3 CDAN 93.8 96.0 97.7 MCD 90.6 94.1 97.6 AFN 88.2 88.6 97.2 BSP+DANN 84.2 95.7 97.8 MDD 88.4 94.8 97.8 MCC 76.6 95.1 94.6 =========== =========== =========== =========== .. _Office-31: -------------------------------- Office-31 accuracy on ResNet-50 -------------------------------- =========== ====== ====== ====== ====== ====== ====== ====== ====== Methods Origin Avg A → W D → W W → D A → D D → A W → A Source Only 76.1 79.5 75.8 95.5 99.0 79.3 63.6 63.8 DANN 82.2 86.1 91.4 97.9 100.0 83.6 73.3 70.4 DAN 80.4 83.7 84.2 98.4 100.0 87.3 66.9 65.2 JAN 84.3 87.0 93.7 98.4 100.0 89.4 69.2 71.0 ADDA / 86.5 91.2 98.5 100.0 84.3 73.7 71.2 ADDAgrl / 87.3 94.6 97.5 99.7 90.0 69.6 72.5 CDAN 87.7 87.7 93.8 98.5 100.0 89.9 73.4 70.4 MCD / 85.4 90.4 98.5 100.0 87.3 68.3 67.6 AFN 85.7 88.6 94.0 98.9 100.0 94.4 72.9 71.1 BSP+DANN 87.7 87.8 92.7 97.9 100.0 88.2 74.1 73.8 MDD 88.9 89.6 95.6 98.6 100.0 94.4 76.6 72.2 MCC 89.4 89.6 94.1 98.4 99.8 95.6 75.5 74.2 =========== ====== ====== ====== ====== ====== ====== ====== ====== .. _Office-Home: ----------------------------------- Office-Home accuracy on ResNet-50 ----------------------------------- =========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= Methods Origin Avg Ar → Cl Ar → Pr Ar → Rw Cl → Ar Cl → Pr Cl → Rw Pr → Ar Pr → Cl Pr → Rw Rw → Ar Rw → Cl Rw → Pr Source Only 46.1 58.4 41.1 65.9 73.7 53.1 60.1 63.3 52.2 36.7 71.8 64.8 42.6 75.2 DANN 57.6 65.2 53.8 62.6 74.0 55.8 67.3 67.3 55.8 55.1 77.9 71.1 60.7 81.1 DAN 56.3 61.4 45.6 67.7 73.9 57.7 63.8 66.0 54.9 40.0 74.5 66.2 49.1 77.9 JAN 58.3 65.9 50.8 71.9 76.5 60.6 68.3 68.7 60.5 49.6 76.9 71.0 55.9 80.5 ADDA / 62.5 47.4 63.9 72.6 53.1 62.6 64.3 56.0 49.1 76.3 68.1 56.5 80.3 ADDAgrl / 65.6 52.6 62.9 74.0 59.7 68.0 68.8 61.4 52.5 77.6 71.1 58.6 80.2 CDAN 65.8 68.8 55.2 72.4 77.6 62.0 69.7 70.9 62.4 54.3 80.5 75.5 61.0 83.8 MCD / 67.8 51.7 72.2 78.2 63.7 69.5 70.8 61.5 52.8 78.0 74.5 58.4 81.8 AFN 67.3 68.2 53.2 72.7 76.8 65.0 71.3 72.3 65.0 51.4 77.9 72.3 57.8 82.4 BSP+DANN 64.9 67.6 54.7 67.7 76.2 61.0 69.4 70.9 60.9 55.2 80.2 73.4 60.3 81.2 MDD 68.1 69.7 56.2 75.4 79.6 63.5 72.1 73.8 62.5 54.8 79.9 73.5 60.9 84.5 MCC / 72.4 58.4 79.6 83.0 67.5 77.0 78.5 66.6 54.8 81.8 74.4 61.4 85.6 =========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= .. _VisDA-2017: ----------------------------------- VisDA-2017 accuracy ResNet-101 ----------------------------------- .. note:: - ``Origin`` means the accuracy reported by the original paper. - ``Mean`` refers to the accuracy average over ``classes`` - ``Avg`` refers to accuracy average over ``samples``. =========== ========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= Methods Origin Mean plane bcycl bus car horse knife mcycl person plant sktbrd train truck Avg Source Only 52.4 51.7 63.6 35.3 50.6 78.2 74.6 18.7 82.1 16.0 84.2 35.5 77.4 4.7 56.9 DANN 57.4 79.5 93.5 74.3 83.4 50.7 87.2 90.2 89.9 76.1 88.1 91.4 89.7 39.8 74.9 DAN 61.1 66.4 89.2 37.2 77.7 61.8 81.7 64.3 90.6 61.4 79.9 37.7 88.1 27.4 67.2 JAN / 73.4 96.3 66.0 82.0 44.1 86.4 70.3 87.9 74.6 83.0 64.6 84.5 41.3 70.3 ADDA / 79.3 93.6 70.8 83.2 63.5 90.6 93.2 89.0 75.3 88.4 79.3 87.4 37.2 76.4 ADDAgrl / 77.5 95.6 70.8 84.4 54.0 87.8 75.8 88.4 69.3 84.1 86.2 85.0 48.0 74.3 CDAN / 80.1 94.0 69.2 78.9 57.0 89.8 94.9 91.9 80.3 86.8 84.9 85.0 48.5 76.5 MCD 71.9 77.7 87.8 75.7 84.2 78.1 91.6 95.3 88.1 78.3 83.4 64.5 84.8 20.9 76.7 AFN 76.1 75.0 95.6 56.2 81.3 69.8 93.0 81.0 93.4 74.1 91.7 55.0 90.6 18.1 74.4 BSP+DANN 75.9 80.5 95.7 75.6 82.8 54.5 89.2 96.5 91.3 72.2 88.9 88.7 88.0 43.4 76.2 MDD / 82.0 88.3 62.8 85.2 69.9 91.9 95.1 94.4 81.2 93.8 89.8 84.1 47.9 79.8 MCC 78.8 83.6 95.3 85.8 77.1 68.0 93.9 92.9 84.5 79.5 93.6 93.7 85.3 53.8 80.4 =========== ========== ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= ======= .. _DomainNet: ----------------------------------- DomainNet accuracy on ResNet-101 ----------------------------------- =========== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== Methods c->p c->r c->s p->c p->r p->s r->c r->p r->s s->c s->p s->r Avg Source Only 32.7 50.6 39.4 41.1 56.8 35.0 48.6 48.8 36.1 49.0 34.8 46.1 43.3 DANN 37.9 54.3 44.4 41.7 55.6 36.8 50.7 50.8 40.1 55.0 45.0 54.5 47.2 DAN 38.8 55.2 43.9 45.9 59.0 40.8 50.8 49.8 38.9 56.1 45.9 55.5 48.4 JAN 40.5 56.7 45.1 47.2 59.9 43.0 54.2 52.6 41.9 56.6 46.2 55.5 50.0 ADDA 38.4 54.1 44.1 43.5 56.7 39.2 52.8 51.3 40.9 55.0 45.4 54.5 48.0 CDAN 40.4 56.8 46.1 45.1 58.4 40.5 55.6 53.6 43.0 57.2 46.4 55.7 49.9 MCD 37.5 52.9 44.0 44.6 54.5 41.6 52.0 51.5 39.7 55.5 44.6 52.0 47.5 MDD 42.9 59.5 47.5 48.6 59.4 42.6 58.3 53.7 46.2 58.7 46.5 57.7 51.8 MCC 37.7 55.7 42.6 45.4 59.8 39.9 54.4 53.1 37.0 58.1 46.3 56.2 48.9 =========== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== ====== Oracle DomainNet accuracy on ResNet-101 -------------------------------------------- =========== ====== ====== ====== ====== ====== ====== Oracle clp inf pnt real skt Avg / 78.2 40.7 71.6 83.8 70.6 69.0 =========== ====== ====== ====== ====== ====== ======