Results on the classification challenge

NOTE: After the challenge in July 2017, we updated the "baseline" methods [159,160,161,162,163,164] for which we optimized the hyperparameters following a k-fold crossvalidation on the training dataset.

Method Cohen Kappa Score Accuracy Mean Precision Mean Recall Articulated Truck Bicycle Bus Car Motorcycle Non-motorized Vehicle Pedestrian Pickup Truck Single Unit Truck Work Van Background
bagging + CNN [107] 0.9666 0.9786 0.9355 0.9041 0.9412 0.8739 0.9593 0.9866 0.9131 0.7078 0.9610 0.9510 0.8273 0.8258 0.9980
Joint fine-tuning with DropCNN [124] 0.9681 0.9795 0.9530 0.8970 0.9324 0.8949 0.9779 0.9853 0.9111 0.5228 0.9406 0.9539 0.8336 0.9166 0.9984
Ensemble of Local Expert and Global Networks [125] 0.9675 0.9792 0.9298 0.9024 0.9358 0.8774 0.9620 0.9889 0.9212 0.6872 0.9425 0.9507 0.8289 0.8353 0.9966
Ensemble of Deep Networks: Network A,B and C [150] 0.9658 0.9780 0.9439 0.9190 0.9451 0.8984 0.9794 0.9790 0.9374 0.7237 0.9348 0.9624 0.8445 0.9059 0.9980
AlexNet [159] 0.8325 0.8917 0.6916 0.5603 0.6220 0.3047 0.8596 0.9058 0.2182 0.0000 0.7604 0.8430 0.2445 0.4203 0.9847
DenseNet [160] 0.9585 0.9734 0.9163 0.8705 0.9142 0.8354 0.9523 0.9857 0.8525 0.5571 0.9585 0.9323 0.7984 0.7919 0.9969
Inception V3 [161] 0.9408 0.9618 0.8478 0.8278 0.9231 0.8266 0.9066 0.9723 0.8121 0.3037 0.8665 0.9355 0.7781 0.7911 0.9903
Xception [162] 0.9627 0.9761 0.9085 0.9064 0.8891 0.8564 0.9752 0.9872 0.9131 0.7763 0.9367 0.9202 0.8125 0.9067 0.9972
ResNet-50 [163] 0.9484 0.9668 0.8582 0.8706 0.9374 0.8827 0.9407 0.9788 0.8808 0.6370 0.8812 0.9272 0.7492 0.7696 0.9917
VGG-19 (with Batch Norm) [164] 0.9323 0.9564 0.8530 0.8245 0.8856 0.7531 0.9426 0.9640 0.8626 0.3858 0.9214 0.9292 0.7047 0.7283 0.9917
DECM-BS [184] 0.9651 0.9776 0.9201 0.8844 0.9312 0.9037 0.9663 0.9889 0.9010 0.5594 0.9022 0.9402 0.7898 0.8468 0.9984
RXD-CV-CW_then_NCW [231] 0.9676 0.9793 0.9343 0.8940 0.9324 0.8897 0.9678 0.9905 0.9293 0.5479 0.9387 0.9329 0.8148 0.8914 0.9984
RXD-CV-CW [232] 0.9638 0.9767 0.8950 0.9127 0.8891 0.9159 0.9698 0.9823 0.9232 0.7192 0.9342 0.9491 0.8438 0.9174 0.9963
RXD Weighted-Average Ensemble [239] 0.9660 0.9783 0.9422 0.8827 0.9451 0.8792 0.9698 0.9912 0.9253 0.5114 0.9489 0.9283 0.7578 0.8547 0.9986
Augmented RXD Super Learner [243] 0.9678 0.9794 0.9215 0.9027 0.9219 0.8722 0.9779 0.9894 0.9172 0.6621 0.9374 0.9370 0.8219 0.8947 0.9979

References :

  • [107] PyongKun Kim, et al., " Vehicle Type Classification Using Bagging and Convolutional Neural Network on Multi View Surveillance Image", Traffic Surveillance Workshop and Challenge, CVPR 2017.
  • [124] H.Jung, MK Choi, J.Jung, JH Lee, S.Kwon, WY Jung "ResNet-based Vehicle Classification and Localization in Traffic Surveillance Systems", Traffic Surveillance Workshop and Challenge, CVPR 2017.
  • [125] Jong Taek Lee, et al., "Deep Learning-based Vehicle Classification using an Ensemble of Local Expert and Global Networks", Traffic Surveillance Workshop and Challenge, CVPR 2017.
  • [150] Rajkumar Theagarajan, et al., "EDeN: Ensemble of Deep Networks for Vehicle Classification", Traffic Surveillance Workshop and Challenge, CVPR 2017.
  • [159] A Krizhevsky, I. Sutskever, G. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of NIPS 2012, p.1097-1105.
  • [160] Gao Huang, Zhuang Liu, Kilian Q Weinberger, Laurens van der Maaten. Densely connected convolutional networks. arXiv preprint arXiv:1608.06993.
  • [161] C.Szegedy, V.Vanhoucke, S.Ioffe, J.Shlens. Rethinking the inception architecture for computer vision. roceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2818-2826).
  • [162] Chollet, François. Xception: Deep Learning with Depthwise Separable Convolutions. arXiv preprint arXiv:1610.02357, 2016.
  • [163] K He, X Zhang, S. Ren, J. Sun. Deep Residual Learning for Image Recognition. Proceedings of CVPR 2016, p.770-778.
  • [164] K. Simonyan, A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of ICLR 2015, p.1-14.
  • [184] Anonymous
  • [231] Anonymous
  • [232] Anonymous
  • [239] Anonymous
  • [243] Anonymous