3D Shape Categorization Benchmark
The 3DSCB (3D Shape Categorization Benchmark) provides an elaborate experimental setup (with publicly available data and code) for the problem of 3D shape categorization. The objective of the methods using the 3DSCB is to predict the class for each of the test 3D shapes provided.
Our aim is to provide to the research community a clear benchmark so as to establish further comparisons among different methods for 3D shape categorization. We start describing the databases, and then how the evaluation of the results is going to be performed.
Data
The 3DSCB gathers more than 3400 3D shapes. It consists of 3 publicly available databases: SHREC'12 [2], Princeton [3], and TOSCA + Sumner [4,5]. All these datasets consists of clean and segmented 3D shapes.
Here we distribute the experimental setup described in [1], including: the annotations and the training and testing subsets for each dataset; and a set of tools for accessing and managing the database annotations.
Evaluation of Results
For each dataset, we have clearly defined two main subsets: training and testing. Although the ground truth is offered for both subsets, the testing data must be used strictly for reporting of results alone, i.e. it must not be used in any way to train or tune the proposed approaches. Only the training data can be used for parameter tuning or feature selection, e.g. using n-fold cross-validation.
The proposed experimental setup offers a multi-class problem, and we propose the following evaluation metric in order to compare different methods.
The performance p of the classifier defined as
where TP and FP are the number of true positives and false positives, respectively.
Each method reporting results in the 3DSCB has to provide the p for each dataset (i.e. SHREC2012, Princeton or TOSCA + Sumner ) independently.
Downloads - Software Development Kit
This SDK consists of: the training and testing data, all the codes for implementing the 3D shape classification tasks described in our paper.- Download SDK - Last update (17-January-2014).
Latest Results
These are the latest results on the 3DSCB GRAM UAH -- Last update (17-January-2014).
| Method (p/MCC) | Princeton | TOSCA+Sumner | SHREC'12 | Final |
|---|---|---|---|---|
| 3DSP | 66.26% | 95.8% | 63.83% | 75.29% |
Methods:
- 3DSP ->[1]
If you would like to include your algorithm's performance please email us at robertoj.lopez (AT) uah.es with a citation and your results.
Citing
If you make use of this data and software, please cite ALL the following references in any publications:
[1] Lopez-Sastre, R. J. and Garcia-Fuertes, A. and Redondo-Cabrera, C. and Acevedo-Rodriguez, F. J. and Maldonado-Bascon, S. Evaluating 3D Spatial Pyramids for Classifying 3D Shapes, Computers and Graphics, 2013, Vol. 37, Issue 5, pages 473-483. PDF.
[2] Li, B. and Godil, A. and Aono, M. and Bai, X. and Furuya, T. and Li, L. SHREC’12 Track: Generic 3D Shape Retrieval. 3DOR:Eurographics Workshop on 3D Object Retrieval, 2012.
[3] Shilane, P. and Min, P. and Kazhdan, M. and Funkhouser, T. The Princeton Shape Benchmark Shape Modeling International, Genova, Italy, June 2004.
[4] Bronstein, A. and Bronstein, M. and Kimmel, R. Numerical Geometry of Non-Rigid Shapes. Springer Publishing Company, Incorporated (2008)
[5] Sumner, R. W. and Popovic, J. Deformation Transfer for Triangle Meshes. ACM Transactions on Graphics. 23, 3. August 2004.
Acknowledgements
This work was partially supported by projects TIN2010-20845-C03-03, UAH2011/EXP-030 and IPT-2011-1366-390000.