Constructing Benchmark Databases and Protocols for Medical Image Analysis: Diabetic Retinopathy

Table 2

Summary of the DiaRetDB1 V2.1 database in terms of the key questions addressed in Section 2.1.

Key questions	DiaRetDB1 V2.1
C2: “Is there a data set for which the correct answers are known?”	Yes
C3: “Are there data sets in common use?”	Yes (publicly available at [13])
C4: “Are there experiments which show algorithms are stable and work as expected?”	Experimental results reported in Section 4.4
C5: “Are there any strawman algorithms?”	No, but the baseline algorithm sets the baseline results for the DiaRetDB1 database
C6.1: “What code is available?”	Functionality for reading/writing images and ground truth, strawman algorithm, and annotation software (publicly available at [13, 14])
C6.2: “What data is available?”	Images and ground truth (XML) (publicly available at [13])
C7: “Is there a quantitative methodology for the design of algorithms?”	No, but medical practice is used as a guideline at each development step
C8.1: “What should we be measuring to quantify performance?”	Image- and pixel-based ROC analysis (description in Section 4)
C8.2: “What metrics are used?”	Equal error rate (EER) defined in Section 4