2 supervised tasks: coronary artery segmentation following Syntax score methodology and stenosis detection (segmentation) (both tasks are instance segmentation tasks).
1500 images with labeled coronary arteries and 1500 images with labeled stenosis for training and testing.
1. For training 1000 images will be provided
2. For evaluation 200 images will be provided
3. For testing 300 images will be used, but not provided to participants
Labels for "Coronary artery segmentation" task are assigned using the Syntax Score methodology. In short, each region of the vessel tree is assigned a name and its ordinal number, according to its location. More detailed explanation of each class with its landmarks can be found here https://syntaxscore.org/index.php/tutorial/definitions/14-appendix-i-segment-definitions
Evaluation of tasks
F1 score will be calculated for each image, and final F1 score of a participant is the average of F1 Scores of all images, which makes having same results of two participants to be near to impossible. However, in case they are same and are within 0.1% difference, the total inference time will be taken into account (sum of all inference time for all images), and the one with lowest inference time will be considered as winner.There are no specific weight given to any of ranking metrics. However, the main ranking method is F1 score, whereas inference time calculation is taken into account only in cases where F1 score is same.
Not that F1 score is calculated in cases where inference time for image is within the limits, and if not -> 0 F1 score will be given for that image
Evaluation hardware:
We use grand challenge provided hardware where all containers will be evaluated, which is:
Nvidia T4, 16GB GPU memory, 8 CPU, 30GB CPU memory