Frequently Asked Questions (FAQ)

Is it possible to register as a team? How should I do that?
It is sufficient for a single participant per group to register. All the other participants must be listed in the submission form. Each participant can be listed only as a member of one group.

Is it possible to join an already existing team?
Yes, it is. As mentioned above, group participation is not formal until submission.

How do I submit a solution?
In order to be considered valid, a submission needs to include two things: the “Prognosis” for each image of the test set; a two-page (max) document, including a description of the technique adopted to obtain said solution and a brief discussion on the explainability of the proposed model, if any.
Each group is allowed to submit either one or two models. If two models are submitted, one should be tagged as optimized for "performance", the other for "explainability". The document describing the explainable model should also provide a brief highlight of the model features.

Which information will be provided in the test set?
The test set contains the latest batch of curated data from one of the centers involved in the previous phase of the study. As for the train set, exactly one chest x-ray will be provided for each subject, along with the available clinical information. The content of some fields is empty, as is the case for the training set. The only two fields that have been redacted in the test set for this challenge are those relative to “Prognosis” and “Death”.

Why are images so different from one another? Why are some black? Why is the grayscale inverted in some images with respect to others?
Remember that this database was collected in near-emergency conditions. X-rays have been acquired with a vast number of different devices on subjects with conditions ranging from asymptomatic to unconscious. The only pre-processing performed on images in both training and test set has been that of converting them to 14-bit precision. As a consequence, images whose original precision was lower will appear as black or nearly black until normalized.
The inverted grayscale is a feature occurring both in the training and test sets.

Can you provide us with any advice?
A good starting point to understand what has already been done and what a baseline performance could be "Soda, Paolo, et al. "AIforCOVID: predicting the clinical outcomes in patients with COVID-19 applying AI to chest-X-rays. An Italian multicentre study." Medical image analysis 74 (2021): 102216".

Can we take advantage of other CXR databases?
Yes, you can, with the only limitation that you should explicitly mention the resources used in the document describing your algorithm, in the submission.

Which metric will be used for algorithm evaluation?
Algorithms will be ranked according to the resulting balanced accuracy on test set: $BA=\frac{TPR+TNR}{2}$
A second ranking on explainability will be based entirely on the judgment of a panel of experts.