Is it possible to register as a team? How should I do that?
It is sufficient for a single participant per group to register. All the other participants must
be
listed in the submission form. Each participant can be listed only as a member of one group.
Is it possible to join an already existing team?
Yes, it is. As mentioned above, group participation is not formal until submission.
How do I submit a solution?
In order to be considered valid, a submission needs to include two things: the “Prognosis” for each
image of the test set; a two-page (max) document, including a description of the technique adopted to
obtain said solution and a brief discussion on the explainability of the proposed model, if any.
Each group is allowed to submit either one or two models. If two models are submitted, one should be
tagged as optimized for "performance", the other for "explainability".
The document describing the explainable model should also provide a brief highlight of the model
features.
Which information will be provided in the test set?
The test set contains the latest batch of curated data from one of the centers involved in the previous
phase of the study. As for the train set, exactly one chest x-ray will be provided for each subject,
along with the available clinical information. The content of some fields is empty, as is the case for
the training set. The only two fields that have been redacted in the test set for this challenge are
those relative to “Prognosis” and “Death”.
Why are images so different from one another? Why are some black? Why is the grayscale inverted in
some images with respect to others?
Remember that this database was collected in near-emergency conditions. X-rays have been acquired
with a vast number of different devices on subjects with conditions ranging from asymptomatic to
unconscious. The only pre-processing performed on images in both training and test set has been that of
converting them to 14-bit precision. As a consequence, images whose original precision was lower will
appear as black or nearly black until normalized.
The inverted grayscale is a feature occurring both in the training and test sets.
Can you provide us with any advice?
A good starting point to understand what has already been done and what a baseline performance
could be "Soda, Paolo, et al. "AIforCOVID: predicting the clinical outcomes in patients with
COVID-19 applying AI to chest-X-rays. An Italian multicentre study." Medical image analysis 74 (2021):
102216".
Can we take advantage of other CXR databases?
Yes, you can, with the only limitation that you should explicitly mention the resources used in the
document describing your algorithm, in the submission.
Which metric will be used for algorithm evaluation?
Algorithms will be ranked according to the resulting balanced accuracy on test set:
$BA=\frac{TPR+TNR}{2}$
A second ranking on explainability will be based entirely on the judgment of a panel of experts.