SynthRAD2023 Banner
Join

SynthRAD2023

The Challenge has taken place; if you wish to assess you sCT solutions feel free to use the post-challenge phase. Please refer to:

  • Challenge report Huijben EM, Terpstra ML, Pai S, Thummerer A, Koopmans P, Afonso, M, ... & Maspero M. 2024. Generating synthetic computed tomography for radiotherapy: SynthRAD2023 challenge report. Medical Image Analysis, 103276. https://doi.org/10.1016/j.media.2024.103276
  • Challenge dataset Thummerer A, van der Bijl E, Galapon Jr A, Verhoeff JJ, Langendijk JA, Both S, van den Berg CAT, Maspero M. 2023. SynthRAD2023 Grand Challenge dataset: Generating synthetic CT for radiotherapy. Medical Physics, 50(7), 4664-4674. https://doi.org/10.1002/mp.16529

Background

Medical imaging has become increasingly important in the diagnosis and treatment of oncological patients, particularly in radiotherapy. 

Traditionally, X-ray-based imaging is widely adopted in RT for patient positioning and monitoring before, during, or after the dose delivery.

Computed tomography (CT) is considered the primary imaging modality in RT, providing accurate and high-resolution patient geometry and enabling direct electron density conversion needed for dose calculations [Chernak et al., 1975]. Also, cone-beam computed tomography (CBCT) plays a vital role in image-guided adaptive radiation therapy (IGART) for photon and proton therapy. 

However, due to the severe scatter noise and truncated projections, CBCT is affected by artifacts, e.g. as shading, streaking, and cupping that makes it unsuitable for accurate dose calculations [Ramella et al., 2017]. 

Image synthesis has been proposed to improve the quality of CBCT to the CT level, producing the so-called “synthetic CT” (sCT) [Kida et al., 2018]. The conversion of CBCT-to-CT would allow accurate dose computation, enabling adaptive CBCT-based RT and improving the quality of IGART provided to the patients.

In the last decades, magnetic resonance imaging (MRI) has also proved its added value for tumors and organs-at-risk delineation thanks to its superb soft-tissue contrast [Schmidt et al., 2015]. MRI can be acquired to simulate the treatment planning or to match patient positioning to the planned one and monitor changes before, during, or after the dose delivery [Lagendijk et al., 2004].

To benefit from the complementary advantages offered by different imaging modalities, MRI is generally registered to CT. Such a workflow requires obtaining a CT, increasing workload, and introducing additional radiation to the patient. Recently, MRI-only based RT has been proposed to simplify and speed up the workflow, decreasing patients' exposure to ionizing radiation, which is particularly relevant for repeated simulations or fragile populations like children. MRI-only RT may reduce overall treatment costs and workload, and eliminate residual registration errors when using both imaging modalities. Additionally, the development of MRI-only techniques can be beneficial for MRI-guided RT [Edmund and Nyholm, 2017].

The main obstacle in introducing MRI-only RT is the lack of tissue attenuation information required for accurate dose calculations. Many methods have been proposed to convert MR to CT-equivalent images, obtaining synthetic CT (sCT) for treatment planning and dose calculation. 

In recent years, the derivation of sCT from MRI or CBCT has increased interest based on artificial intelligence algorithms such as machine learning or deep learning. However, no public data or challenges have been designed to provide ground truth for this task.

A recent review of deep learning-based sCT generation advocated for public challenges to provide data and evaluation metrics to compare different approaches openly [Spadea & Maspero et al. 2021].

Objective & Tasks

This challenge aims to provide the first platform offering public data evaluation metrics to compare the latest developments in sCT generation methods. The accepted challenge design approved by MICCAI can be found at https://doi.org/10.5281/zenodo.7746019. A type 2 challenge will be run, where the participant needs to submit their algorithm packaged in a docker both for validation and test. Two tasks are defined:

  • Task 1 MRI-to-sCT generation to facilitate MR-only RT.
  • Task 2 CBCT-to-sCT generation to facilitate IGART.

Challenge phases definition

Training input and ground truth dataset are released to allow the teams to develop their algorithms. 

Preliminary test a phase to allow the teams to familiarize with the submission system. The algorithm will run a couple of cases providing feedbacks for the teams based on the image similarity metrics in the related leaderboard.

Validation the teams will submit their algorithms (max 2/week) and image similarity metrics will be calculated and openly available in a public leaderboard.

Test the teams will submit their final algorithms (max 2, only the last submission counts) and image similarity and dose metrics will be calculated and openly available in a public leaderboard. The final ranking will be made available after the test phase is concluded.

Dataset

A multi-center dataset is presented, balancing training/validation and test cases to evaluate the methods on different MRI sequences and acquisition settings. Currently, three centers (UMC Utrecht, UMC Groningen, and Radboud Nijmegen) share data providing 60/10/20 MRI/CBCT and CT in train/validation/test of patients undergoing radiotherapy for each of the two tasks.
A total of 540 paired MRI-CT and 540 CBCT-CT sets is provided, along with a mask that will be considered for evaluation and may be used for inference. The target of the validation and test set will not be shared with the participants to avoid optimistic biases. Input data of the validation and test set will be accessible by the algorithm submitted by the participants, but not directly available to the teams, until the end of the challenge.

A paper has been published describing the dataset. Please cite:
Thummerer A, van der Bijl E, Galapon Jr A, Verhoeff JJ, Langendijk JA, Both S, van den Berg CAT, Maspero M. 2023. SynthRAD2023 Grand Challenge dataset: Generating synthetic CT for radiotherapy. Medical Physics, 50(7), 4664-4674. https://doi.org/10.1002/mp.16529

Participation

Challenge participants may choose to participate either in task 1 or 2 or both. For each task, algorithms should be provided for both pelvic and neurological cases. The participant may decide whether providing one or more models per task.
The evaluation code used to rank the challenge is shared. After completion, the leaderboard will remain open for submission, ensuring that future methods may still be evaluated.

We envision that this challenge will enable a fair and open evaluation of different approaches.
We hope you may have fun taking part in this challenge!
The organizers