The experts explained that when using processed data, images produced by the algorithm were clearer and sharper and, in some cases, up to 48% better than images constructed from raw data. This can create biased results when algorithms are unknowingly trained on processed data.
“Our main observation is that bias stems from the unintentional coupling of hidden data-processing pipelines with later retrospective subsampling experiments,” the authors wrote. “The data processing implicitly improves the inverse problem conditioning, and the retrospective subsampling enables the algorithms to benefit from that.”
The authors suggested guidelines to help avoid such overinflated AI study results, including the important recommendation that data curators provide detailed descriptions of all their processing steps, among others.
“We call for attention of researchers and reviewers: Data usage and pipeline adequacy should be considered carefully, reproducible research should be encouraged, and research transparency should be required,” the experts said.
More on artificial intelligence in imaging:
Legal ramifications to consider when integrating AI into daily radiology practice
Why radiologist virtue is so important in the AI era: 6 pieces of advice
Radiogenomics could personalize cancer care, but experts are still hesitant to embrace the method
Transparent AI platform shows radiologists its decision-making blueprint for diagnosing breast cancer
