Crowdsourcing image segmentation for deep learning: integrated platform for citizen science, paid microtask, and gamification.
Biomed Tech (Berl)
; 69(3): 293-305, 2024 Jun 25.
Article
in En
| MEDLINE
| ID: mdl-38143326
ABSTRACT
OBJECTIVES:
Segmentation is crucial in medical imaging. Deep learning based on convolutional neural networks showed promising results. However, the absence of large-scale datasets and a high degree of inter- and intra-observer variations pose a bottleneck. Crowdsourcing might be an alternative, as many non-experts provide references. We aim to compare different types of crowdsourcing for medical image segmentation.METHODS:
We develop a crowdsourcing platform that integrates citizen science (incentive participating in the research), paid microtask (incentive financial reward), and gamification (incentive entertainment). For evaluation, we choose the use case of sclera segmentation in fundus images as a proof-of-concept and analyze the accuracy of crowdsourced masks and the generalization of learning models trained with crowdsourced masks.RESULTS:
The developed platform is suited for the different types of crowdsourcing and offers an easy and intuitive way to implement crowdsourcing studies. Regarding the proof-of-concept study, citizen science, paid microtask, and gamification yield a median F-score of 82.2, 69.4, and 69.3â¯% compared to expert-labeled ground truth, respectively. Generating consensus masks improves the gamification masks (78.3â¯%). Despite the small training data (50 images), deep learning reaches median F-scores of 80.0, 73.5, and 76.5â¯% for citizen science, paid microtask, and gamification, respectively, indicating sufficient generalizability.CONCLUSIONS:
As the platform has proven useful, we aim to make it available as open-source software for other researchers.Key words
Full text:
1
Collection:
01-internacional
Database:
MEDLINE
Main subject:
Image Processing, Computer-Assisted
/
Crowdsourcing
/
Deep Learning
/
Citizen Science
Limits:
Humans
Language:
En
Journal:
Biomed Tech (Berl)
/
Biomed. tech
/
Biomedizinische Technik
Year:
2024
Document type:
Article
Affiliation country:
Germany
Country of publication:
Germany