Addressing the Challenge of Ambiguous Data in Deep Learning : A Strategy for Creating High-quality Image Annotations with Human Reliability and Judgement Enhancement

In machine learning, the availability of high-quality labeled data is essential for training accurate models. However, humans often disagree among themselves or over time when labeling or annotating image classification data. As a result, they create ambiguous data that poses a significant challenge to deep learning.
This research focuses on addressing this issue by proposing an overview that defines the problem, provides the necessary data for research, establishes evaluation metrics and presents methods for solving the problem. The ultimate goal is to develop an effective annotation strategy for image classification based on several thousand different experiments to improve data quality while reducing cost.

The main idea of this dissertation is not to rely on one annotation per image, but to consider the average of multiple annotations. However, this would not be feasible in reality without reducing the cost of acquiring such annotations. This dissertation uses proposals as a guide during the annotation process to improve the consistency of the annotations and to reduce the annotation time and thus the associated costs. The use of proposals could introduce a bias into the data.
Analysis of this bias and its introduction led to possible methods to minimize or reverse it. All this research is finally unified in an annotation strategy in the form of a flowchart that allows future research to easily understand the necessary steps needed to produce high quality data for their specific use case with as little effort as possible. This strategy is successfully verified on a real biomedical task.

In conclusion, this thesis presents a comprehensive investigation of the challenges posed by annotating ambiguous data in deep learning. This research provides an effective strategy to address the fundamental issue of data quality, thereby improving the quality and reliability of models and advancing the field of deep learning as a whole.

Logo KCSS

Rights

Use and reproduction:


CC BY 4.0

Please note that individual components of the publication may be subject to other licensing or copyright conditions.

Cite

Citation style:
Could not load citation form.