ML linters and other mechanisms enhance labeler productivity when labeling complex images and scenes, resulting in higher quality data for customers.
- Quality is important in ML and affects precision, recall, and IOU.
- Scale AI published four papers this year, including a dataset on Fitzpatrick skin type and a Reddit comment and reply dataset.
- Scale AI's 3D annotation platform and ML-powered linters catch incorrect annotations.
- ML linters and other mechanisms improve labeler productivity and result in higher quality data for customers.
One example of the qualitative study involved setting thresholds for the lender's sensitivity to avoid false positives. The team ran both quantitative and qualitative experiments to set two thresholds: the model prediction score and the jitter threshold in the XY plane. They plotted multiple curves with different thresholds and observed that the curves were fairly stable to jitter within one meter range. They set the jitter threshold as one meter. Another example involved using lighter data to catch missing poles, which were initially flagged as false negatives. The linter was able to detect the missing poles and improve the annotation quality.