Quality-Aware and Explainable Synthetic Data Selection for Robust Computer Vision Under Domain Shift

Abstract

This study proposes a method for selecting synthetic training images based on both their effect on model performance and their interpretability. We evaluate the approach through a practical prototype on CIFAR-10, using synthetic candidate generation, quality-aware selection, a ResNet50 classifier, robustness testing under domain shift, and Grad-CAM explanations. This study provides a methodology for choosing which type of synthetic data—focusing on both how well the data performs when used to train models (performance quality) and how understandable the data is to humans (interpretability)—should be generated and evaluated to provide specialized training for models, and then used to evaluate the final model's performance and robustness. Although using AI to create training data for computer vision models has the potential to increase the amount of training data available, it should be noted that all synthetic images may not necessarily be equally useful. The prototype is built on the CIFAR-10 reference dataset, with a ResNet50 model as its classifier. Three thousand (3000) synthetic test examples were produced, and then one thousand (1000) were chosen based on the suggested selection criteria. Comparisons included three training setups: one that used only the actual data, another that included the actual data plus all produced synthetic candidate samples, and a final configuration that included the actual data plus only the selected portion of produced synthetic candidate samples. The training using actual data gave a classification rate of 90.97%; however, when the actual data was combined with all of the generated synthetic samples, the classification rate increased to 91.50%. When the actual data was combined with only those synthetic samples that were selected for inclusion, the classification rate increased to 91.14%. Therefore, while the prototype improved upon the baseline classification rate by a small amount, combining both types of data (actual and synthetic) and including all of the candidate synthetic samples resulted in the highest classification rate during testing. In tests of robustness for the model developed using the chosen synthetic sample, the model achieved 21.46% accuracy in the presence of noise, 38.60% accuracy in the presence of blur, 90.20% accuracy in the presence of occlusion, and 85.91% accuracy after applying contrast adjustments. Also, the Grad-CAM approach offered a visual explanation as to what parts of the image the model focused on when it was able to accurately classify a picture of a cat. These results suggest that there are some benefits associated with implementing the methodology used in this study; however, they also point toward an area for future research, which is that the selection strategy currently employed needs to be improved and calibrated so that the quality and variety of synthetic images generated can result in improvements that exceed those achieved through unselected synthetic data augmentation techniques.