Chilli is one of the world's most widely grown crops. Among all of the chilli variants, C. annuum and C. frustescents are the most prevalent and consistently liked variants in Asia, where it is appreciated for its strong taste and pungency. Nevertheless, harvesting at the proper ripening stage according to their colour, size, and texture is essential to ensure the best quality, marketability, and shelf life. Currently, visual inspection is the primary method used by farmers, which is time-consuming and complicated. Even though automated chilli classification using computer vision and intelligent methods has received scholars' attention, the classification of C. annuum and C. frustescents ripening stages using deep learning models has not been extensively studied. Hence, this study aims to investigate the effectiveness of three deep learning models, namely EfficientNetB0, VGG16 and ResNet50, in classifying chilli ripening stages into unripe, ripe, and overripe classes. We also introduce a huge dataset comprising 9,022 images of C. annuum and C. frustescents chilli under various growth stages and imaging conditions which provides sufficient samples for the deep learning modelling. The experimental results show that the ResNet50 model outperforms other models with more than 95% accuracy for all classes.

This study helps improve chilli harvesting by creating a large dataset of 9,022 images of C. annuum and C. frutescens at different ripening stages and conditions. It tests three deep learning models—EfficientNetB0, VGG16, and ResNet50—to classify chillies as unripe, ripe, or overripe, solving the problem of slow and inconsistent manual inspection. Results show that ResNet50 achieves over 95% accuracy, offering a reliable and efficient way to determine the right harvest time. This can save labour, reduce post-harvest losses, improve chilli quality, and boost market value. The dataset and approach can also support future research and be applied to other crops.