An engineer from Yandex together with researchers from the Skoltech Artificial Intelligence Center, and St. Petersburg State University of Aerospace Instrumentation have introduced the world’s largest open dataset, PackEat, for computer vision systems in retail. This is a collection of photographs of fruits and vegetables that retailers can use to train algorithms for smart checkout and inventory systems. Using this dataset can significantly improve product recognition accuracy in real supermarkets — it accounts for images of objects in plastic bags, overlapping items, and “noisy” backgrounds on counters.
For the PackEat dataset, the team collected images of 34 types and 65 varieties of fruits and vegetables. These are familiar items from produce aisles, photographed from different angles in actual stores. Over 100,000 images were gathered, containing more than 370,000 individual objects from stores in various cities. Around 9,000 images include annotations for each individual object, with each photo noting the object count and total package weight. This dataset is the largest of its kind in the world and will help solve key computer vision tasks in retail: distinguishing between product types and varieties, isolating each object individually even when they overlap or are partially obscured, and automatically counting the number of items.
Retail chains continue to face the challenge of manually identifying types, varieties, and defects of sold-by-weight goods (fruits, vegetables), leading to losses. Research show that neural networks can achieve 92% accuracy, underscoring the importance of automation. The article describing the dataset has been published open access in the journal Scientific Data. The image collection is hosted on the Zenodo platform, and the code and model examples are on Kaggle allowing researchers and developers to immediately use them in their projects and compare their solutions with the authors’ results.
About the Research Team
A key role in the research was played by Sergey Nesteruk from Yandex Cloud and Svetlana Illarionova from the Skoltech AI Center.
The scientific work of Sergey Nesteruk, a graduate of the Skoltech PhD program “Computational and Data Science in Science and Engineering,” is focused on computer vision. Among his publications are papers in international journals and IEEE proceedings. He regularly speaks at professional conferences. Currently, Sergey leads the Artificial Intelligence Security team at Yandex Cloud. He is a co-author of materials on the secure development of AI agents and multi-agent systems. His team ensures security in the use of artificial intelligence and also develops AI tools for data protection and Yandex Cloud security.
Svetlana Illarionova leads the “Computer Vision for Data Processing” group at the Skoltech AI Center. In 2023, she earned her PhD from Skoltech on the topic of computer vision for Earth remote sensing. At Skoltech, Svetlana manages research projects applying computer vision algorithms to process multimodal data, particularly for satellite environmental monitoring and medical analysis tasks. She has also been the principal investigator of a Russian Science Foundation grant since 2023. She is the author of more than 25 scientific articles in leading peer-reviewed journals, conferences, and books, and holds several invention patents. Svetlana’s primary area of interest is developing computer vision methods aimed at solving applied problems, taking into account their specific nature and analyzing the properties of the objects under study.
