How To Train Your Multi-View Cloth Classifier

Overview

Garment classification is a typical first step for carrying out cloth manipulation tasks. This is a challenging feat for neural networks due to clothing’s easily deformable state, which causes high rates of error during training. To automate cloth-related tasks at a large scale, robots must identify articles of clothing with a comparable efficiency to humans. This project develops a deep learning model that classifies garment type using a dataset of 48,000 simulated cloth images. Multi-view classification was implemented to address the many deformable states of fabrics. The scores of four identical residual networks trained on different cloth angles were summed. To address error, majority voting was implemented. The scores of three networks were stacked, averaged, then argmaxed to obtain the class that the majority of networks were confident in. Domain randomization was implemented to close the Sim2Real gap that occurs when using simulated images. Mask indexing was used to replace the image backgrounds with random images from the Places365 dataset. Domain randomized images were trained on the highest-performing multi-view model to achieve an accuracy of 85.6%. The accuracy of the final model demonstrates its potential to accurately classify articles of clothing in the real world, enabling systems to complete complex tasks that require classification ability. Such technology can contribute to the automation of textile production, which is currently one of the lowest-paying forms of labor in the world.

Competition history

• ISEF 2022 Robotics and Intelligent Machines · Entry ROBO062 · Atlanta, Georgia, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.