Advancing state-of-the-art image recognition with deep learning on hashtags

This post details a novel approach to training image recognition models by leveraging large-scale public image datasets with hashtags as weakly supervised labels, rather than relying solely on labor-intensive manual annotation. It describes the challenges of noisy and vague hashtags and the development of new methods to handle multiple labels per image, sort synonyms, and balance frequent/rare hashtags. The work achieved record-high accuracy on ImageNet (85.4%) by training on 1 billion images with 1,500 matched hashtags, and demonstrated significant performance boosts (up to 22.5% relative improvement) on COCO object detection. The post also highlights the use of distributed training across 336 GPUs to shorten training time from over a year to a few weeks and the importance of selecting relevant hashtags for specific tasks. Future directions include using hashtags for video understanding, image ranking, and recognizing specific subcategories within images.