Measuring AI capabilities in intelligence targeting and conventional weapons

This post details the development of new capability evaluations by Anthropic's Frontier Red Team to measure AI performance in tactical intelligence targeting (identifying individuals/objects based on fragmentary information) and conventional weapons development (e.g., engineering drones). These evaluations demonstrate that AI models are making consistent progress in these specialized domains, with frontier models showing capabilities that historically required scarce, highly-trained human experts. The post introduces specific evaluations for account correlation and individual classification using simulated social media data, and for geolocating social media photographs using only world knowledge. It also describes an evaluation for text-to-geolocation using a corpus of geotagged tweets and a sandboxed search tool. The results show frontier models approaching or exceeding human baselines in geolocating outdoor photos and highlight the speed advantage of AI in analyzing large volumes of text data. The findings underscore the necessity of on-platform safety measures and classifiers to prevent misuse of AI for surveillance and weapons development.