ROK-FORTRESS: What Language and Context Reveal About AI Safety

This post introduces ROK-FORTRESS, a novel benchmark for multilingual AI safety evaluation. It details the benchmark's methodology, which uses a controlled transcreation matrix to independently vary language (English vs. Korean) and geopolitical grounding (U.S. vs. Korean entities/institutions). The post presents findings that language drives more suppression than context, and that translation-only evaluations can be misleading. It also highlights the cost of apparent safety gains in terms of over-refusal of harmless requests. The contribution is a new evaluation framework and empirical findings on the interaction of language, context, and AI safety.