New benchmark reveals major hurdles in real-time AI interaction capabilities

NVIDIA and the University of Texas at Austin (UT-Austin) have released Realtime-Bench, a new evaluation standard designed to measure how large AI models perform in dynamic, fast-changing environments, moving beyond static tasks like text or image generation. The benchmark assesses 50 tasks across eight categories, focusing on three core capabilities: real-time perception, time-adaptive reasoning, and action timing. Testing 12 leading global models revealed significant limitations; the top performer, GPT-6-Astra, achieved only a 44% success rate when the environment advanced according to actual reasoning time. Further analysis indicated that simply boosting reasoning effort does not guarantee better outcomes, nor can reducing inference latency alone solve all real-world interaction problems. The findings suggest that balancing deep thought with precise timing remains a critical challenge for achieving Artificial General Intelligence (AGI).