What happened
OpenAI launched IndQA, a new benchmark designed to evaluate AI systems' understanding and reasoning about Indian culture and languages. IndQA covers 2,278 questions across 12 Indian languages and 10 cultural domains. The questions were created with input from 261 domain experts across India.
Why it matters
- IndQA aims to improve AI models' ability to understand culturally nuanced and reasoning-heavy tasks in Indian languages.
- It addresses limitations of existing benchmarks that focus mainly on translation or multiple-choice questions.
- The benchmark supports OpenAIâs goal to make AI technology more accessible and relevant for Indian users.
Key facts
- 1. IndQA includes 2,278 questions spanning 12 languages such as Hindi, Bengali, Tamil, and English.
- 2. The benchmark covers 10 cultural domains, including History, Law & Ethics, Food & Cuisine, and Sports & Recreation.
- 3. The questions were developed in partnership with 261 domain experts from across India.
