Product Launches

OpenAI's Astra for Law tops Harvey LAB-AA with 20.8% full-credit score

Share
OpenAI's Astra for Law tops Harvey LAB-AA with 20.8% full-credit score

OpenAI's Astra for Law has achieved the top score in the Harvey LAB-AA evaluation, earning full credit on 20.8% of assignments. The model is now available to all ChatGPT Enterprise customers through the Trusted Access Program.

TL;DR

  • OpenAI's Astra for Law ranks first in the Harvey LAB-AA evaluation with a 20.8% full-credit score.
  • The model is now available to all ChatGPT Enterprise customers through the Trusted Access Program.
  • Astra for Law demonstrates significant improvements in legal reasoning and reduced hallucinations compared to other models.

What happened

OpenAI has released an updated version of Astra for Law, a model configuration of GPT-6.2 Astra, optimized for legal professionals. The model has been evaluated using the Harvey LAB-AA, LAB: USCL, and Vals AI's Legal Research Bench, where it achieved top scores in various legal tasks.

Astra for Law earned full credit on 20.8% of assignments in the Harvey LAB-AA evaluation, outperforming GPT-6.2 Astra (15.0%) and Muse Wonder Full (13.1%). The model also demonstrated a lower rate of hallucinations, averaging 0.03 material hallucinations per assignment compared to 0.59 for Claude Opus 5.5.

The model is now available to all ChatGPT Enterprise customers through the Trusted Access Program, including API access for integration into custom tools.

Why it matters

For developers and startups, the availability of Astra for Law through the Trusted Access Program opens up new possibilities for integrating advanced legal intelligence into their applications. The model's superior performance in legal reasoning and reduced hallucinations can enhance the accuracy and reliability of legal tools.

Investors should take note of OpenAI's continued innovation in the legal AI space, as demonstrated by the rapid improvements in Astra for Law. The model's top performance in various benchmarks highlights its potential to become a leading solution in the legal tech market.

However, the competitive landscape remains fierce, with models like Muse Wonder Full and Claude Opus 5.5 also vying for dominance. The real-world impact of Astra for Law will depend on its adoption by law firms and legal professionals, as well as its ability to continuously improve and adapt to the evolving needs of the legal profession.

Key facts

  • Astra for Law earned full credit on 20.8% of assignments in the Harvey LAB-AA evaluation.
  • The model averaged 0.03 material hallucinations per assignment, compared to 0.59 for Claude Opus 5.5.
  • Astra for Law is available to all ChatGPT Enterprise customers through the Trusted Access Program.
  • The model's legal search index has expanded by over a third since its initial release in September.
  • Astra for Law scored 1,844 in the LAB: USCL brief writing evaluation, compared to 1,817 for Claude Opus 5.5.
  • The model earned full credit on 53.8% of questions in the Vals AI Legal Research Bench, up from 46% in September.
  • The legal search index now spans over 310 million URLs, covering more than 99.9% of published U.S. precedential case law.

Context

The legal profession is increasingly embracing AI to enhance efficiency and accuracy in legal research, brief writing, and other tasks. OpenAI's Astra for Law is part of a broader trend of AI models being tailored to specific industries, such as healthcare, finance, and now law.

The competition in the legal AI space is intense, with companies like Harvey, Grubel, and Vals AI also developing advanced models. The performance of these models is critical for law firms and legal professionals who rely on accurate and reliable legal intelligence to serve their clients.

As AI continues to evolve, the legal profession is likely to see further integration of these technologies, potentially transforming the way legal services are delivered. The success of models like Astra for Law will depend on their ability to meet the unique challenges and requirements of the legal industry.

Topics

Related coverage

Join the discussion

Have a take on this story? Weigh in with our community on Facebook.

💬 Discuss on Facebook →