OpenAI's chief scientist Jakub Pachocki has proposed using perplexity as a metric to measure and limit AI development. This comes amid growing concerns about the pace of AI advancements and their potential risks.
TL;DR
- OpenAI's chief scientist suggests perplexity as a metric to measure AI progress and set limits on development.
- Perplexity, which measures a model's predictive accuracy, could serve as a shared benchmark for general intelligence.
- This proposal follows calls from industry leaders to establish common safety standards and slow unchecked AI progress.
What happened
In a recent blog post, OpenAI's chief scientist Jakub Pachocki discussed the need to limit AI development by focusing on perplexity, a measurement of how well a model predicts the next token. Lower perplexity indicates better predictive performance.
Pachocki argues that current benchmarks focusing on AI's downstream capabilities quickly become outdated. However, perplexity has remained informative as AI models advance.
The post includes a chart showing the decline in perplexity of OpenAI's models since GPT 3.5. Pachocki notes that lower perplexity has led to the emergence of transformative capabilities in fields like cybersecurity and mathematics.
Why it matters
This proposal comes amid growing concerns about the rapid pace of AI development and its potential risks. Former researchers at OpenAI and Anthropic have warned about the dangers of unchecked AI progress.
Anthropic CEO Dario Amodei has called for common safety standards and limits on AI development, a plan seemingly endorsed by OpenAI CEO Sam Altman and SpaceX CEO Elon Musk.
Using perplexity as a standard measurement could help AI companies commit to not exceeding certain thresholds until they can better understand and monitor AI behavior.
Key facts
- Perplexity measures how well a model predicts the next token; lower perplexity means better predictions.
- OpenAI's GPT-4 has a perplexity of ~0.91, enabling advanced reasoning capabilities.
- Models with perplexity below 0.8 have shown transformative capabilities in fields like cybersecurity and mathematics.
- OpenAI CEO Sam Altman described Pachocki's post as 'the most important thing OpenAI has published in a while'.
- Anthropic's IPO filing mentioned potential 'catastrophic or existential risks' posed by AI technology.
- Dario Amodei's plan to 'pace the frontier' includes agreeing to common safety standards and limits on AI progress.
Context
The debate over limiting AI development has gained momentum as AI models become more powerful and their implications more profound. Industry leaders are increasingly calling for responsible AI development practices.
Establishing a common benchmark like perplexity could help standardize the measurement of AI progress and ensure that development is paced responsibly.
This discussion is part of a broader conversation about the ethical and safety considerations surrounding AI, as the technology continues to evolve rapidly.
