Anthropic and AE Studio have developed GRAM, a method to selectively remove dual-use knowledge from AI models. The technique allows for the removal of specific knowledge categories without degrading general performance.
TL;DR
- Anthropic and AE Studio introduce GRAM, a method to control access to dual-use knowledge in AI models.
- GRAM allows for the removal of specific knowledge categories without affecting the model's performance on other tasks.
- The technique could enable the benefits of training many separately-filtered models at the cost of training only one.
What happened
Anthropic and AE Studio have collaborated on a new research project aimed at controlling access to dual-use knowledge in AI models. The method, called GRAM (Gradient-Routed Auxiliary Modules), adds extra neurons to every layer of a standard Transformer, dividing them into groups or 'modules' for each dual-use category.
During training, when the model encounters general-purpose text, it learns in the usual way. However, when it encounters text from a dual-use category, only the corresponding module is allowed to learn from that text. This ensures that the knowledge accumulates in the specific module rather than diffusing across the whole network.
The researchers tested GRAM in three settings of increasing realism. They found that the method performed almost identically to separate models trained from scratch with specific topics filtered out. GRAM also resisted attempts to recover removed knowledge about as well as data filtering did.
Why it matters
GRAM offers a potential path toward more robust access control for dual-use capabilities in AI models. This is crucial as AI companies train more capable models, and the need to limit access to such capabilities increases.
The method could enable the benefits of training many separately-filtered models at the cost of training only one. This could significantly reduce the computational and financial costs associated with training multiple models.
However, the research is still in its early stages, and there are clear limitations. GRAM hasn't been tested at frontier scale or in a production training pipeline. Additionally, some dual-use capabilities might be so entangled with general knowledge that no method can separate them cleanly.
Key facts
- GRAM adds extra neurons to every layer of a standard Transformer, dividing them into groups for each dual-use category.
- The method was tested on a synthetic dataset, a realistic mix of web text, code, and scientific papers, and at seven model sizes from 50 million to 5 billion parameters.
- GRAM matched the performance of data filtering at every size, and the gap between 'module on' and 'module off' grew wider as models got larger.
- The technique resisted attempts to recover removed knowledge about as well as data filtering did.
- GRAM hasn't been applied to any of Anthropic's Claude models, and it's not certain that it ever will be.
Context
As AI models become more capable, the need to limit access to dual-use capabilities becomes increasingly important. Current safeguards, such as training models to refuse harmful requests and using classifiers to screen inputs and outputs, are imperfect.
Methods like GRAM offer a potential solution by controlling what the model knows. This could enable the benefits of training many separately-filtered models at the cost of training only one, significantly reducing the computational and financial costs.
However, the research is still in its early stages, and there are clear limitations. It's not yet clear whether GRAM will be applied to production models, and some dual-use capabilities might be so entangled with general knowledge that no method can separate them cleanly.
