An article discussing the concept of monosemanticity in LLMs (Language Learning Models) and how Anthropic is working on making them more controllable and safer through prompt and activation engineering.
Anthropic has introduced a new feature in their Console that allows users to generate production-ready prompt templates using AI. This feature employs prompt engineering techniques such as chain-of-thought reasoning, role setting, and clear variable delineation to create effective and precise prompts. It helps both new and experienced prompt engineers save time and often produces better results than hand-written prompts. The generated prompts are also editable for optimal performance.
"scaling sparse autoencoders has been a major priority of the Anthropic interpretability team, and we're pleased to report extracting high-quality features from Claude 3 Sonnet, 1 Anthropic's medium-sized production model.
We find a diversity of highly abstract features. They both respond to and behaviorally cause abstract behaviors. Examples of features we find include features for famous people, features for countries and cities, and features tracking type signatures in code. Many features are multilingual (responding to the same concept across languages) and multimodal (responding to the same concept in both text and images), as well as encompassing both abstract and concrete instantiations of the same idea (such as code with security vulnerabilities, and abstract discussion of security vulnerabilities)."
"...a feature that activates when Claude reads a scam email (this presumably supports the model’s ability to recognize such emails and warn you not to respond to them). Normally, if one asks Claude to generate a scam email, it will refuse to do so. But when we ask the same question with the feature artificially activated sufficiently strongly, this overcomes Claude's harmlessness training and it responds by drafting a scam email."
Stay informed about the latest artificial intelligence (AI) terminology with this comprehensive glossary. From algorithm and AI ethics to generative AI and overfitting, learn the essential AI terms that will help you sound smart over drinks or impress in a job interview.
Bookmarks on this page are managed by an admin user. 