Last week, Anthropic announced a significant breakthrough in our understanding of how large language models work. The research focused on Claude 3 Sonnet, the mid-sized version of Anthropic’s latest frontier model. Anthropic showed that it could transform Claude's otherwise inscrutable numeric representation of words into a combination of ‘features’, many of which can be understood by human beings. The vectors Claude uses to represent words can be understood as the sum of ‘features’—vectors that represent a variety of abstract concepts from immunology to coding errors to the Golden Gate Bridge. This research could prove useful for Anthropic and the broader industry, potentially leading to new tools to detect model misbehavior or prevent it altogether.
The paper proposes the "law of increasing functional information," a new law of nature that could help explain the evolution of complex systems across multiple scales in the universe, from atoms and molecules to stars and brains.
These systems are characterized by three attributes: they form from numerous components, processes generate numerous configurations, and configurations are preferentially selected based on function.
The law suggests that functional information of a system will increase over time when subjected to selection for function(s). The authors argue this law could help predict the behavior of these systems and provide a unified framework for understanding their evolution.
They suggest it could be a missing piece in our understanding of the universe.
These systems are characterized by three attributes: they form from numerous components, processes generate numerous configurations, and configurations are preferentially selected based on function.
The law suggests that functional information of a system will increase over time when subjected to selection for function(s). The authors argue this law could help predict the behavior of these systems and provide a unified framework for understanding their evolution.
They suggest it could be a missing piece in our understanding of the universe.
This article discusses the implementation of a procedure for defining time using entangled systems according to the Page and Wootters approach. The authors study how quantum dynamics transform into classical-like behavior and analyze the relations that must hold between quantities of the system and the clock for the overall picture to represent physical dynamics.
The article discusses how machine learning is being used to calculate the macroscopic world that would emerge from string theory, a theory that posits the existence of tiny, invisible extra dimensions. These calculations have been difficult due to the enormous number of possibilities, but recent advances in artificial intelligence have made it possible to approximate the shapes of the Calabi-Yau manifolds, the objects that resemble loofahs and host quantum fields in string theory. The calculations have been able to reproduce the number of particles in the standard model, but not their specific masses or interactions. The long-term goal is to use these calculations to predict new physical phenomena beyond the standard model. The article also mentions that some physicists are skeptical of the usefulness of string theory and the role that machine learning will play in it.
Analysis of dozens of ancient genomes reveals that close encounters between Neanderthals and humans took place in a narrow time window. The high-resolution analysis also allowed the authors to track when certain Neanderthal DNA sequences appeared in the H. sapiens genome and determine whether they were retained.
"scaling sparse autoencoders has been a major priority of the Anthropic interpretability team, and we're pleased to report extracting high-quality features from Claude 3 Sonnet, 1 Anthropic's medium-sized production model.
We find a diversity of highly abstract features. They both respond to and behaviorally cause abstract behaviors. Examples of features we find include features for famous people, features for countries and cities, and features tracking type signatures in code. Many features are multilingual (responding to the same concept across languages) and multimodal (responding to the same concept in both text and images), as well as encompassing both abstract and concrete instantiations of the same idea (such as code with security vulnerabilities, and abstract discussion of security vulnerabilities)."
We find a diversity of highly abstract features. They both respond to and behaviorally cause abstract behaviors. Examples of features we find include features for famous people, features for countries and cities, and features tracking type signatures in code. Many features are multilingual (responding to the same concept across languages) and multimodal (responding to the same concept in both text and images), as well as encompassing both abstract and concrete instantiations of the same idea (such as code with security vulnerabilities, and abstract discussion of security vulnerabilities)."
"...a feature that activates when Claude reads a scam email (this presumably supports the model’s ability to recognize such emails and warn you not to respond to them). Normally, if one asks Claude to generate a scam email, it will refuse to do so. But when we ask the same question with the feature artificially activated sufficiently strongly, this overcomes Claude's harmlessness training and it responds by drafting a scam email."
This review article discusses the concept of entropy in statistical physics and its role as both a tool for inference and a measure of time irreversibility. It highlights the developments in stochastic thermodynamics and the principle of maximum caliber, emphasizing the importance of cross-talk among researchers in disparate fields.
- Life's evolution on Earth began with single-celled organisms and evolved into complex life forms through environmental factors and extinction events, culminating in the emergence of mammals and ultimately our own species.
- The article highlights the significance of our species, Homo sapiens, within the grand scheme of things, occurring within the last 300,000 years.
- Despite the limited time our species exists compared to the vastness of cosmic time, it encourages us to consider the potential for future life forms
- The article highlights the significance of our species, Homo sapiens, within the grand scheme of things, occurring within the last 300,000 years.
- Despite the limited time our species exists compared to the vastness of cosmic time, it encourages us to consider the potential for future life forms
