The relationship between predictability and reconstructability, and how it can vary in opposite directions in complex systems. The work is based on information theory and was performed on various dynamics on random graphs, including continuous deterministic systems, and provides analytical calculations of the uncertainty coefficients for many different systems.
This article explains the concept of abstraction in neural networks and its connection to generalization. It also discusses how different components in neural networks contribute to abstraction and reveals an interesting duality between abstraction and generalization.
Minimum Description Length
