The Bitter Lesson (2919) by Rich Sutton explores a recurring pattern in the history of AI research, arguing that general methods leveraging massive computation are ultimately more effective than those relying on human-encoded domain knowledge. While incorporating human intuition can provide short-term gains, long-term breakthroughs are consistently driven by scaling computational power through search and learning as described by Moore's Law.
Key observations include:
- The historical shift in chess, Go, speech recognition, and computer vision from rule-based or human-centric models toward massive computation.
- The tendency for researchers to favor methods that reflect human understanding, which often plateaus compared to scalable learning processes.
- The necessity of developing meta-methods capable of discovering complex patterns rather than hardcoding existing human perceptions into agents.
Key observations include:
- The historical shift in chess, Go, speech recognition, and computer vision from rule-based or human-centric models toward massive computation.
- The tendency for researchers to favor methods that reflect human understanding, which often plateaus compared to scalable learning processes.
- The necessity of developing meta-methods capable of discovering complex patterns rather than hardcoding existing human perceptions into agents.
