An interdisciplinary research project exploring the history and ideas behind the influential ELIZA chatbot, created in the 1960s. The project aims to contextualize ELIZA, analyze its code, and examine its cultural impact on human-computer interaction.
This paper introduces Cross-Layer Attention (CLA), an extension of Multi-Query Attention (MQA) and Grouped-Query Attention (GQA) for reducing the size of the key-value cache in transformer-based autoregressive large language models (LLMs). The authors demonstrate that CLA can reduce the cache size by another 2x while maintaining nearly the same accuracy as unmodified MQA, enabling inference with longer sequence lengths and larger batch sizes.
