Brad Myers writes a proposal arguing that the fundamental building blocks of user interfaces—interaction techniques such as menus, scroll bars, and text fields—should themselves become more intelligent, enabling users to freely mix modalities like voice, text, and gesture even within a single interaction. He observes that these techniques were largely settled in the 1980s for graphical interfaces and saw only minor additions for touchscreens, while today's chat-based interfaces remain a separate paradigm from the traditional toolkit. The proposal calls for research into both new intelligent interaction techniques and the infrastructure needed to construct them, and flags significant security, privacy, and economic implications.
- A short 14 KB proposal rather than a full-length paper
- Cross-listed under both cs.HC and cs.AI
- Submitted 14 September 2026
- A short 14 KB proposal rather than a full-length paper
- Cross-listed under both cs.HC and cs.AI
- Submitted 14 September 2026
Microsoft has released the OmniParser model on HuggingFace, a vision-based tool designed to parse UI screenshots into structured elements, enhancing intelligent GUI automation across platforms without relying on additional contextual data.
Adios, static UIs. Hello, dynamic design systems that adapt to every user, on every screen.
