klotz: arxiv*

Bookmarks on this page are managed by an admin user.

0 bookmark(s) - Sort by: Date ↓ / Title / - Bookmarks from other users for this tag

  1. AutoCrawler is a two-stage framework that leverages the hierarchical structure of HTML for progressive understanding and aims to assist crawlers in handling diverse and changing web environments more efficiently. This work introduces a crawler generation task for vertical information web pages and proposes the paradigm of combining LLMs with crawlers, which supports the adaptability of traditional methods and enhances the performance of generative agents in open-world scenarios. Generative agents, empowered by large language models, suffer from poor performance and reusability in open-world scenarios.
    2024-04-28 Tags: , , , , by klotz
  2. First, using the demonstrations significantly outperforms the no demonstrations method
    even with small k (k = 4), and performance drop
    from using gold labels to using random labels is
    consistently small across varying k, in the range of
    Interestingly, model performance does
    not increase much as k increases when k ≥ 8, both
    with gold labels and with random labels.
    2024-02-14 Tags: , , , , by klotz
  3. Large language models exhibit surprising emergent generalization properties, yet also struggle on many simple reasoning tasks such as arithmetic and parity. This raises the question of if and when Transformer models can learn the true algorithm for solving a task. We study the scope of Transformers' abilities in the specific setting of length generalization on algorithmic tasks. Here, we propose a unifying framework to understand when and how Transformers can exhibit strong length generalization on a given task. Specifically, we leverage RASP (Weiss et al., 2021) -- a programming language designed for the computational model of a Transformer -- and introduce the RASP-Generalization Conjecture: Transformers tend to length generalize on a task if the task can be solved by a short RASP program which works for all input lengths. This simple conjecture remarkably captures most known instances of length generalization on algorithmic tasks. Moreover, we leverage our insights to drastically improve generalization performance on traditionally hard tasks (such as parity and addition). On the theoretical side, we give a simple example where the "min-degree-interpolator" model of learning from Abbe et al. (2023) does not correctly predict Transformers' out-of-distribution behavior, but our conjecture does. Overall, our work provides a novel perspective on the mechanisms of compositional generalization and the algorithmic capabilities of Transformers.
    2024-01-07 Tags: , , , by klotz
  4. Retrieval-Augmented Generation (RAG) models, which merge pre-trained parametric and non-parametric memory systems, such as Wikipedia, to enhance language generation in knowledge-intensive NLP tasks, outperforming traditional seq2seq models and setting new standards in open domain QA tasks​
    2023-11-16 Tags: , , by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0 SemanticScuttle - klotz.me: Tags: arxiv

About - Propulsed by SemanticScuttle