OpenAI introduces GPT-4, a new large language model that surpasses human performance on various tasks. Although not yet publicly available, the article provides insights into its capabilities and how it sets a new standard for AI.
Researchers from NYU Tandon School of Engineering investigated whether modern natural language processing systems could solve the daily Connections puzzles from The New York Times. The results showed that while all the AI systems could solve some of the puzzles, they struggled overall.
This tutorial introduces promptrefiner, a tool created by Amirarsalan Rajabi that uses the GPT-4 model to create perfect system prompts for local LLMs.
- Demonstrates how to improve two pretrained models' proficiency in the Dafny verified programming language.
- Uses 178 programming problems from the MBPP dataset for prompting GPT-4 and PaLM-2 to generate methods in Dafny.
- Three types of prompts were used: a direct contextless prompt, one that includes a signature of the method and test cases, and a third one that decomposes the problem into steps and includes dynamically chosen similar examples.
- GPT-4 was able to generate verified (and human-evaluated) Dafny methods in 58% of the cases with the third prompt.
- Contributes a collection of 153 MBPP problems implemented and formally verified in Dafny, 50 written by authors and 103 synthesized by GPT-4.
- Uses 178 programming problems from the MBPP dataset for prompting GPT-4 and PaLM-2 to generate methods in Dafny.
- Three types of prompts were used: a direct contextless prompt, one that includes a signature of the method and test cases, and a third one that decomposes the problem into steps and includes dynamically chosen similar examples.
- GPT-4 was able to generate verified (and human-evaluated) Dafny methods in 58% of the cases with the third prompt.
- Contributes a collection of 153 MBPP problems implemented and formally verified in Dafny, 50 written by authors and 103 synthesized by GPT-4.
