klotz: prompt engineering*

0 bookmark(s) - Sort by: Date ↓ / Title / - Bookmarks from other users for this tag

  1. Amanda Caswell writes about Claude Fable 5.1, Anthropic's latest high-end model, and proposes a single prompt designed to showcase its agentic capabilities. The prompt asks the model to critique an existing project the user is already working on, identify what's wrong with their thinking, and then refactor or redesign the whole thing while explaining its reasoning at each decision point. Caswell tested it by uploading her entire book manuscript without giving any hints about what she thought was wrong, and found the model preserved her voice while surfacing issues she had missed.
    - Features a 1-million-token context window, suited for large codebases and lengthy documents
    - Anthropic claims reduced costs on agentic workloads through cheaper caching
    - Caswell notes that LLM explanations are justifications, not literal transcripts of internal reasoning
    - She contrasts Claude's more questioning approach with ChatGPT and Gemini, which she says tend to execute edits without challenging assumptions

    >": I'm building describe a real project you're actually working on, in detail » . Here's my current approach: paste your code, outline, or plan » . Tell me what's wrong with my thinking, what I'm missing, and then refactor or redesign the whole thing — but explain your reasoning at each decision point so I can learn from it."
  2. Erik Kristensen and Napalys Klicius write about four changes to the GitHub Copilot harness that reduce token costs without sacrificing task quality. The central insight is that optimizing individual tool calls is the wrong metric — a shorter response can cost more overall if it forces the agent to rerun commands or reread output. The four changes are: selectively compressing repetitive build/test/install output while preserving source-like content, removing unused line-number prefixes from file reads, halving the task-tool prompt via a meta-prompting loop, and batching background completion notifications so results arrive without an extra retrieval turn. Each was validated through offline agentic benchmarks and controlled online A/B experiments before shipping.

    - RTK (Rust Token Killer) was evaluated and found to increase end-to-end cost despite shortening individual responses, because the agent reopened or reran commands to recover omitted details.
    - The prompt compression initially caused a regression that offline tests missed: cautious parallelism guidance was rewritten into a hard scheduling policy, serializing independent agents. The fix was a single sentence: "Independent agents can run in parallel; consider side effects."
    - A tighter file-tool instruction set that worked in Copilot code review actually increased cost in Copilot CLI, illustrating that evidence is local to the workload.
    - The changes ship across all Copilot products sharing the same harness (CLI, app, code review); code review separately saw ~20% cost reduction from a prior migration to shared file tools.
  3. Meredith Shubel writes that Vercel published `design.md`, a public prompt file that cut agent-generated design failures by 57% across 200+ agent runs, though none of the six tested pages was ship-ready. The system has three layers: a prompt encoding design judgment, a public stylesheet for mechanical layout rules, and an evaluation loop that converts human feedback into deterministic checks. A Slack-based agent (`design-agent`) consolidates weekly feedback from GitHub and Figma into proposed guidance updates.

    - The comparison test used Codex with GPT-5.5: 39 failure instances with `design.md` versus 91 without.
    - Vercel's first attempt to port its internal "product design" skill to a public prompt failed because subjective design language was interpreted differently by each model.
    - Recurring complaint counts are tracked over time; if a fix doesn't reduce its count, the fix is flagged for refinement.
  4. Mahnoor Faisal writes that OpenAI Codex tends to over-engineer simple tasks by refactoring surrounding code, adding abstractions and defensive guards not requested, and she fixes this by appending a single boundary line to every prompt telling it to make the smallest change that fully solves the task and not add extras unless strictly required.

    - The same one-line tweak previously improved prompts for Claude, Claude Code, NotebookLM and ChatGPT
    - Over-scoping complaints are common on Reddit, especially with GPT-5.6 Sol
    - OpenAI'''s focus on long-running autonomous work makes the model eager to find adjacent improvements

    >"Make the smallest change that fully solves the task. Do not add abstractions, fallbacks, defensive guards, refactors, or features unless they are strictly required.”
  5. Kanwal Mehreen writes that as large language models move from chatbots to coding agents and autonomous workflows, good prompting is no longer enough and specification engineering is emerging as the key skill. It is the practice of defining goals, constraints, expected outputs, edge cases, tests, success criteria and failure modes so AI-assisted work is testable and reviewable. Prompt engineering asks how to ask, while specification engineering defines what done correctly means, a shift closer to product management, testing and research design than traditional prompting.

    - A 2024 Requirement-Oriented Prompt Engineering paper found ROPE training improved requirement-writing ability by 20% versus 1% for conventional prompt engineering training.
    - OpenAI Structured Outputs, Model Spec and Anthropic Constitution exemplify specification engineering at API and model-behavior levels.
    - SWE-bench Verified is a human-validated subset used to evaluate real-world software issue solving with agents.
    - Google's 2025 DORA report surveyed nearly 5,000 tech professionals and found AI amplifies existing organizational strengths and weaknesses.
  6. Microsoft’s POML offers a declarative way to manage complex instructions by using XML-like tags and CSS-style rules for prompt design. By separating content from stylistic parameters like tone and token limits, the system promotes modularity, reusability, and improved maintainability compared to manual string concatenation.

    - Requires Python 3.10 or higher
    - Facilitates cleaner version control through structured diffs in pull requests
    - Enables non-engineers like product managers to contribute using familiar syntax styles
    - Allows for model-agnostic structures that separate intent from specific API formatting quirks
  7. Yuge Zhang writes about Prompt Orchestration Markup Language (POML), a framework designed to manage complex prompt structures and diverse data types for Large Language Models. It uses component-based markup and specialized tags to integrate documents, tables, and images while employing a CSS-like styling system to decouple content from its presentation format. This approach aims to reduce formatting sensitivity and improve the reliability of prompts when interacting with models.

    - Validated through PomLink application integration and TableQA accuracy tests.
    - Includes templating for dynamic prompt generation and an SDK/IDE toolkit.
    - Designed to enhance version control and collaboration in development workflows.
  8. Yuge Zhang et al. write about Prompt Orchestration Markup Language (POML), a markup language designed to bring structure, maintainability, and versatility to prompt engineering for Large Language Models. By employing an HTML-like syntax, POML modularizes components such as roles and tasks while decoupling content from presentation via a CSS-inspired styling system.

    - Includes built-in templating with support for variables, loops, and conditionals
    - Provides SDKs for Python and Node.js integration into application workflows
    - Offers a Visual Studio Code extension featuring syntax highlighting and real-time previews
    - Supports seamless embedding of external data sources like images and spreadsheets via specialized components
  9. Emmimal P Alexander writes that while prompt engineering focuses on optimizing LLM inputs, managing these templates within evolving codebases often leads to production crashes when variables are renamed or removed. To solve this, she created `promptctl`, a Python tool that applies static analysis—similar to database schema migrations—to ensure prompt variable contracts match their call sites in the codebase.

    - Performs PromptDiff (detects changes), Contract Validation (checks mismatches), and Impact Analysis (traces dependencies).
    - Operates strictly via AST parsing, requiring zero LLM calls or API keys.
    - Detects errors that unit tests often miss by mocking away the actual string formatting step.
  10. >"Enterprise Document Intelligence – A fixed BASE, the rules each question needs, one registry: the dispatcher that turns a parsed question into a typed LLM call"

    Instead of "mega-prompts," use a Dispatcher Pattern to assemble a `BASE` prompt with specific fragments (shape and constraints) at runtime. This improves accuracy, simplifies maintenance, and aids auditing.

    * Modular Prompting: Uses "shape fragments" (formatting/extraction) and "constraint fragments" (specific rules).
    * Execution Modes: Combined (sends all chunks at once) vs sequential (Iterative chunk processing to save costs)
    * Structural Scoping: Uses query hints (e.g., page numbers) to refine retrieval.
    * Best Practices: Use Temperature 0, maintain a 20–30% context window buffer, and log raw model responses.

Top of the page

First / Previous / Next / Last / Page 1 of 0 SemanticScuttle - klotz.me: Tags: prompt engineering

About - Propulsed by SemanticScuttle