Thursday, January 25, 2024

Prompt patterns

 Just sharing a summary of  a paper that tried to develop a catalog of prompt patterns. The sourcez;

"A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT" by Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, Douglas C. Schmidt. Arxiv. https://doi.org/10.48550/arXiv.2302.11382 

  1. Meta Language Creation Pattern: Focuses on creating a custom language for LLMs to improve their understanding of prompts.
  2. Output Automater Pattern: Aims to automate the generation of actionable steps or scripts in response to prompts.
  3. Flipped Interaction Pattern: Involves reversing the typical interaction flow, with the LLM posing questions to the user.
  4. Persona Pattern: Assigns a specific persona or role to an LLM to guide its output generation.
  5. Question Refinement Pattern: Enhances the LLM's responses by refining the user's questions for clarity and focus.
  6. Alternative Approaches Pattern: Encourages the LLM to offer different methods or perspectives for tackling a task.
  7. Cognitive Verifier Pattern: Involves the LLM generating sub-questions to better understand and respond to the main query.
  8. Fact Check List Pattern: Guides the LLM to produce a list of facts or statements in its output for verification.
  9. Template Pattern: Involves using a predefined template to shape the LLM's responses.
  10. Infinite Generation Pattern: Enables the LLM to continuously generate output without repeated user prompts.
  11. Visualization Generator Pattern: Focuses on generating text outputs that can be converted into visualizations by other tools.
  12. Game Play Pattern: Directs the LLM to structure its outputs in the form of a game.
  13. Reflection Pattern: Encourages the LLM to introspect and analyze its own outputs for potential errors or improvements.
  14. Refusal Breaker Pattern: Designed to rephrase user queries in situations where the LLM initially refuses to respond.
  15. Context Manager Pattern: Controls the contextual information within which the LLM operates to tailor its responses.
  16. Recipe Pattern: Helps users obtain a sequence of steps or actions to achieve a desired result.

Each pattern is detailed with its intent, context, structure, key ideas, example implementations, and potential consequences.

I want to acknowledge a good attempt, but am not sure this list is very intuitive or very helpful. In practical terms, we either ask questions or give tasks, defining some output parameters - like genre, audience, style, etc. However someone might find this helpful to keep thinking. We do need some way of classifying prompts. 

Do AI bots deceive?

The paper, Frontier Models are Capable of In-Context Scheming , arrives at a time when fears about AI’s potential for deception are increasi...