Write Prompts Like a Pro: Checkout these Prompt Engineering Tools

The rise of ChatGPT has led to a boom in large language models (LLMs) in recent years. Prompt engineering has become an important skill for engineers, whether for generating code with LLMs or using custom LLMs in projects.

Recently, I started learning prompt engineering techniques and tools to solve a tricky problem for our product, LiveAPI— a super-convenient API docs generation tool.

In this article, we’ll explore some prompt engineering tools I discovered, including those for creating prompts, testing and experimenting, managing prompts, and popular prompt libraries on GitHub.

Prompt Generation Tools

YiVal

YiVal helps users generate and evaluate prompts for AI models. YiVal addresses challenges such as prompt development, fine-tuning issues, evaluation, and performance testing.

Here is a demo of prompt generation and evaluation using YiVal.

GPT Prompt Engineer

GPT Prompt Engineer automates the prompt engineering process by generating and testing multiple prompts to find the most effective ones. The system tests each prompt against all test cases, compares their performance, and ranks them using an ELO rating system.

PromptHub Prompt Generator

PromptHub provides an automatic prompt generator that creates high-quality prompt templates optimized for various models.

Prompt Testing and Experimentation Tools

OpenPrompt

OpenPrompt is a open-source framework for prompt-learning. OpenPrompt is a flexible and extensible framework that supports various prompt-learning methods. It can be used to adapt pre-trained language models to downstream NLP tasks.

Guidance

Guidance is a programming paradigm designed to steer large language models. It allows users to control the structure of the output and get high-quality results tailored to their needs. It’s like giving the LLM a set of rules to follow while generating text.

For example, you could use Guidance to generate a poem that rhymes and has a specific number of lines, or to write a code snippet that adheres to certain formatting standards. This can help ensure that the LLM’s output meets your specific needs and expectations.

print(guide("Generate a haiku about a cat.",
 stop=".", # Stop generating after a period
 max_tokens=17, # Limit the output to 17 tokens
 ))

In this example, the guide function is used to generate a haiku about a cat. The stop parameter ensures that the LLM stops generating after a period, and the max_tokens parameter limits the output to 17 tokens, which is the typical length of a haiku (short poem).

Prompt Tools

prompttools is an open-source tool for experimenting with, testing, and evaluating LLMs, vector databases, and prompts.

ChatGPT Queue

ChatGPT Queue is a Chrome extension that allows to queue up the messages for ChatGPT in advance. It supports bulk prompting and reduced waiting time.

Prompt Management Tools

Agenta

Agenta provides end-to-end tools for the entire LLMOps workflow: building (LLM playground, evaluation), deploying (prompt and configuration management), and monitoring (LLM observability and tracing). A prompt management system enables everyone on the team—from product owners to subject matter experts—to collaborate in creating prompts.

Here is the documentation for prompt management in Agenta.

LangBear

LangBear is an open-source prompt management platform for LangChain. LangBear allows for on-the-fly editing and deployment of prompts to production. Beyond the management, it offers an array of features to elevate prompt quality, including localization, monitoring, ratings, and A/B testing capabilities.

Best Prompt Libraries

Prompt libraries have ready-to-use prompts designed to optimize interactions with LLM. Prompts in this library will give us an idea about how to write effective prompts for specific tasks. Here are a few prompt libraries available in Github.

ai-boost/awesome-prompts

A large collection of top-rated GPT prompts, covering prompt engineering, attack and defense strategies, and academic research.

GPTS-Prompt-Collection

The GPTS-Prompt-Collection offers a diverse array of prompts covering numerous categories, including Writing, Development, Productivity, Business, Education, Finance, Health, Marketing, Sales, Games, and Design, etc.

abilzerian/LLM-Prompt-Library

Another collection of prompts for large language models like Siri and GPT-4, designed for applications including text manipulation, medical assistance, and code generation.

Prompt-hacker-collections

This repository contains resource on the study and practice of prompt-injection attacks, defenses, and interesting examples. It contains a collection of examples, case studies, and detailed notes aimed at researchers, students, and security professionals interested in this topic.

Prompt engineering is a rapidly growing skill that is undergoing many changes. In this article, we discussed a few tools and libraries related to prompt engineering. You can join prompt engineering communities like

• altenens.is
• Hugging Face Forum
• onehack.us to share prompts, provide feedback, and engage in discussions about prompt engineering.

We are currently developing LiveAPI, a super-convenient API docs generation tool designed for teams and organizations. LiveAPI helps create API documentation for your web projects quickly and efficiently within minutes. Try it out and share your feedback!

LiveAPI: Super-Convenient API Docs That Always Stay Up-To-Date

Many internal services lack documentation, or the docs drift from the code. Even with effort, customer-facing API docs can be hard to use. With LiveAPI, connect your Git repository to automatically generate interactive API docs with clear descriptions, references, and "try it" editors.