Analysis of Prompt Chaining Technology

To enhance the performance and reliability of Large Language Models (LLMs), a crucial prompt engineering technique is decomposing tasks into multiple subtasks. After identifying subtasks, prompt words for each subtask are provided to the language model, and the results serve as part of new prompt words. This is known as prompt chaining—a task is broken down into multiple subtasks, and a series of prompt operations are created based on these subtasks.

Prompt chaining enables the completion of complex tasks that LLMs may fail to handle with a single, highly detailed prompt. In prompt chaining, the prompt chain performs transformations or other processing on generated responses until the desired result is achieved.

Beyond improving performance, prompt chaining also enhances the transparency, controllability, and reliability of LLM applications. This means issues in the model can be more easily identified, and the performance of different stages can be analyzed and improved.

Prompt chaining is particularly useful for building LLM-driven conversational assistants and enhancing the personalized user experience of applications.

Example of Prompt Chaining in Document QA

Prompt chains can be applied to scenarios involving multiple operations or transformations. For instance, a common use of LLMs is answering questions based on large text documents. To better process lengthy documents, two distinct prompts can be designed:

• The first prompt extracts relevant citations from the document to address the question.
• The second prompt uses these citations and the original document as input to formulate the final answer.

In other words, two separate prompts can be created to execute the task of answering questions based on documents.

Example of the first prompt (extracting relevant citations from the document based on a question):

Note: For simplicity, a placeholder {{Document}} is used. To test this prompt, you can copy and paste an article from Wikipedia, such as this page on prompt engineering. Due to the long context required for this task, we used OpenAI’s gpt-4-1106-preview model. You can also use this prompt with other long-context LLMs (e.g., Claude).