Prompt 工程完全指南：LLM 能力最大化

近年来，语言模型进步神速，以 GPT-3 和 GPT-4 为代表的大语言模型（LLM）逐渐成为焦点。这些模型能够以惊人的水平完成种类繁多的任务，因此广受欢迎。此外，随着模型参数量不断增加（高达数十亿！），它们还出人意料地涌现出了一些新能力。

本文将介绍 LLM、它们能够完成的任务、存在的不足，以及各种 prompt engineering 策略。

LLM 是使用海量文本数据训练而成的神经网络。训练过程让模型能够学习文本中的模式，包括语法、句法和词语之间的关联。模型利用这些学到的模式生成人类风格的文本，因此非常适合用于自然语言处理（NLP）任务。

有哪些 LLM 可供使用？

目前有多种 LLM 可供使用，其中 GPT-4 最为流行。其他模型还包括 LLaMA、PaLM、BERT 和 T5。每种模型都有自己的优势和不足：有些是开放的，有些则是封闭的，只能通过 API 使用。

尽管 LLM 的表现令人印象深刻，但它们仍存在一些局限。其中一个显著的缺点是，它们无法在 prompt 所提供的信息之外进行推理。此外，LLM 还可能基于训练数据生成带有偏见的文本。控制 LLM 的输出同样很困难，因此必须借助 prompt engineering 策略才能得到期望的结果。

可以完成哪些任务？

我们可以根据具体场景设计 prompt 的格式，引导 LLM 为我们完成特定任务。下面列出了这些任务，以及每项任务对应的 prompt 模板和示例。

LLM 可以为长文本生成摘要，让内容更容易理解和消化。我们可以使用以下 prompt 模板，让模型总结一段文本：

<Full text>
Summarize the text above: / Explain the text above in <N> sentences:

例如，我们可以使用下面的 prompt，让模型总结这段文本：

In a mechanical watch the watch's gears are turned by a spiral spring called a mainspring. In a manual watch energy is stored in the mainspring by turning a knob, the crown on the side of the watch. Then the energy from the mainspring powers the watch movement until it runs down, requiring the spring to be wound again.
A self-winding watch movement has a mechanism which winds the mainspring using the natural motions of the wearer's body. The watch contains an oscillating weight that turns on a pivot. The normal movements of the watch in the user's pocket (for a pocketwatch) or on the user's arm (for a wristwatch) cause the rotor to pivot on its staff, which is attached to a ratcheted winding mechanism. The motion of the watch is thereby translated into circular motion of the weight which, through a series of reverser and reducing gears, eventually winds the mainspring. There are many different designs for modern self-winding mechanisms. Some designs allow winding of the watch to take place while the weight swings in only one direction while other, more advanced, mechanisms have two ratchets and wind the mainspring during both clockwise and anti-clockwise weight motions.
Explain the text above in 2 sentences:

模型会给出以下输出：

A mechanical watch uses a mainspring to power the movement of the watch's gears. In a manual watch, the mainspring is wound by turning a knob, while a self-winding watch uses the natural motion of the wearer's body to wind the mainspring through an oscillating weight and ratcheted winding mechanism.

信息提取

LLM 可以从大量文本中识别并提取特定信息。这在新闻和商业智能等领域非常有用。下面的模板可以指示模型提取信息：

<Full text>
Mention X in the text above: / List X in the text above:

例如，我们可以让模型从下面的文本中提取地名：

The earliest credible evidence for a successful design is the watch made by the Swiss watchmaker Abraham-Louis Perrelet, who lived in Le Locle. In late 1776 or early 1777, he invented a self-winding mechanism for pocket watches using an oscillating weight inside the watch that moved up and down. The Geneva Society of Arts, reporting on this watch in 1777, stated that 15 minutes walking was necessary to fully wind the watch.
List the names of places in the text above:

Le Locle, Geneva.

LLM 还可以根据 prompt 中提供的信息回答问题。可以使用下面的模板，引导 LLM 回答问题：

Answer the question based on the context below. <Other instructions>. Respond "Not sure" if not sure about the answer.
Context: <Context>
Q: <Question>
A:

例如，使用下面的 prompt：

Answer the question based on the context below. Answer in a single sentence. Respond "Not sure" if not sure about the answer.
Context: An automatic watch, also known as a self-winding watch or simply an automatic, is a mechanical watch where the natural motion of the wearer provides energy to wind the mainspring, making manual winding unnecessary if worn enough. It is distinguished from a manual watch in that a manual watch must have its mainspring wound by hand at regular intervals.
Q: Do automatic watches need to be manually wound?
A:

ChatGPT 会给出以下回答：

No, automatic watches do not need to be manually wound if worn enough because the natural motion of the wearer provides energy to wind the mainspring.

LLM 可以将文本归入预先定义好的类别，例如进行情感分析或垃圾信息检测。你可以按照下面的格式编写文本分类指令：

Classify the text into <List of choices>:
Text: <Full text>
Classification:

例如，我们可以尝试判断一段文本的情感是积极、中性还是消极：

Classify the text into positive, neutral or negative:
Text: That shot selection was awful
Classification:

Negative

LLM 可以生成类似人类的对话，因此适用于聊天机器人和客户服务应用。我们可以要求 LLM 以特定身份和方式参与对话。

The following is a conversation with <Actor>. The <Actor> should answer <Further instructions>.
Person: <Conversation>
<Actor>: <Response>
Person: <Conversation>
<Actor>:

LLM 会以我们指定的角色类型补全回答。之后，我们可以把包含新回答在内的完整对话再次传给模型，让对话继续下去。例如：

The following is a conversation with a lawyer. The lawyer should answer simply and in layman terms.
Person: Hello, I would like to know how to dispute a parking ticket
Lawyer:

Sure, I can help. The first step is to check the ticket for any errors or mistakes. If you find any, you can use those as grounds to dispute the ticket. If there are no mistakes, you can still dispute the ticket by providing evidence to support your case, such as photos or witness statements. You can also request a hearing to present your case in person.

The following is a conversation with a lawyer. The lawyer should answer simply and in layman terms.
Person: Hello, I would like to know how to dispute a parking ticket
Lawyer: Sure, I can help. The first step is to check the ticket for any errors or mistakes. If you find any, you can use those as grounds to dispute the ticket. If there are no mistakes, you can still dispute the ticket by providing evidence to support your case, such as photos or witness statements. You can also request a hearing to present your case in person.
Person: How do I request a hearing?
Lawyer:

To request a hearing, you'll need to follow the instructions on the back of the ticket or on the website provided. Usually, you'll need to submit a written request within a certain timeframe, along with any evidence or supporting documents you have. Once your request is received, you'll be notified of the date, time, and location of your hearing.

LLM 可以生成代码，让开发者更轻松地创建复杂的软件程序。我们可以编写一段代码注释，描述希望代码完成的工作，LLM 就会尽其所能为我们实现。希望代码完成的事情越复杂，prompt 注释就越需要具体、详尽。

具体模板可能会因要生成的编程语言而异，但我建议大致遵循下面的格式：

<Comment symbol of your language of choice (/* or ''' or #)>
Method: <Name of the method>
Language: <Programming language>
Description: <Description of what the method must do>
Parameters: <Parameters that the method will receive>
Context: <Further information needed to implement the method (URLs, databases, etc.)>
Output: <Output of the method>
<Comment symbol of your language of choice (/* or ''' or #)>

例如，如果我们想调用一个支持分页的 API，可以编写下面这段注释：

/* Method: fetchFirst5Pages
 * Language: Javascript
 * Description: Fetch the first 5 pages from this paginated API: http://example.com/pages/?n=x
 * Context: This API accepts the following URL parameters:
 *   n: the number of the page to fetch
 * Output: the content of the first 5 pages
 */

Here's an example implementation of the fetchFirst5Pages method in JavaScript, assuming the use of the fetch function:

async function fetchFirst5Pages() {
  const baseUrl = 'http://example.com/pages/?n=';
  const pages = [];

  for (let i = 1; i <= 5; i++) {
    const url = baseUrl + i;
    const response = await fetch(url);

    if (!response.ok) {
      throw new Error(`Error fetching page ${i}`);
    }

    const pageData = await response.json();
    pages.push(pageData);
  }

  return pages;
}

This function uses a loop to fetch the first 5 pages from the API by appending the page number to the base URL. It checks if the response from the server is okay and throws an error if it's not. Then, it parses the JSON data from the response and adds it to an array of pages. Finally, it returns the array of pages.

LLM 在一些基础任务上也可能表现不佳，例如算术题。不过，如果我们要求 LLM 对自己的回答进行推理，就能帮助它得到正确答案。我推荐使用下面的模板：

Q: <Question>
A: Let's work this out in a step by step way to be sure we have the right answer. First, <Instructions>.

在这个例子中可以看到，如果不要求推理，一个简单的算术问题也可能回答错误：

Answer the following question. Respond "Not sure" if not sure about the answer.
Q: The sum of prime numbers in the following list are a multiple of 3: [2, 4, 6, 7, 9, 14]
A: [Yes or No]

No. The sum of prime numbers in the list [2, 4, 6, 7, 9, 14] is not a multiple of 3. Therefore, the answer is "No".

但是，如果我们要求 LLM 对答案进行推理，就会得到截然不同的结果：

Q: The sum of prime numbers in the following list are a multiple of 3: [2, 4, 6, 7, 9, 14]
A: Let's work this out in a step by step way to be sure we have the right answer. First, we find the prime numbers in the list and add them together. Then we check whether the resulting number can be divided by 3.

Sure, let's work this out step by step.

The prime numbers in the list are 2 and 7. So we can add them together:

2 + 7 = 9

The resulting number is 9. Now we need to check whether it is a multiple of 3. We can do this by checking if the remainder of 9 divided by 3 is equal to 0:

9 % 3 = 0

Since the remainder is 0, we know that 9 is indeed a multiple of 3.

Therefore, the answer is "Yes", the sum of prime numbers in the list [2, 4, 6, 7, 9, 14] is a multiple of 3.

尽管这些模板可以帮助你根据自身需求充分挖掘 LLM 的潜力，但它们有时仍然难以应对某些复杂任务。为帮助 LLM 得到令人满意的答案，人们已经开发出了一些技术。到目前为止，我们使用的都是 zero-shot prompting，下面将对它进行定义。不过，我们也可以采用更复杂的技术。

本文前面的所有示例使用的都是 zero-shot prompting。zero-shot prompting 是指将 prompt 传给 LLM 时，只提供希望它回答的问题，不给出在其他情况下应该如何作答的示例。例如，在前面的一个示例中，我们使用了：

Classify the text into positive, neutral or negative:
Text: That shot selection was awful
Classification:

Negative

这些上下文可能不足以让模型完全按照我们期望的方式回答。下面看看还有哪些替代方案。

与 zero-shot prompting 不同，few-shot prompting 会在提出问题之前，先给 LLM 几个回答示例。例如，假设我们想像前面的示例一样对文本情感进行分类，但希望它每次都以非常具体的格式给出答案。我们可以事先提供几个示例：

Classify the text into positive, neutral or negative:
Text: Today the weather is fantastic
Classification: Pos
Text: The furniture is small.
Classification: Neu
Text: I don't like your attitude
Classification: Neg
Text: That shot selection was awful
Classification:

Neg

Chain of thought prompting

对于更加复杂的任务，我们可以将 few-shot prompting 与要求模型对答案进行推理结合起来。继续使用之前的例子，可以看到推理和 few-shot prompting 会怎样影响结果：

Q: The sum of prime numbers in the following list are a multiple of 3: [2, 4, 6, 7, 9, 14]
A: The prime numbers in the list are 2 and 7. Their sum is 9. 9 % 3 = 0. The answer is yes.
Q: The sum of prime numbers in the following list are a multiple of 3: [1, 2, 4, 7, 8, 10]
A: The prime numbers in the list are 1, 2 and 7. Their sum is 10. 10 % 3 = 1. The answer is no.
Q: The sum of prime numbers in the following list are a multiple of 3: [1, 2, 3, 4]
A: The prime numbers in the list are 1, 2 and 3. Their sum is 6. 6 % 3 = 0. The answer is yes.
Q: The sum of prime numbers in the following list are a multiple of 3: [5, 6, 7, 8]
A:

The prime numbers in the list are 5 and 7. Their sum is 12. 12 % 3 = 0. The answer is yes.

LLM 已经彻底改变了自然语言处理领域，但要最大限度发挥其潜力，prompt engineering 必不可少。通过了解 LLM 能够完成的任务、它们的不足以及各种 prompt engineering 策略，开发者可以利用 LLM 的力量，创造出创新而有效的解决方案。未来不久，很可能还会出现更多策略和技术，因此需要持续关注这一领域的新进展，进一步释放 LLM 的潜力。此外，随着 LLM 的规模继续扩大，参数量增加数十亿，它们很可能能够完成更多我们现在甚至无法想象的任务。想到这些新工具未来能够实现什么、又会在哪些使用场景中为我们服务，实在令人惊叹。