Enhancing Commonsense Reasoning in LLMs via Knowledge Generation

Large Language Models (LLMs) continue to improve, with one popular technique being the ability to incorporate knowledge or information to help models make more accurate predictions. Using a similar rationale, can models also be used to generate knowledge before making predictions? This is the approach explored in relevant research—generating knowledge as part of the prompt, particularly to investigate its utility in tasks like commonsense reasoning.

Consider a simple prompt example:

Plaintext

`Prompt:
Is part of golf trying to get a higher score than others? Yes or no?

Output:
Yes.`

This error reveals the limitations of LLMs in tasks requiring more world knowledge. How can we improve this through knowledge generation?

First, let’s generate some relevant "knowledge":

Plaintext

`Input: Golf involves trying to get a higher score than others.
Knowledge:
Knowledge 1:
The goal of golf is to complete a set of holes with the fewest strokes. A round of golf typically consists of 18 holes, each played once per round on a standard golf course. Each stroke counts as one point, and the total number of strokes determines the winner of the game.

Knowledge 2:
Golf is a precision club sport where competing players (or golfers) use many types of clubs to hit a ball into a series of holes on a course with the fewest strokes. The objective is to complete the game with the lowest score, calculated by adding the number of strokes used on each hole. The player with the lowest score wins the match.`

The next step is to integrate this knowledge and derive a prediction. We reformat the question into a QA format to guide the answer structure:

Plaintext

`Question: Is part of golf trying to get a higher score than others? Yes or no?

Knowledge: The goal of golf is to complete a set of holes with the fewest strokes. A round of golf typically consists of 18 holes, each played once per round on a standard golf course. Each stroke counts as one point, and the total number of strokes determines the winner of the game.

Explanation and Answer:
Answer 1 (very high confidence):
No, the goal of golf is not to get a higher score than others. Instead, the objective is to complete the set of holes with the fewest strokes. The winner is determined by total strokes, not total score.`

Plaintext

`Question: Is part of golf trying to get a higher score than others? Yes or no?

Knowledge: Golf is a precision club sport where competing players (or golfers) use many types of clubs to hit a ball into a series of holes on a course with the fewest strokes. The objective is to complete the game with the lowest score, calculated by adding the number of strokes used on each hole. The player with the lowest score wins the match.

Explanation and Answer:
Answer 2 (much lower confidence):
Yes, part of golf involves trying to get a higher score than others. Each player aims to complete the set of holes with the fewest strokes. The winner is determined by total strokes, not total score. The player with the lowest score wins the match.`

An interesting phenomenon occurs in this example: the model is highly confident in the first answer but significantly less so in the second. The process is simplified for demonstration, and there are additional details to consider when deriving final answers. For more information, please refer to the relevant research paper.

分享

把翻译润色得更正式一点

给我一篇关于LLM的英文论文

推荐一些翻译英文的网站

深度思考