Importance

Loading Importance Indicators...

Abstract

A technique produces a reduced-size language model using explanation tuning. Explanation tuning composes a prompt that includes two parts: a system instruction and a client instruction. The client instruction expresses a query. The system instruction requests a language model to formulate responses to queries that describe final results and processes of producing the final results. The language model responds to the prompt by providing a language-model response that describes a final result and a process of producing the final result, e.g., by providing a step-by-step explanation of how the final result is derivable. In some implementations, the technique uses a teacher-student approach to producing the reduced-size language model. In some examples, the technique performs training in two stages using two respective teacher language models, the first-stage model being less versatile and accurate compared to the second-stage model.

Loading the Abstract Image...

First Claim