Energy Efficient Computations of Attention-based Inferences
Number of patents in Portfolio can not be more than 2000
United States of America Patent
Stats
-
N/A
Issued Date -
Aug 22, 2024
app pub date -
Jan 17, 2024
filing date -
Feb 16, 2023
priority date (Note) -
Published
status (Latency Note)
Importance
Abstract
An apparatus to compute an attention matrix implementing an attention mechanism in artificial neural networks, having: memory to store key value pairs; a reorder buffer to provide a reordered list of keys from the key value pairs; an analog dot product accelerator configured to compute dot products of key elements of keys from the reordered list of keys with respective query elements of a query row of a query matrix; a processing device configured to generate, based on results of the dot products, a row of attention scores corresponding to the query row of the query matrix for the reordered list of keys; and a further accelerator configured to compute dot products of segments of the attention scores with value elements of respective segments of values from a list of values from the key value pairs to generate an attention matrix.
First Claim
Family
International Classification(s)
Cited Art Landscape
Patent Citation Ranking
Recipient Email Address
Recipient Email Address
Comment
Recipient Email Address
Add to Portfolio(s)
To add this patent to one, or more, of your portfolios, simply click the add button.
This Patent is in these Portfolios:
Add to additional portfolios:
Last Refreshed On:
Changes done successfully