PROPAGATING ATTENTION INFORMATION IN EFFICIENT MACHINE LEARNING MODELS
Number of patents in Portfolio can not be more than 2000
United States of America Patent
Stats
-
N/A
Issued Date -
May 16, 2024
app pub date -
Jun 15, 2023
filing date -
Nov 11, 2022
priority date (Note) -
Published
status (Latency Note)
Importance
Abstract
Certain aspects of the present disclosure provide techniques and apparatus for improved attention-based machine learning. A first attention propagation output is generated using a first transformer block of a plurality of transformer blocks, this generation including processing input data for the first transformer block using a first self-attention sub-block of the first transformer block. The first attention propagation output is propagated to a second transformer block of the plurality of transformer blocks. An output for the second transformer block is generated, this generation including generating output features for the second transformer block based on the first attention propagation output.
First Claim
Family
International Classification(s)
Cited Art Landscape
Patent Citation Ranking
Recipient Email Address
Recipient Email Address
Comment
Recipient Email Address
Add to Portfolio(s)
To add this patent to one, or more, of your portfolios, simply click the add button.
This Patent is in these Portfolios:
Add to additional portfolios:
Last Refreshed On:
Changes done successfully