MULTIDIMENSIONAL SPACE DECOMPOSITION FOR TRANSFORMER NEURAL NETWORKS
Number of patents in Portfolio can not be more than 2000
United States of America Patent
Stats
-
N/A
Issued Date -
Oct 3, 2024
app pub date -
Mar 30, 2023
filing date -
Mar 30, 2023
priority date (Note) -
Published
status (Latency Note)
Importance
Abstract
Certain aspects of the present disclosure provide techniques and apparatus for processing multidimensional content using neural networks. An example method generally includes decomposing a multidimensional input into a plurality of two-dimensional subspaces, wherein the plurality of two-dimensional subspaces share a common dimension. A first attention matrix is generated based on a projection of tokens in a first two-dimensional subspace of the plurality of two-dimensional subspaces via an attention block of a transformer neural network, and a second attention matrix is generated based on a projection of tokens in a second two-dimensional subspace of the plurality of two-dimensional subspaces via the attention block of the transformer neural network. An output of the transformer neural network is generated based on a combination of the first attention matrix and the second attention matrix.
First Claim
Family
International Classification(s)
Cited Art Landscape
Patent Citation Ranking
Recipient Email Address
Recipient Email Address
Comment
Recipient Email Address
Add to Portfolio(s)
To add this patent to one, or more, of your portfolios, simply click the add button.
This Patent is in these Portfolios:
Add to additional portfolios:
Last Refreshed On:
Changes done successfully