TRANSFORMER NETWORK WITH NORMALIZATION INCLUDING SCALING PARAMETER
Number of patents in Portfolio can not be more than 2000
United States of America Patent
Stats
-
N/A
Issued Date -
Sep 26, 2024
app pub date -
Feb 28, 2023
filing date -
Feb 28, 2023
priority date (Note) -
Published
status (Latency Note)
Importance
Abstract
A computing system is provided, including a processor configured to receive a training data set. Based at least in part on the training data set, the processor is further configured to train a transformer network that includes a plurality of layers. The plurality of layers each respectively include a plurality of sub-layers including an attention sub-layer, a feed-forward sub-layer, and a plurality of normalization sub-layers. The plurality of normalization sub-layers are downstream from corresponding sub-layers of the plurality of sub-layers. Each of the plurality of normalization sub-layers is configured to apply layer normalization to a sum of: a first scaling parameter multiplied by an input vector of the sub-layer; and an output vector of the sub-layer.
First Claim
Family
International Classification(s)
Cited Art Landscape
Patent Citation Ranking
Recipient Email Address
Recipient Email Address
Comment
Recipient Email Address
Add to Portfolio(s)
To add this patent to one, or more, of your portfolios, simply click the add button.
This Patent is in these Portfolios:
Add to additional portfolios:
Last Refreshed On:
Changes done successfully