Attention mechanisms

Public graph

Created about 6 years ago, updated about 6 years ago · Owners: @mihail

ID26903

Attention Is All You Need ... but scarcity of attention drives Internet world

Intro to Attention

https://machinelearningmastery.com/how-does-attention-work-in-encoder-decoder-recurrent-neural-networks/

The basic idea is to read the input structure twice: once to encode the gist and another time (at each step while decoding) to "pay attention" to certain details.

Attention mechanism

See in the graph Search below

Parents2More general nodes this one belongs to