Transition to Transformers

Encoder

Decoder

h0

h1

h2

h3

h4

h5

I

enjoyed

the

movie

transformers

Naan

transfarmar

padaththai

s0=h5

s1

s2

s3

Naan

transfarmar

padaththai

rasiththen

Transition to Transformers

Encoder

Decoder

Naan

transfarmar

padaththai

s1

s2

s3

Naan

transfarmar

padaththai

rasiththen

h1

h2

h3

h4

h5

α11

α12

α13

α14

α15

Transition to Transformers

Encoder

Decoder

s1

s2

s3

Naan

transfarmar

padaththai

rasiththen

h1

h2

h3

h4

h5

Feed Forward Networks

Encoder-Decoder Attention

Self-Attention

Feed Forward Networks

Self-Attention

s4

s5

z1

z2

z3

z4

z5

Word Embedding

I

enjoyed

the

movie

transformers

We will see each of these components (and a few more) one at a time in detail and connect them together to synthesize the final architecture

Transition to Transformers

Encoder

Self-Attention

I

enjoyed

the

movie

transformers