Encoder-Decoder Architecture in Transformers in Generative AI
Encoder-Decoder Architecture in Transformers
The original Transformer model had two parts: Encoder and Decoder.
1) Encoder
Processes input text and creates contextual representation.
2) Decoder
Generates output using encoder context and previous tokens.
3) Examples
- T5 - Encoder-Decoder
- BERT - Encoder-only
- GPT - Decoder-only
4) Why GPT Uses Decoder Only
Decoder-only architecture is efficient for next-token prediction, which is the foundation of generative text models.
5) Summary
Understanding architecture helps you choose the right model for your use case.
