# ML Club Video: The Transformer
Date: 2024-02-20
Tags: ChatGPT, Natural Language Processing, Time Series

![Optimus Prime, the Transformers movie robot, as a joke reference to the post's actual topic: the transformer neural network architecture](/assets/images/2024/02/image-598x1024.png){: w="250"}

No, not that kind of transformer (sorry Optimus)!

The transformer is an neural network architecture proposed at Google for dethroning the LSTM. Rather than having a “sliding window” like LSTMs and RNNs, the transformer looks considers every word, every value, all together for prediction. This concept of attention, where every word is factored in when determining internal representations and predictions is shown to be so powerful, that researchers found that attention itself can provide great performance. In other words, **Attention is All You Need**. (In fact that is exactly what the [paper](https://arxiv.org/pdf/1706.03762.pdf) is titled!)

The transformer architecture is what is behind the language models of today – from Google Translate to ChatGPT. Want to learn how these work? Watch the video!

[Interactive: ML Club Video: Transformers](https://www.youtube.com/embed/PC1PDz9mYU0?feature=oembed)