Absurdly Early Bedtimes and the Transformer
Some notes & quotes from recent reads:
Generative AI Exists Because of the Transformer: This Is How It Works
Quotes:
As the model processes this set of words, it produces a vector — or list of values — and adjusts it based on each word’s proximity to work in the training data. This vector is known as a word embedding.
Notes:
Just a really lovely, illustrated and animated guide to a fundamental component of the new wave of fancy AI tools that are popping up all over the place.
Once you understand how this works, it’s impossible not to read all writing output by these systems through the lens of these probabilities and limitations.
Keep reading with a 7-day free trial
Subscribe to Aspiring Generalist to keep reading this post and get 7 days of free access to the full post archives.