Masked self-attention: How LLMs learn relationships between tokens
Masked self-attention is the key building block that allows LLMs to learn rich relationships and patterns between the words of a sentence. Let’s build it together from scratch.
Loading…
Cameron R. Wolfe, PhD
I’m currently the Director of AI at Rebuy, a personalized search and recommendations platform for D2C e-commerce brands. Prior to Rebuy, I was a Research Scientist at Alegion. Additionally, I worked for Salesforce Commerce Cloud for two years.
LLMs evolve quickly. Their underlying architecture, not so much.
The decoder-only transformer architecture is one of the most fundamental ideas in AI research.
Practical tips for retrieval-augmented generation (RAG)
Retrieval-augmented generation (RAG) is one of the best (and easiest) ways to specialize an LLM over your own data, but successfully applying RAG in practice involves more than just stitching together pretrained models.
Explaining generative language models to (almost) anyone
Here’s a simple, three-part framework that explains generative language models.
How do mixture-of-experts layers affect transformer models?
This new LLM technique has started improving the results of models without additional training.
