Stories by gpjt

LLM from scratch, part 28 – training a base model from scratch on an RTX 3090

Writing an LLM from scratch, part 27 – what's left, and what's next?

Writing an LLM from scratch, part 26 – evaluating the fine-tuned model

Writing an LLM from scratch, part 25 – instruction fine-tuning

Writing an LLM from scratch, part 24 – the transcript hack

Retro Language Models: Rebuilding Karpathy's RNN in PyTorch

Writing an LLM from scratch, part 23 – fine-tuning for classification

Writing an LLM from scratch, part 22 – training our LLM

Revisiting Karpathy's 'Unreasonable Effectiveness of Recurrent Neural Networks'

Writing an LLM from scratch, part 21 – perplexed by perplexity

Writing an LLM from scratch, part 20 – starting training, and cross entropy loss

How Do LLMs Work?

The maths you need to start understanding LLMs

What AI chatbots are doing under the hood

LLM from scratch, part 18 – residuals, shortcut connections, and the Talmud

The fixed length bottleneck and the feed forward network

Writing an LLM from scratch, part 17 – the feed-forward network

Writing an LLM from scratch, part 16 – layer normalisation

Leaving PythonAnywhere

Writing an LLM from scratch, part 15 – from context vectors to logits

1