Stories by gpjt

Writing an LLM from scratch, part 32d – Interventions: adding attention bias

Writing an LLM from scratch, part 32c – Interventions: removing dropout

Writing an LLM from scratch, part 32B – Interventions: gradient clipping

Writing an LLM from scratch, part 32a – Interventions: training a baseline model

Getting a Custom PyTorch LLM onto the Hugging Face Hub

Writing an LLM from scratch, part 31 – the models are now on Hugging Face

Writing an LLM from scratch, part 30 – digging into the LLM-as-a-judge results

LLM from scratch, part 29 – using DDP to train a base model in the cloud

LLM from scratch, part 28 – training a base model from scratch on an RTX 3090

Writing an LLM from scratch, part 27 – what's left, and what's next?

Writing an LLM from scratch, part 26 – evaluating the fine-tuned model

Writing an LLM from scratch, part 25 – instruction fine-tuning

Writing an LLM from scratch, part 24 – the transcript hack

Retro Language Models: Rebuilding Karpathy's RNN in PyTorch

Writing an LLM from scratch, part 23 – fine-tuning for classification

Writing an LLM from scratch, part 22 – training our LLM

Revisiting Karpathy's 'Unreasonable Effectiveness of Recurrent Neural Networks'

Writing an LLM from scratch, part 21 – perplexed by perplexity

Writing an LLM from scratch, part 20 – starting training, and cross entropy loss

How Do LLMs Work?

1
2