Skip to content
paul.notebook

#transformer

1 post

Tags
  1. № 004

    Training a Transformer from scratch in the browser: watching attention grow

    No PyTorch and no libraries. A tiny autodiff engine and a Transformer of about fourteen thousand parameters, written in TypeScript. Press a button and it learns to reverse a string of digits within seconds.

    Interactive

    8 min