AI

AI-generated content is everywhere now. Your morning Instagram doom-scrolling routine, those 47 unread emails in your inbox that arrived overnight, the friendly assistant that tells you that you are absolutely right - I'm sure at least some of that was powered by large language models. These are usually run on powerful hardware, in large fancy data centers consuming ridiculous amounts of power.

But what if I told you it's possible to run AI from the comfort of your own home, right on your PowerMac G3 workstation?

~/src/llama2.c % uname -srm
NetBSD 10.1 macppc

~/src/llama2.c % ./run sw-stories15M.bin \
  -z sw-tokenizer.bin \
  -i 'Mary had a little lamb.'
Mary had a little lamb.
She was very gentle and kind.
One day she was walking in the forest
when she saw something very strange.
There was a dragon
and it was making a loud roar.
(...)
achieved tok/s: 2.702613

…