SULTAAN.AI

16 July 2026

First contact

The day a model trained from nothing on a laptop stopped producing noise and started producing English. Small, honest, and entirely its own.

The record

The run, in numbers

  • 33,890,816parameters
  • 16Ktoken vocabulary, trained in-house
  • 500public-domain books
  • 54.1Mtraining tokens
  • 6 GBlaptop GPU, an RTX 3050
  • ~26Ktokens per second
  • 20,000training steps
  • 0pretrained weights used

9.78 to 3.89Training loss, first step to last

The first words

First words, at step 250

“It was on that it had said”

Not a sentence anyone would keep. But every part of the chain from raw text to that line, the tokeniser included, was the project’s own, with no borrowed weights.

The claim

The zero is the claim, not the 34 million

A 34-million-parameter model is tiny. What mattered was proving that the project’s own trainer, its own tokeniser and public-domain text could take a model from nothing to word-shaped English on an ordinary laptop.

The first-contact tooling was built for this: a supervisor script that steps the batch size down when the GPU runs out of memory and resumes from the last checkpoint after a crash, so a run can be started once and left overnight. A local playground then let the same prompt run across checkpoints: at step 1,000 the model babbles, and by step 20,000 it writes sentences in the style of the old books it read.

What it cannot do

The project’s identity document says this plainly, and so do we:

  • Cannot answer questions. It is a base model that predicts the next word, with no instruction tuning.
  • Cannot reason or do mathematics.
  • Cannot write code.
  • Cannot understand the document that describes it.

For the same reason the playground is a text-completion page, not a chat window. A chat box would invite questions the model cannot answer and make a correct result look broken. A chat interface becomes honest only after instruction tuning.

Why 500 books

The first corpus was 8 books, about 1.34 million tokens. For a model this size the usual compute-optimal budget is around 680 million tokens, so training on 8 books would have meant roughly 98 passes over the same text: a memoriser, not a language model. The corpus was rebuilt from 500 public-domain books, fetched politely one at a time, with the Project Gutenberg licence header and footer stripped so the model would not learn to recite them.

What comes next

The first-contact weights are proof of life, not a product, and they are not published. The next rung is a roughly 200-million-parameter run on a rented GPU, trained on FineWeb-Edu, with a target of perplexity under 30 on English text.

See the full roadmapInside the kernel