Now, let's build your very own AI.
A toy : it looks only at the previous word. It does not learn , , or facts. The word splitter is a teaching example, not an .
A Plan That Leads to Real Development
Start small like a toy, understand the principles, then grow it bigger over time.
Gather words
Collect sentences about the topic you want to learn. Just 10 sentences is enough to start.
Count next words
Make a table of how many times each word follows another word.
Compute probability scores
Count each transition, including sentence endings. A candidate's probability is its count divided by the total count after the selected word. Neural language models learn weights instead of just storing this small table.
Compare two selection methods
Choose the most likely candidate (greedy), or draw according to probabilities (sampling). Sentence-end stops generation. Sampling can produce the same result again. Generated text is not checked for truth.
Improve it your way
Add better sentences, more data, and different selection rules to grow it into your own AI.
A Tiny Probability Calculator
The table shows observed frequencies at 1. Temperature changes , not the learned counts or factual accuracy.
There are no candidates yet following the last word of the starting sentence.
| Previous wordNext word | |
|---|---|
Many words — showing only the first 8
Key Concept
Count each transition, including sentence endings. A candidate's probability is its count divided by the total count after the selected word. Neural learn instead of just storing this small table.
A neural language model learns numerical weights from examples. At generation time, those weights and the available determine scores for candidate . The model does not simply look up a stored internet sentence or count only the word immediately before it.
This lab vs a real language model
| This lab (bigram) | Neural language model | |
|---|---|---|
| What it looks at | Only the previous word | The long context it is given |
| What it learns | A table of counts | Numeric weights |
| Embeddings and attention | No | Yes |
| Checks facts by itself | No | No |