PHOSDEEP / TECHNOLOGY / 02

GENERATIVE AI · FROM FIRST PRINCIPLES

MACHINES THAT MAKE THINGS.

Text, images, code and music, written by software that was never told what to write. Scroll through the machine, from raw data to a finished sentence, and see how it actually works.

01 / DATA

IT STARTS WITH EXAMPLES.

A generative model is trained on an enormous collection of examples: books, websites, photos, code, recordings. It is never handed a rulebook. It learns the patterns that run through all of it: which words follow which, what a face looks like, how a melody resolves.

02 / TOKENS

WORDS BECOME NUMBERS.

Computers only do arithmetic, so text is cut into tokens: whole words, pieces of words, punctuation. Every token has an ID, and every ID maps to a long list of numbers the model can compute with. Type below and watch the cut.

TYPE ANY SENTENCE

Generative▁models▁learn▁patterns,▁then▁create▁something▁new.

Simplified tokenizer. Real ones learn their pieces from data (byte-pair encoding).

03 / EMBEDDINGS

MEANING HAS A SHAPE.

Those lists of numbers are coordinates. During training, words used in similar ways drift close together, and relationships turn into directions: the step from “man” to “woman” is roughly the step from “king” to “queen”.

Hover any point to see its nearest neighbours. Drag to orbit.

04 / NEURAL NETWORKS

A NETWORK OF TINY DIALS.

A neural network is layers of simple units. Each one adds up its inputs, multiplied by adjustable numbers called weights, and passes the result on. Switch the inputs on the left and watch this small network change its mind.

05 / TRAINING

LEARNING IS ROLLING DOWNHILL.

At first the weights are random and the answers are wrong. Training measures how wrong (the loss), works out which way to nudge every weight to reduce it (the gradient), and takes a small step. Billions of steps later the model settles into a valley of low error.

06 / ATTENTION

EVERY WORD LOOKS AT EVERY OTHER.

Transformers, the architecture behind today's language models, let each token weigh every other token while working out what it means. Here, “it” attends mostly to “cat”, which is how the model knows who was tired. Hover a word to move the focus.

Hover a word, or tap its label, to see where its attention goes.

07 / GENERATION

ONE WORD AT A TIME.

A language model does one thing: given the text so far, it outputs a probability for every possible next token. Pick one, append it, repeat. Temperature decides how adventurous the pick is. This is a tiny real model, trained in your browser.

TEXT SO FAR

the model

TEMPERATURE1.00

08 / DIFFUSION

IMAGES FROM NOISE.

Image models learn to reverse noise. They start from pure static and, step by step, remove a little of it in the direction the prompt describes. Big shapes resolve first, fine detail last. Pick a prompt.

09 / MODEL FAMILIES

FOUR WAYS TO GENERATE.

Transformers predict the next token using stacked attention layers. They power chat assistants, code completion and translation.

10 / CONTEXT

THE PROMPT IS THE STEERING WHEEL.

A model can only use what fits in its context window: instructions, retrieved documents, your question and its own answer so far. Anything outside it simply doesn't exist for the model. Better context, better answers.

11 / THE WHOLE MACHINE

PATTERNS, NOT KNOWLEDGE.

Everything you just flew through is prediction from patterns. That is why it can write a sonnet, and why it can be confidently wrong. With good data, clear context and human judgement, it is a remarkable tool.

WARMING UP THE MODEL

THE LAB · 01

WRITE WITH A LANGUAGE MODEL.

Give the model a few words and let it continue, one sampled word at a time. Then hover any word to see the choices it weighed at that step, and how temperature changes them.

THE LAB · 02

PAINT FROM NOISE.

Describe a scene and run the sampler. Watch fifty steps of noise removal in two seconds, then scrub back and forth to see big shapes arrive before detail.

WHERE IT MATTERS

POWERFUL, WITH SHARP EDGES.

Generative AI is already reshaping how work gets done. Using it well means knowing where it shines and where it fails.

01
Writing & knowledge work
Drafting, summarising, translating and answering questions over your own documents.
Everyday
02
Software development
Code completion, explanation, tests and refactoring inside the editor.
Everyday
03
Design & media
Concept art, product imagery, video, voice and music, generated and edited from text.
Growing fast
04
Science & medicine
Proposing protein structures, molecules and materials, then testing the best ideas in the lab.
Research
05
Agents
Models that plan, call tools and take multi-step actions on your behalf, with humans in the loop.
Emerging

CAPABILITIES

WHAT
WE DO.

LLM Applications & Assistants

↗

Retrieval-Augmented Generation

↗

AI Agents & Tool Use

↗

Fine-tuning & Evaluation

↗

Responsible AI & Governance

↗

Generative AI Training

↗

PHOSDEEP INTERNATIONAL

BUILD WITH
GENERATIVE AI.

DISCUSS A PROJECT↗