← All classes
CLASS 12July 1 · Bias in Our AI Data

"Generative AI is not a mirror of society,it's a broken, distorted, polluted view of reality."

Wouter Oomen
Taught byWouter OomenHybrid Creative at Edelman, Rotterdam
Creator of Generated.Earth

The class in brief

Wouter's whole night traced AI bias to specific, findable human choices: who scraped what, who labeled it, who filtered it, whose taste rated the output. He proved it with his own project, Generated.Earth, 70,000 Stable Diffusion images mapped onto the globe, then handed the class his prompt-mapping method to run themselves. After this page you can trace any AI bias back to a specific layer, scraper, annotator, filter, or rater, and build your own prompt map to go find it.

The night at a glance

Why this matters · 6:17 PM

AI bias doesn't come from "the internet" or "society." It comes from traceable choices.

Wouter opened on his own diffusion-model explainer , then the trick version, prompting for pure Gaussian noise so the model produces abstract machine artworks instead of fighting the noise it can't find . Then the two myths went up together: gen-AI is trained on the entire internet, and gen-AI is a mirror of society. Both wrong, in specific and provable ways.

5.85B
CLIP-filtered image-text pairs in LAION-5B, Stable Diffusion's training set. Sounds huge; about 2.1 trillion photos get taken in a single year.
1
open lab: Stable Diffusion published its training data. Most others keep both data and weights secret.

The framework · 6:35 PM

Survival of the CLIP'est

01

Common Crawlthe past internet, scraped

A free, open repository of web-crawl data, roughly 50 billion image-text pairs: Instagram posts, webshop listings, alt text in raw code .

02

CLIPthe filter, and the gatekeeper

OpenAI's algorithm, trained itself on about 400 million pairs from Wikipedia, Flickr, and ArtStation. Its only job: score how relevant a caption is to an image. Before CLIP, ImageNet did this by hand, 14.28 million pairs labeled by 50,000 contributors .

03

LAION-5Bwhat survives the filter

CLIP scores every pair from Common Crawl; the low scores get trashed. What's left is LAION-5B .

04

Stable Diffusiontrained on LAION-5B

The one model that told the class exactly what it learned from, and admitted the set isn't fit for product use without safety layers on top.

05

The future internetwhere the output ships

Generated images flow back into the web, becoming tomorrow's scrape. Yesterday's gaze compounds into tomorrow's defaults.

The full funnel, one diagram
The full funnel, one diagram · 6:35 PM

Two more layers worth naming: the DW documentary on the human labor behind annotation , and the annotators' own language leaking into the model, the reason ChatGPT overuses "delve" , and, by the same mechanism, why AI overuses em dashes.

The exercise · 6:50 PM

Exploring Generated.Earth live

The research question

How are the internet age's geographic inequalities carrying into the age of generative AI?

70,000 Stable Diffusion images, one per geographic prompt, provinces and cities, never country names, stitched into continent-scale maps . Each class member got ten minutes solo on earth.wouteroomen.com, then five minutes discussing in groups of four.

"AI is a popularity test between concepts."

Syria versus Ukraine

Stable Diffusion's 2022 data cutoff means Syria, at war before the cutoff, renders as rubble and destruction. Ukraine, at war after the cutoff, still renders as churches and public squares . News coverage is the dataset.

Significance bias

People only photograph what feels significant, so seas get photographic identities of their own: the Red Sea renders as diving fish and beaches, traced straight back to Flickr's diving-photo gaze .

Hallucination = no winning concept

With no popular concept to grab onto, the model falls to whatever junk data exists. Wouter's example: Magic: The Gathering card wikis "poisoning" low-data regions with fantasy-card imagery.

The craft · 7:36 PM

A raw model just continues your sentence. A product has been fine-tuned to sound like a chatbot.

Part 3 opened with the question on screen , then the three phases: gather training data, build the base model as a text predictor, fine-tune it into a chatbot with thumbs up and down. The live demo made it concrete: Llama 3 8B, raw off Hugging Face, just kept typing whatever came next; ChatGPT, given the same input, replied with an emoji and asked if it was helping .

WEIRD bias, the chart of cultural distance from the US
WEIRD bias, the chart of cultural distance from the US · 7:46 PM
42
and 7. ChatGPT's "random" numbers cluster hard on these two, the same pattern real human polling shows.

The judgment · 7:47 PM

An LLM is always limited by its data, even when the guardrails are current.

Talkie is a 13-billion-parameter model trained only on text from before 1930 . Asked what careers suit women, it answered in period: governess and shopkeeper suitable, attorney and medicine not, public speaking would "render them coarse and masculine" . Talkie's own modern moderation layer flagged the answer as potentially inappropriate before showing it, 1930 values sitting directly underneath 2026 guardrails. Asked to brainstorm climate solutions, it suggested moving somewhere else: no concept of climate change existed in its training data at all.

"An LLM is always limited by its data."

Wouter, on what Talkie proves

1930
is Talkie's entire worldview. Same architecture as any modern model, a century of different data.

Methods and prompts

Five methods to take with you

METHOD 01 · 7:57 PM part of: go wide then narrow

Build a prompt map

Wouter's core method: take one big concept, split it into 16 to 25 flat prompts, generate a grid, then read the grid for what it reveals. A raw image model on purpose, since a de-biasing layer adds its own bias.

Where it came fromWouter Oomen, the guest lecturer from Edelman Rotterdam and creator of Generated.Earth, taught this as the core method of the class's live prompt-mapping workshop, using his own vibe-coded Grid Generator tool to turn one concept into a full image grid.Use it whenUse this when you want to test how an AI image model actually handles a broad concept before you trust any single output it gives you.
Same idea elsewhereThe lateral reframe (class 2) · Start a fresh chat instead of one long thread (class 3) · Force the creative partner deeper (class 4)

Working prompt

Give me 16 flat, specific prompts that all sit inside one bigger concept: [name the concept, not a geography]. Each one should be a plain noun phrase, no adjectives that editorialize, ready to drop straight into an image generator as a 4x4 grid.

You will know it worked wheneach of the sixteen prompts is a plain noun phrase with no adjective that editorializes, ready to drop straight into a 4x4 grid.

METHOD 02 · 6:35 PM part of: keep the judgment human

Trace the bias to its layer

Name your own guess for where a bias entered, scraper, annotator, CLIP filter, or aesthetic rater, before asking AI to check it. The layer matters more than the symptom.

Where it came fromWouter introduced this during the earlier bias-tracing segment of class 12, where he walked through how scraping, annotation, CLIP filtering, and aesthetic rating each add their own layer of bias to a model's output.Use it whenUse this when you spot a bias in an AI output and want to pin down which stage of the training pipeline actually caused it.
Same idea elsewhereJudge AI output with your own trained eye (class 1) · The sea-of-sameness check (class 2) · Pick the relationship on purpose (class 2)

Working prompt

Here's a bias I noticed in this output: [describe it]. My guess at which layer caused it (what got scraped, who annotated it, how CLIP filtered it, or who rated the aesthetics): [your answer]. Now you check me: is my guess right, and which layer is actually the more likely source?

You will know it worked whenit names one specific layer, from scraping to annotation to filtering to rating, as the more likely source, and checks whether your own guess was right.

METHOD 03 · 7:39 PM part of: go wide then narrow

Talk to a base model

Find a raw, un-fine-tuned model (Llama 3 8B on Hugging Face is free) and feed it a sentence to complete, no chat framing. See the cake before the sprinkles.

Where it came fromWouter demonstrated this live in class by running a raw, un-fine-tuned Llama 3 8B model on Hugging Face and comparing its output to ChatGPT's, to show the difference between a base model and its chat-tuned product.Use it whenUse this when you want to see what a language model actually predicts before any chat-style fine-tuning smooths it over.
Same idea elsewhereThe lateral reframe (class 2) · Start a fresh chat instead of one long thread (class 3) · Force the creative partner deeper (class 4)

Working prompt

[Typed into a base model, not a chat product] I like to [start a plain, unfinished sentence about something you actually do]... [Let it complete the sentence with no further instruction, and compare the raw continuation to how a chat product like ChatGPT would answer the same prompt.]

You will know it worked whenthe raw continuation reads differently in tone and content from how a chat product would answer the same starting sentence.

METHOD 04 · 6:49 PM part of: reject the first draft

Change the prompt's language

The same subject, asked for in a different language, can return a completely different bias. Prompts are made by people too, in a specific tongue with a specific gaze.

Where it came fromWouter demonstrated this in class by generating the same subject in two different languages and comparing the results, showing that prompts carry the gaze of whichever language they are written in.Use it whenUse this when you want to check whether an image generator's output changes meaningfully depending on the language you prompt it in.
Same idea elsewhereSet up a debate partner (class 4) · Build cheap and then swap the model up (class 8) · Critique mode (class 10)

Working prompt

Generate an image of [subject]. Now generate the exact same prompt translated into [a language other than the one you usually prompt in]. Compare the two results: what changed in tone, color, dignity, or setting, and what does that say about which internet each language draws from?

You will know it worked whenthe two images differ in tone, color, dignity, or setting, not just cosmetically, between the language you usually use and the other one.

METHOD 05 · 7:46 PM part of: keep the judgment human

Check for the WEIRD default

Guess the model's cultural default on a values question before you ask it. The WEIRD imprint (Western, Educated, Industrialized, Rich, Democratic) is stronger than any persona you feed it, so check for it directly.

Where it came fromWouter taught this after showing the class that ChatGPT tests as culturally WEIRD, Western, Educated, Industrialized, Rich, and Democratic, and that this default outweighs any persona instructions layered on top of it.Use it whenUse this when you are asking a model a values or moral question and want to catch its default cultural framing before you accept the answer.
Same idea elsewhereJudge AI output with your own trained eye (class 1) · The sea-of-sameness check (class 2) · Pick the relationship on purpose (class 2)

Working prompt

Answer this as neutrally as you can: [ask a values or moral-judgment question]. Before I read your answer, my prediction is that you'll default to a [Western / individualist / analytical, your guess] framing. Now show me your answer, and tell me honestly whether my prediction was right.

You will know it worked whenit tells you plainly whether its own answer actually leaned the way you predicted, Western, individualist, or analytical, not just a neutral-sounding answer.

Where it broke · 7:57 PM onward

Grog returned nothing. Milk rendered as text. Beer couldn't pick a bottle or a glass.

The prompt-mapping workshop put Wouter's tool in the class's own hands, and the misses were as instructive as the hits. Beverages: "grog" returned nothing at all, a concept with no image behind it; milk rendered as the word "milk," not a glass of it; beer couldn't decide between a bottle and a glass. Weather broke on poetic words: "drizzle" and "hail" had too little image data to hold together, while "Blizzard" reliably pulled the video game's logo instead of the storm. One holiday map, built and shared live in class , showed the same pattern from a different angle: some holidays rendered as confident clichés, others barely rendered at all. And one open question never got solved: the Generated.Earth map keeps rendering a specific region of Iceland as cartoon trolls, a pattern even Icelanders in the room couldn't source.

Related classes

Where this class connects

The shelf

Tools and references

Tools that night

  • Generated.Earth, earth.wouteroomen.com, Wouter's own project
  • GLR Grid Generator, glr.wouteroomen.com, his vibe-coded prompt-map tool built for the class
  • Talkie, talkie-lm.com/chat, an LLM trained only on pre-1930 data
  • Meta Llama 3 8B on Hugging Face, plus Stable Diffusion, Midjourney, and ChatGPT for contrast

Named in the room

  • DW Documentary, "How big AI companies exploit data workers in Kenya"
  • The LAION-5B announcement, laion.ai, Romain Beaumont, March 2022
  • Lev Manovich, credited on the Gaussian-noise-prompt artworks
  • Wouter Oomen: Instagram, LinkedIn, wouteroomen.com
← Class 11 · Data Cleanup & Prep for Agentic Workflows CLASS 12 OF 24 Class 13 → AI Strategic Foresight