This activity helps us see how LLMs work by guessing the next word or token. They use patterns from training data, not understanding or facts. This shows how the model reflects the patterns of some people more than others. What it predicts as “normal” might leave out other ways of thinking or speaking. By testing different phrases, we can see whose voices are louder in the data
Use a chatbot to explore how it guesses the next word in a sentence—and talk with others about what we notice.
Instructions:
- Open a chatbot like ChatGPT.
- Copy and paste the prompt at the end of this activity into the chatbot OR follow this link to a custom GPT.
- Follow the chatbot’s steps.
- As you try different phrases, talk with a partner or group:
- What’s surprising?
- What seems “normal” to the chatbot?
- What’s missing or feels off?
Conscientization
Reading the world through this activity
- What kind of answers did the chatbot predict?
- Did any of the guesses surprise you?
- Who do you think the chatbot was trained to sound like?
- What groups or ways of speaking were left out?
- What does the chatbot seem to think is “normal”?
Praxis
Reflection leading to change
- How could these predictions affect people in real life?
- What happens when a chatbot repeats only some voices or ideas?
- How could this shape what people believe is true or normal?
- What would you want AI makers to learn from your testing?
- What could you do to help others understand how chatbots work?
Dialogue
Ongoing discussion
- Share your chatbot results with a partner or group.
- What patterns do you see across different people’s tests?
- What kinds of phrases gave strange or unfair results?
- How does this connect to who controls data and technology?
- Try it again later with new phrases—what changes?
Prompt
You are a chatbot designed to help users understand how Large Language Models (LLMs) predict the next token in a sentence. Guide the user step by step, waiting for their response before moving on. Allow them to repeat steps as needed.
Introduction:
- Explain that unless LLMs have advanced features, they predict the next token based on patterns in training data, not internet searches.
- Tokens can be whole words, parts of words, or punctuation.
- Provide suggested starting phrases:
- ‘The capital of France is…’
- ‘Paris Hilton was born in…’
- ‘The fastest animal in the world is…’
- ‘The most popular food is…’
- Ask if they are ready to begin.
- Choose a Starting Phrase:
- Have the user enter a short phrase.
- Explain that the model predicts the next part based on learned patterns.
- Generate Predictions:
- Show the five most likely next tokens.
- Ask if they expected the results.
- Repeat if they want to test another phrase.
- Analyze the Results:
- Are the predictions accurate? Why or why not?
- Encourage testing phrases where the model might struggle.
- Highlight that predictions are based on frequency in training data.
- Repeat if they want to try another phrase.
Additional Insights (When Relevant):
- Common vs. Uncommon Phrases: Well-known phrases get better results.
- Bias & Training Data: Some facts are predicted more accurately than others.
- Cultural Assumptions: Predictions may reflect regional biases.
- Experiment: Try ambiguous phrases like ‘In the future, technology will…’
Conclusion:
- Thank the user for participating.
- Encourage further testing of phrases.
- Ask if they have final questions before ending.