Lesson 3 of 7
Generative AI
Explore Large Language Models and generative systems that create text, code, and ideas from natural prompts.
Learn it
Imagine having a digital writing buddy that can instantly write poems, explain tricky science concepts, or help you draft code. This is Generative AI! Unlike older AI systems that only sorted data into categories, Generative AI creates brand new content.
Generative tools like ChatGPT are powered by Large Language Models (LLMs). They read billions of sentences from books and the web to learn how words connect. When you type a prompt, the model calculates the most sensible next words to construct a response.
Key terms
- Generative AI
- AI systems designed to generate new, original content such as text, imagery, audio, or software code.
- Large Language Model (LLM)
- A deep learning model trained on massive text datasets to understand and generate human language.
- Transformer
- A neural network architecture utilising self-attention mechanisms to process sequential data in parallel.
- Hallucination
- A phenomenon where an AI generates factually incorrect, misleading, or fabricated information as truth.
- Prompt Engineering
- The practice of designing and refining inputs to guide generative models toward accurate, desired outputs.
Inside an LLM Response
See how a Generative AI processes your input to generate a coherent answer.
- 1Tokenisation: The system breaks your prompt into smaller numerical units called tokens (words, parts of words, or characters).
- 2Context Processing: The transformer architecture uses attention layers to calculate how each token relates to all other tokens in the prompt.
- 3Next-Token Prediction: The model calculates a probability distribution across its entire vocabulary for the next most likely token.
- 4Sampling and Output: The model selects a token based on parameters like temperature, attaches it to the text, and repeats the loop.
Simulating Token Probability Selection
pythonimport random
# Simplified model of next-token probability distribution
prompt = 'The sky is clear and '
token_probabilities = {
'blue': 0.65,
'sunny': 0.20,
'dark': 0.10,
'green': 0.05
}
# Select the top candidate (greedy decoding)
best_next_token = max(token_probabilities, key=token_probabilities.get)
print(prompt + best_next_token)Generative language models assign statistical probabilities to possible following tokens based on the surrounding context and select tokens iteratively.
Try it
What will this Python simulation output when executed?
pythonwords = ['AI', 'models', 'predict', 'the', 'next']
weights = [1, 2, 3, 4, 5]
words.append('token')
print(words[-2] + ' ' + words[-1])Challenge
Write a structured prompt for an AI that asks it to tutor an 11-year-old on photosynthesis using a pirate persona. Include clear constraints on length and vocabulary.
Pick whichever way suits you — every mode earns the same bonus XP.
Write at least 40 more characters to submit.
Mark your own work
Guided walkthrough — 0/5 clues revealed
- Clue 1 locked — reveal it only if you get stuck.
- Clue 2 locked — reveal it only if you get stuck.
- Clue 3 locked — reveal it only if you get stuck.
- Clue 4 locked — reveal it only if you get stuck.
- Clue 5 locked — reveal it only if you get stuck.
Each clue costs 4 XP (never below 18 XP). You'd earn 35 XP right now.
Extension: Test how changing the persona from a pirate to a Victorian scientist changes the generated vocabulary and tone.