A-Z AI & Engineering Glossary
Clear, mathematically grounded definitions for the core terms defining modern artificial intelligence, machine learning, and prompt engineering.
Attention Mechanism
A neural layer that enables transformer models to dynamically weight the relevance of different tokens in an input sequence regardless of their distance from one another.
Byte-Pair Encoding (BPE)
A subword tokenization algorithm that iteratively merges the most frequent pairs of characters or bytes into unified token vocabularies.
CRISPE Framework
A prompting blueprint comprising Capacity/Role, Recipient, Intent, Scenario, Parameters, and Examples/Format for repeatable AI outputs.
Diffusion Models
Generative models (like Flux, Midjourney, and Runway) that synthesize images or video by iteratively removing Gaussian noise from random latents.
Embeddings
High-dimensional vector representations of text, images, or audio where semantic similarity corresponds to spatial proximity (e.g., cosine distance).
Few-Shot Prompting
Providing 2 to 5 representative input-output exemplars inside the prompt context to guide model behavior without modifying underlying neural weights.
Grounding
Restricting an LLM's factual claims strictly to verified external documents or database retrievals to eliminate hallucinations.
Hallucination
A phenomenon where an AI generates factually incorrect, fabricated, or ungrounded assertions while maintaining high conversational confidence.
In-Context Learning
The emergent ability of foundation models to adapt to new tasks during inference purely through prompt instructions without parameter fine-tuning.
LoRA (Low-Rank Adaptation)
A parameter-efficient fine-tuning technique that injects low-rank decomposition matrices into transformer layers, cutting training VRAM requirements by over 70%.
Quantization
Compressing neural network weights from 16-bit or 32-bit floating points down to 4-bit or 8-bit integers (e.g., GGUF, AWQ) to run models on consumer hardware.
RAG (Retrieval-Augmented Generation)
An architectural pattern that retrieves relevant domain documents from a vector store or search index and injects them into the prompt before generating an answer.
Temperature
A hyperparameter controlling the entropy and randomness of token sampling during generation (0.0 for deterministic code, 0.8 for creative writing).
Transformer
The foundational neural network architecture introduced in 'Attention Is All You Need' (2017) based entirely on self-attention mechanisms without recurrence.
Zero-Shot CoT
Encouraging step-by-step reasoning by appending phrases like 'Let us think step by step' to prompt queries.