#ai #basics
A compact, practical reference for foundational AI concepts.
1. LLM (Large Language Model)
A model trained on large amounts of text to understand and generate human-like language.
Why it matters: Powers chat, summarization, code generation, and many language tasks.
2. Tokenization
Breaking text into smaller units called tokens — such as words, subwords, or characters — so a model can process language as discrete inputs.
Why it matters: Determines how the model reads text and affects vocabulary size, context length, and efficiency.
3. Vectorization
Converting tokens or other inputs into numeric vectors (embeddings) that capture semantic meaning and relationships.
Why it matters: Enables similarity search, clustering, and semantic matching.
4. Attention
A mechanism that lets the model focus on the most relevant parts of the input when producing an output.
- Nearby contextual vectors: Tokens can influence each other based on their context and relationships.
- Ambiguous vectors: Attention helps determine the intended meaning of a word by considering surrounding context.
Why it matters: Allows the model to understand relationships between words, including those that are far apart in a sentence.
5. Self-Supervised Learning
A training method where a model learns from raw data without manually created labels, usually by creating its own learning objective such as predicting the next token.
Why it matters: Makes it possible to train models on massive amounts of unlabeled data.
6. Transformer
A neural network architecture built primarily around attention mechanisms and feed-forward layers.
Basic flow: input tokens → attention layers → feed-forward layers → output representations.
Why it matters: Transformers are the foundation of most modern LLMs.
7. Fine-tuning
Further training a pretrained model on task-specific or domain-specific data so it behaves in the desired way.
- Pretraining: The base model learns general language patterns.
- Fine-tuning: The model is adapted to produce the expected type of answers or perform a specific task.
Why it matters: Helps customize a general model for a particular business, domain, style, or task.
8. Few-shot Prompting (Example Prompting)
Providing a small number of examples directly in the prompt so the model can infer the expected pattern without being retrained.
Why it matters: A simple way to guide model behavior at inference time.
9. RAG (Retrieval-Augmented Generation)
A technique that retrieves relevant information from an external knowledge source and provides it to the LLM as context before generating an answer.
Why it matters: Helps ground answers in specific information and can reduce hallucinations.
10. Vector DB (Vector Database)
A database designed to store and search embeddings (vectors) using similarity search.
Why it matters: Commonly used in RAG systems to quickly find documents or passages that are semantically related to a query.
11. MCP (Model Context Protocol)
A protocol that provides a standardized way for AI models to access external tools, systems, and data sources.
Why it matters: Allows an LLM to work with live information and external capabilities instead of relying only on its internal knowledge.
12. Context Engineering
The practice of designing, selecting, and managing the context provided to an AI model so it can produce more reliable and useful results.
Common techniques include:
- Few-shot prompting
- RAG
- MCP
Challenges:
- User preferences: Maintaining and applying relevant user preferences over time.
- Context summarization: Compressing large amounts of previous information while preserving important details.
13. Agent
An AI system that can plan, use tools, take actions, observe results, and continue working toward a goal.
Why it matters: An agent can do more than generate text; it can perform multi-step tasks and interact with external systems.
14. Reinforcement Learning
A learning method where an agent learns through actions and feedback.
The agent:
- Takes an action.
- Receives a reward or penalty.
- Learns which actions lead to better long-term results.
Why it matters: Useful for decision-making, optimization, robotics, and game-playing.
15. Chain of Thought
A reasoning technique where a model breaks a complex problem into multiple intermediate steps instead of trying to produce the final answer immediately.
Basic flow: problem → intermediate reasoning steps → final answer.
Why it matters: Step-by-step reasoning can help models solve complex mathematical, logical, and multi-step problems more effectively.
16. Reasoning Models
Models designed to perform multi-step reasoning and problem solving, making them more effective for tasks involving logic, mathematics, planning, or complex decisions.
Why it matters: Better suited to problems where reaching the answer requires several reasoning steps rather than simple pattern matching.
17. Multimodal Models
AI models that can work with multiple types of data, such as:
- Text
- Images
- Audio
- Video
Why it matters: Allows AI systems to understand and connect information across different modalities.
18. Small Language Model (SLM)
A smaller and more efficient language model designed for specific tasks, domains, or environments.
Common use cases include:
- Company-specific applications
- Task-specific automation
- Edge or local deployments
Why it matters: Smaller models can be faster, cheaper, and easier to deploy than large models.
19. Distillation
A technique where a larger teacher model teaches a smaller student model to reproduce similar behavior.
Why it matters: Helps create smaller, faster, and cheaper models while retaining much of the original model's capabilities.
20. Quantization
A technique that reduces the numeric precision used by a model's weights and activations, such as converting from 32-bit values to 8-bit or 4-bit values.
Why it matters: Reduces model size, memory usage, and inference cost, and can improve speed with only a limited loss in accuracy.
How These Concepts Fit Together
A simplified AI lifecycle looks like this:
Self-supervised learning → Base LLM → Fine-tuning / Prompting → Context Engineering (RAG, MCP) → Agent / Application
For reasoning:
Prompt / Problem → Chain of Thought / Reasoning → Answer
For efficient deployment:
Large Model → Distillation / Quantization → Smaller, faster model
Together, these concepts explain how modern AI systems are trained, customized, connected to external knowledge, made capable of reasoning and action, and optimized for production use.