Skip to main content

Learning path

LLM Fine-Tuning & RAG learning path: everything to study, topic by topic

Comprehensive assessment covering large language model fundamentals, fine-tuning techniques (LoRA, QLoRA, PEFT), retrieval-augmented generation architecture, vector databases, embeddings, prompt engineering, evaluation metrics, inference optimization, deployment, guardrails, agent frameworks, and multimodal models.

Short answer

To learn LLM Fine-Tuning & RAG, work through 6 modules and 22 topics in the order they build on each other. It starts with LLM Fundamentals, then moves on to Fine-Tuning Techniques and RAG Architecture. Use the map as a checklist, or measure your level and study only what is missing.

6
modules
22
topics
45
min to assess

What you will study

Topics are grouped by module. Open one to see its objectives and key concepts.

1 · LLM Fundamentals4 topics
  • Transformer Architecture

    Encoder-decoder structure, self-attention layers, feed-forward networks, positional encodings, layer normalization, residual connections, and decoder-only models

    Objectives

    • Explain the core components of the transformer architecture
    • Understand the difference between encoder-only, decoder-only, and encoder-decoder models

    Key concepts

    • encoder-decoder
    • self-attention
    • feed-forward network
    • positional encoding
    • layer normalization
    • residual connections
    • decoder-only models
    • causal masking
  • Tokenization

    BPE, SentencePiece, WordPiece, tokenizer training, vocabulary size trade-offs, special tokens, subword segmentation, and multilingual tokenization

    Objectives

    • Compare different tokenization algorithms and their trade-offs
    • Understand how vocabulary size affects model performance and efficiency

    Key concepts

    • byte-pair encoding (BPE)
    • SentencePiece
    • WordPiece
    • vocabulary size
    • subword segmentation
    • special tokens
    • tokenizer training
    • multilingual tokenization
  • Attention Mechanisms

    Scaled dot-product attention, multi-head attention, key-query-value projections, flash attention, grouped-query attention, sliding window attention, and KV cache

    Objectives

    • Explain scaled dot-product attention and multi-head attention
    • Understand modern attention optimizations like flash attention and GQA

    Key concepts

    • scaled dot-product attention
    • multi-head attention
    • key-query-value
    • flash attention
    • grouped-query attention (GQA)
    • sliding window attention
    • KV cache
    • attention masking
  • Prompt Engineering

    Zero-shot, few-shot, chain-of-thought, system prompts, prompt templates, instruction tuning interaction, structured output prompting, and prompt injection defense

    Objectives

    • Design effective prompts for different tasks
    • Apply chain-of-thought and few-shot techniques to improve LLM output quality

    Key concepts

    • zero-shot prompting
    • few-shot prompting
    • chain-of-thought (CoT)
    • system prompts
    • prompt templates
    • structured output
    • prompt injection defense
    • temperature and sampling
2 · Fine-Tuning Techniques4 topics
  • Full Fine-Tuning

    End-to-end model fine-tuning, learning rate schedules, gradient accumulation, catastrophic forgetting, mixed-precision training, DeepSpeed, and FSDP

    Objectives

    • Explain the full fine-tuning process and its resource requirements
    • Identify when full fine-tuning is appropriate versus parameter-efficient methods

    Key concepts

    • full parameter fine-tuning
    • learning rate schedules
    • gradient accumulation
    • catastrophic forgetting
    • mixed-precision (fp16/bf16)
    • DeepSpeed
    • FSDP
    • training loss curves
  • LoRA & QLoRA

    Low-rank adaptation, rank selection, alpha scaling, target modules, quantized LoRA, 4-bit NormalFloat, double quantization, and adapter merging

    Objectives

    • Explain how LoRA reduces trainable parameters through low-rank decomposition
    • Configure QLoRA for memory-efficient fine-tuning on consumer hardware

    Key concepts

    • low-rank adaptation (LoRA)
    • rank parameter (r)
    • alpha scaling
    • target modules
    • QLoRA
    • 4-bit NormalFloat (NF4)
    • double quantization
    • adapter merging
  • PEFT Techniques

    Prefix tuning, prompt tuning, adapters, IA3, DoRA, PEFT library usage, comparison of parameter-efficient methods, and multi-task PEFT

    Objectives

    • Compare different parameter-efficient fine-tuning methods and their trade-offs
    • Select the appropriate PEFT technique for a given use case and resource constraint

    Key concepts

    • prefix tuning
    • prompt tuning (soft prompts)
    • adapter layers
    • IA3
    • DoRA
    • PEFT library
    • method comparison
    • multi-task PEFT
  • Dataset Preparation & Curation

    Instruction formatting, data quality filtering, deduplication, synthetic data generation, data augmentation, train/validation splits, dataset size trade-offs, and annotation best practices

    Objectives

    • Design a data curation pipeline for fine-tuning datasets
    • Apply quality filtering and deduplication techniques to improve training data

    Key concepts

    • instruction-response formatting
    • data quality filtering
    • deduplication
    • synthetic data generation
    • data augmentation
    • train/validation split
    • dataset size trade-offs
    • annotation guidelines
3 · RAG Architecture5 topics
  • RAG Fundamentals

    Retrieval-augmented generation pipeline, indexing, retrieval, generation, context window management, grounding, hallucination reduction, and naive RAG

    Objectives

    • Describe the end-to-end RAG pipeline and its three main phases
    • Explain how RAG reduces hallucinations by grounding responses in retrieved documents

    Key concepts

    • RAG pipeline
    • indexing phase
    • retrieval phase
    • generation phase
    • context window management
    • grounding
    • hallucination reduction
    • naive RAG architecture
  • Chunking Strategies

    Fixed-size chunking, recursive character splitting, semantic chunking, document-aware chunking, overlap strategies, chunk size optimization, and hierarchical chunking

    Objectives

    • Compare different chunking strategies and their impact on retrieval quality
    • Select appropriate chunk sizes and overlap for different document types

    Key concepts

    • fixed-size chunking
    • recursive character splitting
    • semantic chunking
    • document-aware chunking
    • chunk overlap
    • chunk size optimization
    • hierarchical chunking
    • parent-child chunks
  • Embeddings

    Text embedding models, sentence transformers, embedding dimensions, cosine similarity, fine-tuning embeddings, embedding benchmarks (MTEB), and hybrid search

    Objectives

    • Explain how text embeddings represent semantic meaning in vector space
    • Compare embedding models and evaluate them using benchmarks like MTEB

    Key concepts

    • text embedding models
    • sentence transformers
    • embedding dimensions
    • cosine similarity
    • fine-tuning embeddings
    • MTEB benchmark
    • dense vs sparse retrieval
    • hybrid search
  • Vector Databases

    Pinecone, Weaviate, ChromaDB, pgvector, FAISS, Milvus, indexing algorithms (HNSW, IVF), filtering, and scalability

    Objectives

    • Compare vector database solutions and their trade-offs
    • Understand approximate nearest neighbor algorithms like HNSW

    Key concepts

    • Pinecone
    • Weaviate
    • ChromaDB
    • pgvector
    • HNSW algorithm
    • IVF indexing
    • metadata filtering
    • scalability patterns
  • Retrieval Strategies

    Semantic search, keyword search, hybrid retrieval, re-ranking, multi-query retrieval, contextual compression, self-querying, and multi-hop retrieval

    Objectives

    • Design hybrid retrieval pipelines combining semantic and keyword search
    • Apply re-ranking and contextual compression to improve retrieval precision

    Key concepts

    • semantic search
    • BM25 keyword search
    • hybrid retrieval
    • re-ranking (cross-encoder)
    • multi-query retrieval
    • contextual compression
    • self-querying retriever
    • multi-hop retrieval
4 · Evaluation & Optimization4 topics
  • Evaluation Metrics

    BLEU, ROUGE, BERTScore, human evaluation, LLM-as-judge, RAG evaluation (context relevance, faithfulness, answer relevance), and A/B testing

    Objectives

    • Select appropriate evaluation metrics for different LLM tasks
    • Design evaluation frameworks combining automated metrics and human judgment

    Key concepts

    • BLEU score
    • ROUGE score
    • BERTScore
    • human evaluation
    • LLM-as-judge
    • RAG triad metrics
    • faithfulness
    • A/B testing
  • Quantization

    Post-training quantization (PTQ), quantization-aware training (QAT), GPTQ, AWQ, GGUF, INT8/INT4, calibration, and accuracy-performance trade-offs

    Objectives

    • Compare quantization methods and their impact on model quality
    • Apply appropriate quantization techniques for deployment scenarios

    Key concepts

    • post-training quantization (PTQ)
    • quantization-aware training (QAT)
    • GPTQ
    • AWQ
    • GGUF format
    • INT8/INT4 quantization
    • calibration datasets
    • accuracy-performance trade-offs
  • Knowledge Distillation

    Teacher-student training, logit matching, feature distillation, task-specific distillation, data distillation, and distillation loss functions

    Objectives

    • Design a knowledge distillation pipeline from a large teacher to a smaller student model
    • Explain the trade-offs between distillation approaches for different deployment constraints

    Key concepts

    • teacher-student framework
    • logit matching
    • feature distillation
    • task-specific distillation
    • data distillation
    • temperature scaling
    • distillation loss (KL divergence)
    • progressive distillation
  • Cost Optimization

    Token usage optimization, caching strategies, model routing, batch processing, prompt compression, tiered model selection, and TCO analysis

    Objectives

    • Design cost-efficient LLM deployment strategies with caching and model routing
    • Calculate and optimize total cost of ownership for LLM-based systems

    Key concepts

    • token usage optimization
    • semantic caching
    • model routing
    • batch processing
    • prompt compression
    • tiered model selection
    • total cost of ownership (TCO)
    • latency-cost trade-offs
5 · Deployment & Serving2 topics
  • Inference & Serving

    vLLM, Text Generation Inference (TGI), continuous batching, PagedAttention, speculative decoding, model parallelism, and API gateway patterns

    Objectives

    • Compare inference serving frameworks and their optimization techniques
    • Design scalable LLM serving architectures with appropriate batching strategies

    Key concepts

    • vLLM
    • Text Generation Inference (TGI)
    • continuous batching
    • PagedAttention
    • speculative decoding
    • tensor parallelism
    • pipeline parallelism
    • API gateway patterns
  • Guardrails & Safety

    Input/output filtering, content moderation, PII detection, prompt injection defense, constitutional AI, red teaming, and responsible AI practices

    Objectives

    • Design multi-layered guardrail systems for LLM applications
    • Implement prompt injection detection and content moderation pipelines

    Key concepts

    • input/output filtering
    • content moderation
    • PII detection and masking
    • prompt injection defense
    • constitutional AI
    • red teaming
    • RLHF for safety
    • responsible AI practices
6 · Advanced Patterns3 topics
  • Agent Frameworks

    LangChain, LlamaIndex, tool calling, function calling, ReAct pattern, planning agents, memory management, and multi-agent systems

    Objectives

    • Design agent architectures using LangChain or LlamaIndex with tool calling
    • Implement ReAct-based agents with appropriate memory and planning strategies

    Key concepts

    • LangChain
    • LlamaIndex
    • tool calling / function calling
    • ReAct pattern
    • planning agents
    • agent memory management
    • multi-agent systems
    • agent evaluation
  • Multimodal Models

    Vision-language models, image-text alignment, CLIP, multimodal embeddings, visual question answering, document understanding, and multimodal RAG

    Objectives

    • Explain how vision-language models align visual and textual representations
    • Design multimodal RAG pipelines for document understanding tasks

    Key concepts

    • vision-language models
    • CLIP architecture
    • image-text alignment
    • multimodal embeddings
    • visual question answering
    • document understanding
    • multimodal RAG
    • cross-modal attention
  • Advanced RAG Patterns

    Graph RAG, agentic RAG, RAPTOR, self-RAG, corrective RAG, adaptive retrieval, long-context RAG, and RAG pipeline orchestration

    Objectives

    • Compare advanced RAG patterns and identify appropriate use cases for each
    • Design sophisticated RAG architectures combining multiple retrieval and generation strategies

    Key concepts

    • graph RAG
    • agentic RAG
    • RAPTOR (recursive abstractive processing)
    • self-RAG
    • corrective RAG (CRAG)
    • adaptive retrieval
    • long-context RAG
    • RAG pipeline orchestration

Levels: Junior, Mid, Senior, and Expert

Explain it in your own words

Graded by AI · no account · 1 question

No options to guess from: you write, the AI grades against the topic's rubric and tells you what level your answer reads at.

0 / 600

Already studying LLM Fine-Tuning & RAG?

The assessment measures topic by topic, and your plan skips what you already know.

Measure my level

Open to read. An account is only needed to be measured.

Frequently asked questions

Where should I start studying LLM Fine-Tuning & RAG?

Start with LLM Fundamentals: Transformer Architecture, Tokenization, and Attention Mechanisms. The map orders topics by what they depend on, so each one prepares the next.

How long does a full LLM Fine-Tuning & RAG assessment take?

A full assessment of this path takes about 45 minutes. Study time depends on where you start; a diagnostic shows which topics you can skip.

Do I need an account to use this page?

No. The map, objectives and key concepts are open. You only need an account to be measured, get your gap by topic and receive a plan.