Prompt Engineering learning path: everything to study, topic by topic

Crafting effective prompts without examples, leveraging model knowledge through clear instructions, task framing, and constraint specification

Using examples to guide model behavior, selecting representative examples, formatting exemplars, and managing example ordering effects

Writing unambiguous instructions, decomposing complex tasks, using delimiters, specifying output length, and managing prompt structure

Controlling model output format including JSON, XML, tables, markdown, and structured data extraction through prompt design

Understanding temperature, top-p, top-k, frequency penalty, presence penalty, and their effects on output creativity vs determinism

Comparing major LLM providers (OpenAI, Anthropic, Google), understanding token economics, pricing tiers, context window sizes, and selecting the right model for different tasks

Fine-grained control over LLM output using stop sequences, repetition penalty, frequency penalty, max tokens, top_p vs temperature interaction, and logit bias

Eliciting step-by-step reasoning from models, zero-shot CoT, manual CoT, and understanding when reasoning chains improve accuracy

Sampling multiple reasoning paths and selecting the most consistent answer, majority voting, and confidence calibration through repetition

Assigning personas and roles to models, system prompts, persona design, and domain expert simulation for improved task performance

Enforcing JSON schemas, function calling for structured data, grammar-constrained generation, and Pydantic model integration

Decomposing complex tasks into sequential LLM calls, passing context between steps, and designing multi-step prompt pipelines

Advanced reasoning techniques beyond chain-of-thought: Tree of Thoughts exploration, ReAct pattern, step-back prompting, self-consistency, and plan-and-solve strategies

Combining multiple prompts for improved reliability: self-consistency voting, diverse verifiers, prompt portfolios, and multi-prompt aggregation strategies

Fixed-size chunking, semantic chunking, recursive splitting, document-aware chunking, and overlap strategies for retrieval-augmented generation

Text embedding models, vector similarity, embedding dimensions, model selection criteria, and fine-tuning embeddings for domain-specific retrieval

Combining keyword-based (BM25) and semantic search, reciprocal rank fusion, weighted scoring, and query routing for optimal retrieval

Cross-encoder reranking, LLM-based reranking, ColBERT, score calibration, and cost-quality trade-offs in retrieval pipelines

Managing limited context windows, context compression, lost-in-the-middle effects, citation tracking, and prompt budgeting for RAG systems

Measuring prompt quality with automated metrics, human evaluation, LLM-as-judge, BLEU, ROUGE, and domain-specific evaluation criteria

Designing controlled experiments for prompt variants, statistical significance, traffic splitting, and multi-armed bandit approaches for prompt optimization

Building prompt test suites, golden datasets, continuous evaluation, version comparison, and detecting prompt degradation over time

Designing domain-specific benchmarks, evaluating model capabilities, leaderboard construction, and comparing models for prompt-specific tasks

Adversarial testing of prompts, jailbreak detection, prompt injection attacks, boundary testing, and systematic vulnerability discovery

Identifying and mitigating biases in prompt-based systems through calibration techniques, balanced examples, perspective prompting, and systematic fairness testing

Using RAGAS and TruLens for systematic LLM evaluation: faithfulness, relevancy, context recall metrics, LLM-as-judge patterns, and building evaluation pipelines

Defining tool schemas, function calling APIs, parameter extraction, error handling in tool calls, and parallel function execution

ReAct pattern, plan-and-execute, tool-augmented generation, memory systems, and comparing agentic frameworks like LangGraph, CrewAI, and AutoGen

Task planning, self-reflection, self-critique, iterative refinement, and metacognitive prompting for improved agent performance

Agent collaboration patterns, debate and consensus, specialized agent roles, communication protocols, and orchestrating multi-agent workflows

Input/output validation, content filtering, safety classifiers, hallucination detection, and building robust guardrail pipelines for LLM systems

Building agent workflows with LangGraph's state machine model, conditional edges, human-in-the-loop patterns, persistence, and streaming for complex agentic systems

Multi-agent orchestration with CrewAI: defining agent roles, task delegation, crew hierarchies, agent tool integration, and collaboration patterns

Version control for prompts, prompt registries, rollback strategies, migration paths, and managing prompt evolution across deployments

Token usage optimization, caching strategies, model routing, prompt compression, and building cost-aware LLM pipelines

Streaming responses, parallel inference, speculative decoding, request batching, and optimizing time-to-first-token for production systems

LLM logging and tracing, prompt analytics, quality monitoring dashboards, alert systems, and debugging production prompt failures

Constitutional AI principles, RLHF concepts, responsible AI deployment, bias detection, content policies, and governance frameworks for LLM systems

Systematically optimizing few-shot example selection and chain-of-thought prompts for consistent performance at scale

Reliably generating structured outputs (JSON, XML) using constrained decoding, function calling, and validation strategies

Designing prompt architectures for agentic systems: ReAct, Tree of Thoughts, Plan-and-Solve, and self-reflection loops

Identifying and mitigating prompt injection attacks in LLM applications: input sanitization, privilege separation, and detection

Building systematic LLM evaluation pipelines: LLM-as-judge, human preference datasets, benchmark design, and regression testing