Appendix D — Going Deeper

A map, not a manual. For each module you outgrow, the grown-up neighbourhood to move into, and the idea from this book to carry through the door.

You will outgrow this package at a named moment: a budget it cannot express, a volume it cannot serve, an error it returns as a string where production needs a policy. Frameworks and services change quickly enough that this appendix names neighbourhoods and the current well-known residents, and trusts you to check who lives there when you arrive. The concepts are the durable part.

D.1 The Call Itself (Part I)

Your next step is the providers’ official SDKs, which add streaming, structured outputs, provider-native tool calling, and proper exceptions where this package returns error strings. If several providers must sit behind one interface in your own code, that is the job of routing layers such as LiteLLM, which generalise Chapter 4’s three variables into a managed surface. Carry with you: the payload is still the truth; read what any SDK actually sends before trusting its abstractions, as Listing 1.2 taught you.

D.2 Grounding (Part II)

Production retrieval adds vector stores, hybrid keyword-plus-vector search, metadata filtering, incremental reindexing, and evaluation sets for retrieval quality. LlamaIndex is the neighbourhood’s best-known framework address, with LangChain’s retrieval pieces and every vector database vendor nearby. Carry with you: it is still chunk, fetch, answer. The engineering is in chunking policy, freshness, and proving with scores and sampled answers that the fetch deserves the prompt built on it.

D.3 Action (Part III)

Agent frameworks such as Pydantic AI and the SDKs’ native agent tooling replace text-format parsing with validated function calling, add dependency injection for tools, retries with policy, and tracing as a first-class output. Carry with you: the model still only proposes. Your registry, your allowlists, and your worst-input discipline from Chapter 7 are the safety case, whatever the framework calls its decorators. Read Ship Python, Orchestrate AI in this series for this transition taken at professional length.

D.4 Control (Part IV)

Workflow engines and durable-execution platforms add scheduling, parallelism, persistence across crashes, and human-task queues where this book had folders and a person at a bench. Evaluation matures into dataset-driven evals: fixed case sets, regression gates in CI, judge ensembles with measured agreement. Carry with you: stage outputs you can inspect, goals stated before loops run, and moderation records for every judge. A platform that hides any of the three removes evidence you need. Typed judgments grow up into structured-output modes in the official SDKs, schema-constrained decoding in local servers such as Ollama (verify the current format behaviour before relying on it), and dedicated System One services such as TypeSafe’s Jev where calibrated probabilities are worth paying for.

D.5 The Senses (Part V)

Vision production work adds image pipelines, detection and OCR specialists beside the general VLM, and thinking about resolution and cost per image. Voice production is its own stack, and Pipecat is the front door the package’s final project points at: streaming transcription and synthesis, interruption handling, and the latency budget fought hop by hop. Carry with you: translators around a text core, and the consent and retention questions, which do not get simpler at scale.

D.6 The Model Itself

One door remains if the black box itself is what you want opened. Andrej Karpathy’s Zero to Hero series builds a small language model from first principles, at the cost of the mathematics Parts I to V promised to spare you: gradients, tensors, training loops. Its codebase line now runs from the deprecated nanoGPT to its successor nanochat, a full pretrain-to-chat pipeline kept deliberately hackable. Part VI opens this briefly, with toy training and a LoRA adapter; Karpathy covers it at full size. Nothing in Parts I to V requires either, and most builders of AI systems will never train a model.