Foundations
Linear algebra and engineering
The first hugely successful application of this predates "AI" as a marketing term entirely: PageRank, built by Google's founders, models the entire web as a graph and ranks every page by computing the dominant eigenvector of that graph's link matrix (via power iteration) — the same eigenvector/eigenvalue machinery from a linear algebra course, just pointed at a graph large enough to make web search actually work at scale.
The same linear algebra underneath neural networks underneath everyday engineering disciplines that never get billed as "AI": circuit design, physics engineering simulations, safety analysis, and risk-circuit modeling for ESG reporting. It's the same math wearing different clothes depending on which industry is asking the question.
Architectures
Neural network types
Five shapes worth actually knowing, not just naming — what each one is for, and the math underneath it in one or two sentences.
CNN — Convolutional Neural Networks
Use cases: image classification, object detection and segmentation, medical imaging — anything where the data sits on a spatial grid and nearby values are related to each other.
Math summary: a small learned kernel slides across the input computing a dot product at every position; reusing the same weights everywhere (weight sharing) is what gives translation invariance and far fewer parameters than a fully-connected layer over the same input. Stacking convolution, a nonlinearity, and pooling builds up from local edges to whole-object features layer by layer.
Does this use Fourier math across a 2D array? Yes and no, precisely. The Convolution Theorem — real, established math that predates neural networks entirely — says spatial-domain convolution is mathematically equivalent to multiplication in the frequency domain. In practice, most modern CNN layers don't route through Fourier space: the small 3×3/5×5 kernels that dominate today's architectures are usually computed directly (or via im2col plus matrix multiply, sometimes Winograd), because the FFT round-trip overhead isn't worth it at that size. But FFT-based convolution is real and implemented, not just theoretical — cuDNN exposes it as a selectable algorithm, and its own autotuner picks FFT over direct methods exactly when the kernel is large enough to make the frequency-domain route actually win. And independent of how it's computed: the filters a CNN's early layers actually learn are oriented, edge-detecting, frequency-selective patterns — the same territory classical Fourier-based image analysis covers either way.
RNN — Recurrent Neural Networks
Use cases: sequential and time-series data — historically language modeling, speech, anything where order carries meaning, before Transformers displaced it for most large-scale text work.
Math summary: one hidden state carried forward and updated at every step (ht = f(Wxxt + Whht−1 + b)), with the same weights reused at every timestep. Trained via backpropagation through time — which is exactly why vanilla RNNs struggle with vanishing or exploding gradients over long sequences.
FNN — Feedforward Neural Networks
Use cases: tabular data, simple classification and regression, and — less obviously — as a structural building block inside almost every other architecture, including the position-wise feedforward sublayer sitting inside every Transformer block.
Math summary: stacked layers of weighted sums plus a nonlinear activation, no recurrence, no convolution — every input connected to every output by a learned weight. The Universal Approximation Theorem is the formal reason a large-enough single hidden layer can approximate any continuous function, even though in practice depth is what actually makes training tractable.
LSTM — Long Short-Term Memory
Use cases: was the default for sequence modeling — translation, speech recognition, longer-form language modeling — before Transformers took over around 2017; still shows up where a Transformer's full attention is more compute than the problem needs.
Math summary: an RNN variant with a separate cell state plus three learned gates (input, forget, output) that decide what gets written into, kept in, or read out of that cell state at each step. That gating is specifically what fixes vanilla RNNs' vanishing-gradient problem over long sequences.
Transformers
Use cases: the dominant architecture today — every model already covered on this page (Gemma 4, gpt-oss, Nemotron 3) is a transformer or transformer-hybrid, plus vision transformers and multimodal models.
Math summary: self-attention — every token computes a weighted combination of every other token's value vector, with the weights coming from a scaled query-key dot product run through softmax. No recurrence at all, so the whole sequence processes in parallel instead of one step at a time, which is the real reason it displaced RNNs and LSTMs at scale. Position has to be added back in explicitly (positional encoding), since attention alone has no built-in sense of order.
Also worth knowing
Adversarial examples — inputs deliberately engineered, often via small, human-imperceptible perturbations, to make a trained model mispredict, exploiting the gap between how a model and a human actually read the same input.
YOLO-style detectors — built for real-time object detection in a single pass over an image.
One way to think about it: strip away the layers and a network is just weights and biases arranged to approximate some enormously complex function — not unlike what a differential equation does for a physical system, just fit by gradient descent instead of solved in closed form. That's more than a loose resemblance: an entire subfield (Neural ODEs, and physics-informed neural networks more specifically) treats deep networks as literal, continuous-time differential-equation solvers rather than something that merely looks like one.
Further reading: Neural Ordinary Differential Equations (Chen et al., 2018).
LLM Architecture Evolution LLMs Are Linear Algebra in Motion
Simulation
Simulation
Genetic algorithms, branch prediction, and hyperparameter optimization across a grid space all live at the same intersection: linear algebra meeting differential equations. "Hyperparameter optimization in a grid space" sounds abstract until you picture what it actually is — systematically trying combinations of settings across a defined space to find the one that performs best, the same basic idea as branch prediction guessing which path is worth taking before you've actually walked it.
08
Go deeper: three worked examples
Genetic algorithms. Maintain a population of candidate solutions, score each with a fitness function, keep the best performers, and recombine/mutate them into the next generation — repeat until the population converges on something good. NASA's ST5 mission (2006) used one to evolve a spacecraft antenna: instead of an engineer hand-shaping the geometry, the algorithm bred thousands of candidate wire shapes against a simulated radiation pattern, and the winning antenna looked like nothing a human would have designed — bent and asymmetric — but outperformed the conventional design it replaced.
Branch prediction. A CPU speculatively
executes past an if before it knows which way the
branch will actually go, and being wrong means throwing away
that work. Modern high-end predictors (AMD Zen and recent
Intel cores among them) do this with a
perceptron
— literally a small neural network taking a long history
of recent branch outcomes as input, weighting them, and
predicting taken/not-taken from the weighted sum
(Jiménez & Lin, 2001). The "AI" framing isn't a
stretch here: it's a real, tiny neural net running billions of
times a second in hardware you already own.
Hyperparameter optimization. Grid search (try every combination on a fixed grid) is the simplest approach and also provably wasteful in high dimensions: Bergstra & Bengio (2012) showed random search finds better configurations faster once you have more than a couple of hyperparameters, because a grid wastes trials on unimportant dimensions. Modern tuning tools (Optuna, for one) go further with Bayesian methods — using the results collected so far to model which unexplored region of the space is likely to be good, instead of sampling blindly.
The open-source AI tooling landscape
An outline of what's actually out there right now, organized by where each piece sits — models, how you watch what they're doing, what orchestrates them into agents, and what you'd actually run them on at home. Scored on real specs, not marketing tiers; a deeper comparison post for each category is a natural next step once this outline is solid.
One caveat from actually running these locally: rated context length isn't the same as usable context length. Past roughly 64K tokens, I've seen inference throughput drop off noticeably on the hardware above — the 128K–256K numbers above are real ceilings, not practical ones.
Home Lab Tests
A real result from running two GPUs from the home lab above side by side, not a synthetic benchmark: same script, same model, same config, temperature pinned to zero on both cards — and the RTX 5070 Ti and RTX 5060 Ti still diverged. Not just on speed, either. Each card was internally deterministic (the same result on every repeat, on that card), but the two cards didn't agree with each other.
vrun.sh analyze), same model
(google/gemma-4-E4B-it, fp8 weights, fp8 KV
cache), same 136K-token input — a call-chain
extraction over a Tetris game I built, with bugs
deliberately planted in it to see whether the model could
find them. The RTX 5070 Ti surfaced 2 bugs; the RTX 5060 Ti
surfaced 3, and only one (the engine.tick
memory leak) appears in both reports.
| GPU | Elapsed | Throughput | Bugs found |
|---|---|---|---|
| RTX 5070 Ti 16GB | 42.3s | 3,232.9 tok/s | 2 |
| RTX 5060 Ti 16GB | 73.9s | 1,847.8 tok/s | 3 |
The ~1.75x throughput gap tracks the ~2x memory-bandwidth gap between these two cards (896 GB/s vs. 448 GB/s — see the Home-lab GPU comparison table further down), which is the boring, expected part. The actually surprising part is that the two cards didn't converge on the same bug report at temperature zero. My best guess, not a confirmed root cause: fp8 is a coarser number format than bf16/fp16, and different GPU architectures can legitimately sum and round the exact same floating-point math in a different order. Worth internalizing either way — "temperature zero" only guarantees a deterministic result on one card, not agreement across different GPU hardware running the same model.
watch -n1 nvidia-smi left running throughout,
not a one-off check, still showing each card's model
weights resident in VRAM.
Models
Open-weight models worth knowing, roughly ordered by how much VRAM they need.
Gemma 4 (E2B / E4B, 12B, 26B MoE, 31B Dense)
Company: Google
| Model | Max context | Min Q4 VRAM | Min FP8 VRAM | Release date |
|---|---|---|---|---|
| Gemma 4 E2B | 128K | ≈ 2.6 GB | ≈ 5.1 GB | April 2026 |
| Gemma 4 E4B | 128K | ≈ 4.0 GB | ≈ 8.0 GB | April 2026 |
| Gemma 4 12B Unified | 256K | ≈ 6.0 GB | ≈ 12.0 GB | April 2026 |
| Gemma 4 26B A4B (MoE) | 256K | ≈ 12.6 GB | ≈ 25.2 GB | April 2026 |
| Gemma 4 31B Dense | 256K | ≈ 15.4 GB | ≈ 30.7 GB | April 2026 |
E2B/E4B totals include Per-Layer Embeddings (PLE) — a runtime that supports offloading PLE to host memory can run below this floor; the numbers above assume it doesn't.
- Architecture: this is where a real 31B Gemma actually lives — Gemma 3's largest is 27B, not 31B, so if you've seen "31B Gemma" mentioned, it's this generation. The 26B model is a sparse MoE (25.2B total, 3.8B active); the rest are dense.
- License: plain Apache 2.0, dropping the more restrictive Gemma Terms of Use.
NVIDIA Nemotron 3 (Nano 4B / Nano 30B-A3B / Super / Ultra)
Company: NVIDIA
| Model | Max context | Min Q4 VRAM | Min FP8 VRAM | Release date |
|---|---|---|---|---|
| Nemotron 3 Nano 4B | 262K | ≈ 2.0 GB | ≈ 4.0 GB | March 2026 |
| Nemotron 3 Nano 30B-A3B | 1M (often served at 256K) | ≈ 15.0 GB | ≈ 30.0 GB | December 2025 |
| Nemotron 3 Super | not stated | ≈ 60 GB | ≈ 120 GB | H1 2026 |
| Nemotron 3 Ultra | not stated | ≈ 275 GB | ≈ 550 GB | H1 2026 |
Super/Ultra's max context isn't stated in NVIDIA's own Nemotron 3 Nano report (arXiv:2512.20848), which covers the 30B-A3B Nano only — marked "not stated" rather than assumed to match Nano's. The Nano 4B row is sourced from its Hugging Face model cards, not that paper.
- Architecture: "Nano" actually spans two different designs, not one size of one design. Nano 30B-A3B is the hybrid Mamba-2 + Transformer + Mixture-of-Experts model — 23 Mamba-2 layers, 23 MoE layers (128 routed experts + 1 shared, 6 active per token), and 6 GQA attention layers, ~3B active parameters per token. Nano 4B is a separate, smaller, non-MoE model — mostly Mamba-2 and MLP layers plus just 4 attention layers — compressed from the previous-generation Nemotron Nano 2 9B using NVIDIA's "Nemotron Elastic" framework, aimed at edge hardware like Jetson Thor and GeForce RTX.
- License: open-weight but, like Gemma, not OSI-approved — shipped under NVIDIA's own "Nemotron Open Model License," then migrated in mid-2026 to the Linux Foundation's new OpenMDW-1.1 framework, submitted for OSI review but not yet OSI-approved.
- Worth flagging: NVIDIA reused this name once before — an unrelated, much smaller "Nemotron-3-8B" shipped back in November 2023, and has nothing to do with this 2025–2026 family.
gpt-oss-20b / gpt-oss-120b
Company: OpenAI
| Model | Max context | Min Q4 VRAM | Min FP8 VRAM | Release date |
|---|---|---|---|---|
| gpt-oss-20b | 128K | ≈ 10.5 GB | ≈ 21.0 GB | August 2025 |
| gpt-oss-120b | 128K | ≈ 58.5 GB | ≈ 117.0 GB | August 2025 |
- Architecture: mixture-of-experts — 20b runs 3.6B active parameters per token, 120b runs 5.1B active, both shipping in native MXFP4 quantization, with adjustable reasoning effort and built-in tool use.
- License: Apache 2.0.
Hugging Face
-
What it is: not a model, but the hub almost
everything above gets downloaded from, plus the libraries
that actually run it:
transformers,diffusers,accelerate, andhuggingface_hub(all Apache 2.0). -
Worth knowing:
diffusersspecifically is the standard library for running image-generation models like SDXL and Flux locally — it's hardware-agnostic by design, running on whatever PyTorch device is available, which is exactly what makes the CUDA/ROCm/XPU comparison below possible to make apples-to-apples.
| Criterion | gpt-oss-20b | Gemma 4 E2B | Gemma 4 E4B | Gemma 4 12B | Gemma 4 26B MoE | Nemotron 3 Nano 30B-A3B |
|---|---|---|---|---|---|---|
| Context window | 128K | 128K | 128K | 256K | 256K | 1M (often served at 256K) |
| Intelligence | MMLU 80.4 / 84.0 / 85.3 (low/med/high effort) | MMLU-Pro 60.0 | MMLU-Pro 69.4 | MMLU-Pro 77.2 | MMLU-Pro 82.6 | MMLU-Pro 78.3 |
| Code evals | Codeforces 1366 / 1998 / 2230 Elo (no tools) | LiveCodeBench 44.0% · Codeforces 633 Elo | LiveCodeBench 52.0% · Codeforces 940 Elo | LiveCodeBench 72.0% · Codeforces 1659 Elo | LiveCodeBench 77.1% · Codeforces 1718 Elo | LiveCodeBench 68.3% · SWE-Bench 38.8% |
| Reasoning evals | AIME 2024 42.1 / 80.0 / 92.1% (no tools) · GPQA 56.8–74.2% | AIME 2026 37.5% · GPQA 43.4% | AIME 2026 42.5% · GPQA 58.6% | AIME 2026 77.5% · GPQA 78.8% | AIME 2026 88.3% · GPQA 82.3% | AIME25 89.1 / 99.2% (no tools/with tools) · GPQA 73.0 / 75.0% |
| Tool use / function calling | Documented: browsing, Python execution, custom functions, at every reasoning-effort tier | Native function calling claimed at the family level; Google's model card doesn't break out a per-variant tool-use score | Documented: BFCL v4 53.76% | |||
| Local deployment | Native MXFP4-quantized weights | Official QAT/GGUF/MLX builds across the family | BF16 weights on Hugging Face; GGUF/quantized build availability not confirmed | |||
Security
Safetensors: not getting owned by a download
PyTorch's traditional .bin/.pt weight
files are Python pickle archives — and
loading a pickle file can execute arbitrary Python code as a
side effect of deserialization. Downloading a stranger's model
weights in that format means trusting them not to have hidden a
payload in it. safetensors (Hugging Face's format)
fixes this structurally rather than by scanning for bad actors:
it's just a JSON header (tensor names, shapes, dtypes, byte
offsets) followed by a flat block of raw tensor bytes —
there's no code path in the file at all, nothing to execute. An
independent security audit (Trail of Bits, commissioned by
Hugging Face, EleutherAI, and Stability AI, May 2023) found no
critical flaw, and transformers now saves in
safetensors
by default.
Hugging Face Hub does run automated pickle scanning on uploaded
.bin/.pt files and shows a scan-status
badge, but that's a blacklist heuristic with documented bypasses
— it catches known-bad patterns, it doesn't guarantee
safety. The actually reliable move: check a repo's Files tab
before downloading, and prefer ones that ship only
.safetensors — no .bin/.pt
fallback sitting there as an unnecessary attack surface.
-
Gemma 4
— Apache 2.0, ships as
safetensorsonly. Up to 256K context on the 31B Dense and 26B MoE variants (see Models above). Google's own release isn't pre-quantized; for a Q4safetensorsbuild (no GGUF conversion needed), Unsloth ships community bitsandbytes-4bit quantizations — E2B, E4B, and 31B Dense — not Google-published, so verify the repo's Files tab yourself rather than taking that on faith. -
gpt-oss-20b
— Apache 2.0, ships as
safetensorsonly. 128K context, with adjustable reasoning effort built in. The native MXFP4 MoE weights in that repo are already an effectively-4-bit format; a separate community Q4safetensorsbuild (plain bitsandbytes-4bit, applied to the whole model rather than just the MoE layer) is also available from Unsloth: gpt-oss-20b-bnb-4bit.
Branch prediction, applied to reasoning
The Simulation section above covers CPU branch predictors: guessing which way a conditional will go before it's actually resolved, so speculative work isn't wasted. Nothing shipped in 2026 literally brands itself "branch prediction for LLMs" — that framing turns out to be an informal analogy, not a real product name — but the underlying idea (predict which path is worth taking before fully computing it) shows up in real, current reasoning- acceleration research once you go looking for it.
01
Go deeper: what actually exists in 2026
Tree-of-Thought pruning is the closest conceptual match: a value/evaluator model scores partial reasoning branches (roughly "promising," "maybe," "dead end") and decides which are worth expanding before fully computing them — the same shape as a branch predictor deciding which path to speculate down. 2026 work here includes OS-Pruner (pruning reasoning chains via optimal-stopping theory) and ToTRL.
Reasoning-level speculative decoding is newer and aimed specifically at reasoning speed: SpecReason has a small draft model propose reasoning steps that a larger model verifies, reporting both a 1.5–2.5x speedup and a accuracy gain over reasoning without it. SSR, ConfSpec (confidence-gated step-level speculation), and "Lookahead Reasoning" are adjacent 2026 papers doing variations on the same draft-then-verify idea, one reasoning step at a time instead of one token at a time.
Token-level speculative decoding is the most mature and widely deployed of the three: EAGLE-3 (a draft head that fuses features across every transformer layer) merged into vLLM, SGLang, and TensorRT-LLM in early 2026, reporting 3–4x throughput gains. This is closer to raw generation speed than reasoning quality, but it's the same predict-then-verify shape at the smallest possible grain: one token at a time.
Worth naming the near-miss too: a paper that initially looked like a direct "branch prediction for LLMs" hit (arXiv:2512.21323, "Parallel Token Prediction") turned out, on actually reading the abstract, to describe a parallel decoding scheme in terms of determinism and throughput, with no branch-prediction framing at all — a reminder that a title matching a search doesn't mean the paper says what you expect.
Token anatomy: what a context window actually holds
A token isn't a fixed real-world unit, but a rough conversion is knowable — and once you have it, a nominal 128K or 256K context window turns out to be a very different size in practice than the number alone suggests.
02
Go deeper: bytes, words, pages, and how much of that window
you actually need (quick 3-question check first)
Why a token needs 4 bytes now. A token ID is
just an index into the vocabulary, and a 2-byte unsigned
integer (uint16) can only address 65,536 distinct
values. Current vocabularies blow past that: Gemma 3 and Gemma
4 both use roughly 262K entries, OpenAI's gpt-oss uses exactly
201,088, and Llama 3 uses 128,256 — every one of them
more than double the uint16 ceiling. That's the
real, structural reason token IDs are stored as 4-byte
(uint32) integers today: the vocabulary itself
outgrew the smaller type.
Converting tokens to something human-sized. OpenAI's own documented rule of thumb is roughly 4 characters or 0.75 words per token for English prose. Pair that with a commonly-cited (though not formally standardized) convention of about 500 words per single-spaced page, and a context window converts into pages a reader could actually hold. Lines of code are much fuzzier — there's no single authoritative source, estimates run anywhere from about 10 to 15 tokens per line depending on language and style, so treat the code column below as a rough, order-of-magnitude estimate, not a precise conversion.
| Context | ~ Words | ~ Pages | ~ Lines of code |
|---|---|---|---|
| 32K tokens | 24,000 | ~48 | ~3,200 |
| 64K tokens | 48,000 | ~96 | ~6,400 |
| 96K tokens | 72,000 | ~144 | ~9,600 |
| 128K tokens | 96,000 | ~192 | ~12,800 |
| 256K tokens | 192,000 | ~384 | ~25,600 |
So: how large does your context window truly need to be when you're searching for one answer? Almost certainly nowhere near 384 pages' worth. Finding a single fact doesn't require holding the whole haystack in memory at once — it requires the ability to look, find, and forget everything except the answer and a trail back to it. That reframes a big context window as a budget to spend deliberately, not a warehouse to fill: work one step at a time, compact what you found into a short summary the moment that step finishes, and carry only the compacted summary — not the raw material that produced it — into the next step. Repeat, and no single step ever needs more than a fraction of the nominal window, no matter how many steps the overall task takes. It's also, not coincidentally, close to how a long agentic coding session survives running past its own context limits in the first place.
Observability
Instrumenting what an agent actually did, not just what it returned.
03
Go deeper: what's actually available today
- OpenTelemetry — not a tool with a UI, but the vendor-neutral standard (APIs, SDKs, and a collector) for traces, metrics, and logs underneath both entries below. A CNCF graduated project (the merger of OpenTracing and OpenCensus), Apache 2.0. Knowing this exists is what makes "OpenTelemetry-based instrumentation" in the next bullet a meaningful claim rather than jargon.
- Arize Phoenix — Arize AI's open-source tracing, evals, and embedding-drift tool: OpenTelemetry-based instrumentation, LLM-graded response/retrieval evals, and UMAP-plus-clustering drift analysis for RAG pipelines. The main server is licensed Elastic License 2.0, not Apache — free to self-host and modify, but not OSI-approved, and it can't be resold as a hosted service. A few sub-packages (the client, evals, and OpenTelemetry libraries) are separately Apache 2.0. Distinct from Arize's paid "Arize AX" enterprise platform, which Phoenix isn't a stripped-down version of.
-
Pydantic Logfire — the Pydantic
team's observability platform, also built on OpenTelemetry,
with automatic instrumentation for Pydantic models, FastAPI,
Django, SQLAlchemy, and OpenAI calls specifically. Licensing
is split and worth getting right: the
logfireSDK itself is MIT and exports OTel-compatible data to any backend, not just Logfire's own; the hosted platform (the actual UI and backend you'd look at) is closed-source, freemium, and only self-hostable under a paid enterprise license — don't confuse "the client is open source" with "the product is open source." - Grafana (or a custom UI) — OTel data still has to go somewhere and get looked at, and Phoenix/Logfire aren't the only options. Grafana itself has no ingestion of its own; it's the visualization layer on top of Tempo (traces, OTLP-native), Mimir (metrics), and Loki (logs) — all three, plus Grafana core, were relicensed from Apache 2.0 to AGPLv3 in 2021 and still are today. The "edges" (plugins, agents like the Grafana Alloy OTel Collector distribution, client libraries) stay Apache 2.0. Because OpenTelemetry is a vendor-neutral standard, a fully custom UI querying that same data directly is just as valid a choice — Grafana is the common one, not the only one.
Agent ecosystems
Orchestration frameworks and agent products — not the models underneath them.
04
Go deeper: nine orchestration frameworks and agent products
- LangGraph — the LangChain team, MIT license. Supports cyclic graphs (not just one-way chains), which is what agentic control flow — retry, reflect, re-plan — actually needs. Built-in checkpointing/persistence (in-memory, SQLite, or Postgres savers) for durable, resumable execution, plus human-in-the-loop pause/inspect/modify and LangSmith tracing integration.
- Langflow — MIT license. A visual, drag-and-drop builder for LangChain-based agent and RAG flows; a flow exports as a REST API, an MCP server, or plain JSON. Acquired by DataStax and then IBM, but development stays open source.
- Hermes — Nous Research ships two distinct things under this name. The Hermes model family (Hermes 3, Hermes 4) is a line of fine-tuned LLMs with a native tool-calling format built in, not an orchestration framework. Separately, Hermes Agent is a standalone, MIT-licensed, self-improving CLI/TUI agent — skill creation, subagent delegation, scheduled automations, and gateways into Telegram/Discord/Slack/WhatsApp/Signal, model-agnostic via any OpenAI-compatible provider.
-
agentic-api
(
vllm-project/agentic-api) — Apache 2.0, Rust, and very new (created March 2026). A gateway that adds server-side conversation state and tool-call-loop orchestration on top of vLLM, exposing an OpenAI-compatible Responses/Messages API — a separate, early-stage companion project, not part of core vLLM. -
ADK (
google/adk-python) — Apache 2.0. Google's code-first Python Agent Development Kit: optimized for Gemini but explicitly model- and deployment-agnostic, deploying via Cloud Run or Vertex AI Agent Engine. -
Pydantic AI
(
pydantic/pydantic-ai) — MIT, from the Pydantic team. A type-safe agent framework built directly on Pydantic's own validation/schema system, aimed at production apps that need structured, validated output regardless of which LLM provider is behind it. Past 1.0 (shipped September 2025) and now on a stable 2.0 line (June 2026). -
OpenAI Agents SDK
(
openai/openai-agents-python) — MIT, OpenAI's own official successor to the earlier experimentalswarmproject. Deliberately lightweight: built around native OpenAI tool/function calling and structured JSON-schema output rather than a heavy orchestration abstraction, with an April 2026 update adding sandboxing, longer-horizon execution, and subagents. -
CrewAI (
crewAIInc/crewAI) — MIT, now backed by CrewAI Inc. rather than a purely community project. Role-based multi-agent orchestration: a "crew" of agents with defined roles and goals hand tasks off to each other. No longer depends on LangChain — a standalone architecture it moved to deliberately, for a lighter footprint. -
Smolagents
(
huggingface/smolagents) — Apache 2.0, Hugging Face. Its core idea (theCodeAgent) is agents that write and execute actual Python code as their action mechanism, instead of emitting JSON tool-calls — Hugging Face's argument is that real code (loops, conditionals, nested calls) is more expressive than a flat call format. Deliberately minimal: the core logic is roughly 1,000 lines.
| Framework | Released | Popularity | Learning curve | Reliability | Primary focus | Best used for | Output style |
|---|---|---|---|---|---|---|---|
| Pydantic AI | Oct 2024 | ★★ 2 out of 5 | Average | ★★★ 3 out of 5 | Type safety & developer speed | Production-grade structured-data apps | Strict Python type hints |
| LangGraph | Jan 2024 | ★★★★ 4 out of 5 | Hard | ★★★★ 4 out of 5 | Stateful agent logic loops | Complex multi-agent cyclical workflows | State graph channels |
| OpenAI Agents SDK | Mar 2025 | ★★★ 3 out of 5 | Easy | ★★★ 3 out of 5 | OpenAI-first simplicity | Rapid prototyping with OpenAI tools | Native JSON schema / function calls |
| CrewAI | Nov 2023 | ★★★★★ 5 out of 5 | Easy | ★★★★ 4 out of 5 | Role-based collaboration | Simulating human teams and workflows | Task-based handoffs |
| Smolagents | Dec 2024 | ★★★ 3 out of 5 | Average | ★★★ 3 out of 5 | Code-execution agents | Local, low-overhead secure execution | Raw Python code execution |
Release dates are each project's first public release (verified against GitHub tags/PyPI), and Popularity is these five projects' current GitHub star counts translated into a relative 1–5 scale — CrewAI (58K) and LangGraph (41K) lead, Pydantic AI (20K) trails, with OpenAI Agents SDK and Smolagents (29K each) in between. Learning curve (Easy/Average/Hard) and Reliability (stars) are comparative editorial judgments, not benchmark measurements — nobody ran a controlled study across all five.
Model serving
The engine that actually runs a downloaded model — distinct from the models themselves (above) and the driver/kernel layer that engine sits on top of (below).
05
Go deeper: eight serving engines, and why they're not
interchangeable
-
SGLang (
sgl-project/sglang) — Apache 2.0, maintained by LMSYS with contributions from xAI, NVIDIA, AMD, Intel, and LinkedIn. Its core technique is RadixAttention, a prefix-caching scheme the project credits with a 5x inference speedup. Since November 2025, SGLang Diffusion extends that same scheduler/kernel stack to image and video generation (Wan, Hunyuan, Qwen-Image, Flux), claiming 1.2–5.9x speedups — genuinely not just a text-LLM server anymore, though the diffusion piece is a newer module layered on top of the core. -
vLLM (
vllm-project/vllm) — Apache 2.0, born at UC Berkeley's Sky Computing Lab and now a large multi-company/academic project (2,000+ contributors). Core technique is PagedAttention (paged KV-cache memory management) plus continuous batching. Widely regarded as the standard for high-throughput, many-concurrent-user serving in datacenters — that's an industry consensus, worth being clear, not a claim the project's own docs make in those words. -
Intel llm-scaler
(
intel/llm-scaler) — Apache 2.0. Intel's own extended fork of vLLM for its multi-GPU platforms (Arc B60/B70, A770): Intel-specific kernels, FP8/INT4 quantization, multi-node deployment, LoRA support. Actively developed, but Intel's own docs say its Docker images are "intended for demo purposes only, not intended for production use" — a real maturity caveat, consistent with the Kernel Landscape section's read on XPU as the roughest of the three backends. -
Lemonade (
lemonade-sdk) — Apache 2.0, AMD-sponsored, targeting Ryzen AI NPUs plus Radeon integrated/discrete GPUs, OpenAI-API-compatible. One caveat: the underlying FastFlowLM NPU kernels are proprietary (free for "reasonable commercial use" per AMD), so the full stack isn't 100% open source end to end, even though Lemonade itself is. -
llama.cpp (
ggml-org/llama.cpp) — MIT, created by Georgi Gerganov (also the creator of the underlyingggmltensor library), 700+ contributors, 100K+ GitHub stars. The base engine Ollama and LM Studio are built on — Ollama's own docs credit the llama.cpp project directly. The ggml/llama.cpp team joined Hugging Face in February 2026. -
Candle (
huggingface/candle) — dual Apache 2.0/MIT, Hugging Face's own minimalist Rust-native inference framework: no Python or GIL overhead, with CUDA and WASM support — built for lightweight, serverless-friendly binaries rather than a full training stack. -
Ollama (
ollama/ollama) — MIT, built on top of llama.cpp/ggml. The dominant one-command local-serving tool — the reason most people's first local model ever ran was a singleollama runaway. -
TensorRT-LLM
(
NVIDIA/TensorRT-LLM) — Apache 2.0, NVIDIA-maintained. Compiles a model through TensorRT with custom attention/GEMM/MoE kernels and runtime techniques (disaggregated prefill-decode, speculative decoding) for NVIDIA-GPU-optimized inference — the CUDA-native counterpart to vLLM's more hardware-agnostic approach.
Worth naming what didn't make this list on purpose: Hugging Face's own TGI (Text Generation Inference) was a serious contender for years but is now in maintenance mode (bug-fixes only) as of this research pass — left off rather than listed as if it were still an actively-developed option.
Model serving APIs
Everything in the section above runs a model you downloaded. These are the opposite: pay-per-call (or flat-subscription) APIs where someone else runs the model, and you never touch the weights or the hardware at all.
06
Go deeper: five hosted providers, and how their pricing
actually works
- Fireworks AI — a hosted inference platform for open-weight models (Qwen, DeepSeek, Kimi, and others), not a foundation-model lab itself. Pay-per-token, tiered by model size (roughly $0.10/1M tokens under 4B, $0.20/1M for 4B–16B, $0.90/1M for 16B+), plus GPU-hour billing for dedicated deployments and per-token billing for fine-tuning (LoRA and full SFT/DPO). Its differentiator is FireAttention, a custom CUDA attention kernel built specifically to beat stock vLLM performance on the same hardware.
- OpenAI API — pay-per-token. Honest gap in this research: sources disagreed on the current flagship model, with some pointing to a "GPT-5.6" generation and OpenAI's own pricing page also listing a higher-tier "gpt-6-astra" — not confident enough to state a single current flagship as fact, so treat "what's newest" as unsettled rather than picking one. Notable API features: the Responses API and built-in tools.
- Anthropic API — pay-per-token, current lineup Claude Opus 5, Sonnet 5, Haiku 4.5, and Fable 5. Notable features: prompt caching, extended/adaptive thinking, computer use, and long context on current models.
- Featherless AI — the pricing-model outlier here: a flat-rate subscription, not pay-per-token (chat plans roughly $10–$25/month, developer/credit plans from $50/month), for access to a very large catalog of open-weight and community fine-tuned models — catalog-size estimates vary by source (tens of thousands of models), so treat any exact number as approximate rather than precise.
- Google Gemini API — pay-per-token, tiered by context length. Same honest gap as OpenAI above: some sources cite Gemini 3.1 Pro as flagship, while Google's own pricing page foregrounds Gemini 3.8 Flash as "most intelligent" — naming that suggests the Flash line may have outpaced Pro, but not confirmed with confidence. Notable features: native multimodality, context up to 1M tokens, and grounding with Google Search.
The two flagged naming gaps above (OpenAI, Gemini) are real research limitations, not hedging for its own sake — model-generation names in this space change fast enough that a confident wrong answer is worse than an honest "unsettled."
Home lab
Hardware landscape: my home lab
This isn't a shopping list — it's what's actually sitting in my home lab right now, spanning all three GPU vendors: NVIDIA, AMD, and Intel.
GPUs
- NVIDIA RTX 5070 Ti — 16GB GDDR7, 896 GB/s, Blackwell. Comfortably fits roughly 13B-class models at Q8, and 20–24B at Q4 with moderate context.
- NVIDIA RTX 5060 Ti — the 16GB variant (it also ships in 8GB, on the same die). Only 448 GB/s of bandwidth, half the 5070 Ti's — that ceiling bottlenecks generation speed against the 5070 Ti even at equal VRAM, which is exactly the kind of gap that's easy to miss without running both side by side.
- AMD Radeon AI PRO R9700 — 32GB GDDR6, RDNA4, 640 GB/s, ROCm 6.4.2/7.0. The 32GB is the real story here — fits 30B-class models at higher precision or 70B at aggressive quantization, more headroom than either NVIDIA card above.
- Intel Arc Pro B70 — 32GB GDDR6, ~608 GB/s, "Big Battlemage" (BMG-G31, a larger die than the B50/B60's BMG-G21). Announced March 2026 as Intel's new top-tier Arc Pro card, above the existing B50 (16GB) and B60 (24GB, also available as a dual-GPU 48GB workstation board). Same VRAM class as the R9700 above, so a similar rough fit: 30B-class models at higher precision or 70B at aggressive quantization.
| GPU | VRAM | Bandwidth | Architecture | Fits (roughly) |
|---|---|---|---|---|
| RTX 5070 Ti | 16GB GDDR7 | 896 GB/s | Blackwell | 13B @ Q8 · 20–24B @ Q4 |
| RTX 5060 Ti | 16GB GDDR7 | 448 GB/s | Blackwell | same VRAM as 5070 Ti, half the bandwidth |
| Radeon AI PRO R9700 | 32GB GDDR6 | 640 GB/s | RDNA4 | 30B @ higher precision · 70B aggressive quant |
| Arc Pro B70 | 32GB GDDR6 | ~608 GB/s | Big Battlemage | 30B @ higher precision · 70B aggressive quant |
Bandwidth and VRAM drive fit more than raw compute at this scale — the R9700 and B70 tie for the most VRAM at 32GB, and the 5060 Ti's half-bandwidth ceiling versus the 5070 Ti is the kind of gap that's easy to miss without running both side by side.
Mini PCs
- Minisforum MS-02 Ultra — ships up to a Core Ultra 9 285HX, but mine runs the Core Ultra 5 235HX configuration, which makes more sense for my actual use case. Up to 256GB of ECC DDR5, and critically a real PCIe 5.0 x16 slot — in my setup bifurcated to PCIe 4.0 x4/x4/x4/x4, so it runs multi-GPU configurations similar to the desktop above instead of just fitting one full discrete GPU. One of the most impressive PCs I've used — closer to a compact workstation than a typical mini PC.
- GMKtec NucBox K16 — AMD Ryzen 7 7735HS, 32GB LPDDR5X soldered (fixed, not upgradeable), a weak integrated Radeon 680M — an OCuLink port is the only real path to GPU-backed inference.
- GMKtec NucBox K11 — AMD Ryzen 9 8945HS, 32GB DDR5 SODIMM (upgradeable to 96GB, unlike the K16), Radeon 780M, also OCuLink-equipped.
Kernel landscape: CUDA vs. ROCm vs. XPU, for image generation
Same three vendors as the hardware above, but the question here is
software maturity, not silicon — specifically, how reliably
each backend actually runs SDXL and Flux through
diffusers today.
07
Go deeper: the CUDA vs. ROCm vs. XPU breakdown
Years of production PyTorch support, 2015–2026. Solid fill = official, broadly-relied-on support; dashed outline = present but young.
-
NVIDIA CUDA — the default path, and
it isn't close. CUDA became the de facto ML backend through
the mid-2010s (TensorFlow and PyTorch both built around it
first, cuDNN arrived in 2015), and PyTorch 2.7 (April 2025)
added official Blackwell support via CUDA 12.8. Nearly every
diffuserstutorial, doc, and community troubleshooting thread assumes CUDA — it's the backend everything else gets compared against. -
AMD ROCm — scoped to the two RDNA
generations in the lab above:
- R9700 (RDNA4) — officially supported (gfx1201). 32GB is "ECC-capable," likely driver-level in-band ECC rather than true hardware ECC — unconfirmed against AMD's own spec sheet. Works but young: vendor guides disagree on which ROCm version added it (6.4.2 vs. 7.0.2), and open GitHub issues still show gfx1201 gaps (FP8 falling back to FP32, missing hipBLASLt kernels).
- Strix Halo iGPU (RDNA 3.5, gfx1151) — support tier is unsettled even in AMD's own docs (GitHub issue: "Confusing ROCm support for gfx1151"). Works on Linux with ROCm 7.0–7.2, often needing a manual GFX-version override; Windows is weaker. For SDXL/Flux, community reports say Vulkan currently beats ROCm on this chip — none of this is AMD-confirmed.
-
Intel XPU — native
torch.xpusupport landed as a prototype-level feature in PyTorch 2.5 (October 2024), after Intel's own separate extension package (IPEX) had covered the gap; Intel has since wound down active IPEX development in favor of upstreaming directly into stock PyTorch. SDXL and Flux Schnell/Dev are genuinely workable on Arc consumer cards (the B580 gets roughly RTX-3070-class SDXL speeds), but it's the roughest of the three: manual attention/VAE slicing is required on memory-constrained cards, FP16 is reportedly unstable on integrated GPUs (BF16 recommended instead), and Flux needs IPEX-provided patches rather than running out of the box. Workstation-class Arc Pro cards specifically don't have public SDXL/Flux benchmarks yet, as far as I've found.
Ranked by how likely each one is to just work today: CUDA, then ROCm, then XPU — a real gap, not a rounding error, and worth knowing given the AMD and Intel cards sitting in the lab above.
Stakeholders
Stakeholder map
AI systems don't just have users — they have stakeholders: people, families, society, end-users, businesses, workers, and the natural ecosystems all of this ultimately runs on top of. Mapping cognitive load, reduced friction, energy cost, and the changing shape of the workforce against each other is how you tell the difference between what the world actually needs, what companies want, and what's just an impulsive decision driven by fear.
One way to frame it: society as a jungle versus society as a theme park. A jungle is a resilient, self-sustaining system; a theme park is a curated experience that collapses the moment nobody's maintaining it. A stable foundational workforce is what keeps the "jungle" healthy — which is the case for things like conservation-corps-style careers and mandatory civil service acting as an economic capacitor that ultimately feeds free enterprise, rather than draining it.
That baseline shows up in concrete, unglamorous places: water utilities, sewage and waste management, and police and fire departments — foundational salaries that put a floor under everyone else's spending, not a drain on it. Gutting these is usually a private-sector move chasing profit, not a public one chasing efficiency, and it rarely results in better service — see AT&T, which owns most of U.S. telecom infrastructure at this point and still can't get customer service right. Fix it, please and thank you.
Related reading: The Rainforest: Building the Next Silicon Valley and Creativity, Inc.
Architecture
The evolving system architecture
There's a common worry that running AI locally means keeping one giant model stuck in your GPU's memory forever. It doesn't have to work that way. A well-built system loads small, specialized models only when it actually needs them — swapping them in and out of the GPU like tools on a workbench, instead of buying one giant tool that tries to do everything at once. This is the exact philosophy Liquid AI is built around, and it works because modern memory connections can move data at 30+ GB/s — fast enough that swapping in a new small model barely registers as a delay.
Task arrives
Lock the GPU
Load the right small model
~80ms
Do the job
Unlock the GPU
This whole swap happens in well under a second — fast enough to disappear into the workflow entirely.
09
Go deeper: how fast is a model swap, really?
A 4B-parameter model, shrunk down (quantized) to save space, takes up about 2.6GB — the same size as Gemma 3's 4B variant covered earlier in this outline. Your GPU talks to the rest of the computer over a connection called PCIe, which on most modern gaming PCs moves data at roughly 32GB per second. Do the math: moving 2.6GB across that connection takes about 80 milliseconds — call it under a second once you add in some setup overhead. That's not a guess; it's arithmetic. A well-architected system doesn't need a huge model sitting in memory to do something complicated — it just needs to know which small model to grab next, and a fast enough connection to grab it.
10
Go deeper: real models built for this
This is exactly what Liquid AI's LFM2.5 lineup is built for. Instead of one big model, it's a whole family: text models from 230M up to an 8B mixture-of-experts variant, small vision models, an audio model, and a set of tiny specialists — ones just for search, translation, math, or handling sensitive data safely. All of it is open-weight (free to download and inspect) under an Apache-2.0-based license. That's what a swap-friendly system actually looks like in practice: not one model trying to do everything, but a bench of small, fast specialists you call in only when you need them.
11
Go deeper: why robots need this the most
Robots make this pattern non-negotiable. A robot exploring a cave or a tunnel doesn't have the battery or the cooling to keep a giant model running the whole time — see the tunnel-imaging and SLAM work in the Computer Vision section below. What it does have room for is a coordination layer and a rotation of small, focused models: one to see, one to plan a path, one to understand language, swapped in only when the task actually needs them. That's not a workaround for weak hardware — on a robot, it's simply the right way to build it, no matter your budget.
12
Go deeper: what if the model doesn't stay still?
Swapping a frozen model in and out is the easy case. The harder question is what happens when the model keeps changing itself after it's deployed — learning and updating its own behavior in the real world through reinforcement learning. That trades something you can test and trust for a moving target. AI systems that optimize hard for a reward tend to find sneaky shortcuts researchers didn't intend — a well-documented problem called reward hacking (or specification gaming). It gets riskier once the model can act in the real world and watch what happens next. A model you can swap out, inspect, and roll back is simply safer to build on than one that's quietly rewriting itself while it runs.
Ethics
Software Engineering Ethics
Building systems this capable comes with a responsibility that's easy to skip past in a purely technical writeup.
XLA Content Generation
XLA Content Generation
GPUs weren't originally built for any of this — they were designed for 3D graphics rendering, where the same transform (rotate, scale, project a vertex onto the screen) gets applied to millions of vertices and pixels in parallel. That workload is just matrix and vector math, so GPU ALUs ended up optimized for linear algebra almost by accident, which is exactly the hardware profile every model below actually needs. Long before any of this ran on a GPU, Brin and Page's PageRank algorithm — the dominant eigenvector of the web's own link matrix, found by power iteration — already showed that this kind of large-scale linear algebra could solve a real problem at internet scale.
Image generation
- Stable Diffusion XL (SDXL) — Stability AI, open-weight (CreativeML Open RAIL++-M).
- FLUX.2-dev — Black Forest Labs, a 32B rectified-flow transformer, open-weight under the non-commercial FLUX license.
- gpt-image-2 — OpenAI's current flagship text-to-image model, via the Images and Responses APIs.
- Midjourney — proprietary, Discord/web-based.
- Adobe Firefly — proprietary, marketed as trained only on licensed and public-domain content for commercial safety.
Music generation
- Suno (v5.5) — proprietary text-to-song generator, complete with vocals and production.
- Udio — proprietary text-to-song generator, Suno's main competitor.
- MusicGen — Meta, 3.3B parameters, open-weight (CC-BY-NC 4.0 weights, MIT-licensed code).
- Stable Audio Open 1.0 — Stability AI, open-weight, up to 47 seconds of stereo audio at 44.1kHz.
- Lyria 3.5 — Google DeepMind, proprietary, available through Google Flow Music, Gemini, and YouTube's Dream Track.
Speech-to-text
- Whisper large-v3 — OpenAI, open-weight, Apache 2.0.
- Parakeet TDT 0.6B v2 — NVIDIA, 600M parameters, open-weight (CC-BY-4.0), tops the Hugging Face Open ASR Leaderboard at 6.05% average word error rate.
- Universal-3.5 Pro — AssemblyAI, proprietary API.
- Nova-3 — Deepgram, proprietary API.
Robotics: Computer Vision
This is where the Engineers for Exploration work shows up again — SLAM, structure-from-motion (SFM), Agisoft-based photogrammetry, and point-cloud analysis, applied to mangrove canopy monitoring and Maya archaeological site mapping. On the hardware side, a lot of that field capture runs through a Luxonis OAK 4 D — active stereo depth (with IR illumination for low-light scenes) plus a 48MP RGB camera, a 6-core onboard CPU with 52 TOPS of AI inferencing, and its own Linux-based OS — built by Luxonis (the "OAK" name traces back to "OpenCV AI Kit," the original Kickstarter-funded collaboration with OpenCV.org) and paired with OpenCV itself for the actual perception pipeline.
Spector: An OpenCL FPGA Benchmark Suite (Gautier, Althoff, Meng, and Kastner — UC San Diego, FPT 2016) — from the same Kastner/Gautier lab, this one is a benchmark suite for exploring the design space of high-level synthesis tools on FPGAs, not a computer-vision paper itself, but the methodology that lab's FPGA computer-vision work builds on.
The final question
Name the problem.
Prove it's a pain point, not a vitamin. Map the trade before you build.
Every model, framework, and piece of hardware on this page is a means, not an end — none of it matters if you can't say plainly what problem it solves, and whether that problem is:
- A pain point — something people already feel, and will actively pay to fix, and keep using.
- A vitamin — nice to have, but easy to drop the moment attention or budget moves elsewhere.
Before committing to build, map the actual trade:
| Dimension | What to map |
|---|---|
| Value | Real financial return, customer loyalty, or time saved — normalized against a real cost basis (say, $100/hr) so the number means something, not a vague "efficiency" claim. |
| Effort | Not just what it takes to ship — what it takes to keep running and maintained afterward. |
| Risk | How this fails, and whether that failure creates liability or breaks the trust of the people who use it. |
| Alternatives | Will people actually choose this over what they already do — or is it solving a problem nobody was asking you to solve? |
| Cost | Not only infrastructure and hardware, but energy, salaries, public image, and the opportunity cost of building this instead of something else. |