Soulstone: The Forged Local Engine

"A Portal is a whisper from the remote sky, but a Soulstone is a daemon in a bottle. It lives on local iron. It burns local electricity. It answers only the Magus."

A Soulstone is the local runtime Animator: a Quadlet/systemd-backed service that lives on local iron. A Soulstone Rune is the Codex TOML declaration that describes it. When inscribed in the Codex, the system transmutes the Soulstone Rune into physical Podman Quadlet manifests and services (see Containers (08)).

Unlike a remote API, a Soulstone requires the Magus to understand the physics of local hardware. For model-backed Soulstones, the Discipline of Animation must align with the model's mass and the silicon's capacity. For non-model Soulstones, the same principle applies to CPU, RAM, disk, sockets, credentials, and any other local substrate the service consumes.

💎 The Infrastructure Mapping

Every Soulstone Rune in the Codex is a concrete leaf config under the abstract SoulstoneConfig branch, such as GenericSoulstoneConfig, LlamaCppSoulstoneConfig, VllmSoulstoneConfig, or SglangSoulstoneConfig. The fields defined in the scroll shape the local runtime and the generated container manifest.

Model-Backed Disciplines of Animation

A Soulstone is inert until it is bound to an Animator adapter: the connector that turns a local service into routable capabilities. The current model-backed core ships with built-in Soulstone profiles for vLLM, SGLang, and llama.cpp. Additional disciplines can be introduced through extensions, including non-model services whose adapters expose observation, browsing, execution, or peer-network capabilities.

I. The Kinetic (vLLM)

"The Workhorse of the Iron."

• Best For: High-throughput chat, serving multiple agents simultaneously, and models that fit strictly within VRAM (e.g., Llama-3-70B AWQ on 2x3090).
• The Mechanic (Continuous Batching): The Kinetic engine creates a "fluid" memory space. If two Agents query the Soulstone simultaneously, vLLM splits the GPU's attention cycle, serving both in parallel slots. It is the only way to run a "Hive Mind" on limited silicon without queuing latency.
• The Constraint: It demands purity. The model must fit entirely in VRAM. If it overflows, it crashes.
• The Configuration:
  • Memory Greed: By default, vLLM consumes 90% of VRAM instantly for the KV Cache. During testing, curb its appetite or an OOM occurs before a single token is generated. Use --gpu-memory-utilization 0.9 to tune this.
  • The Batching Trap: For a single user, vLLM's aggressive batching can sometimes increase latency. During debugging, use --max-num-seqs 1 to force serial processing, though this defeats the engine's primary purpose.
  • Quantization: Excellent support for AWQ. Define --quantization awq explicitly.

II. The Weaver (SGLang)

"The Specialist of Loops."

• Best For: Agentic Orchestrators, complex tool-use loops, and structured data extraction.
• The Mechanic (Radix Attention): Unlike the Kinetic engine which sees memory as isolated blocks, The Weaver sees memory as a Tree.
  • The Loop: When an Agent tries a plan, fails, and backtracks to the system prompt to try again, The Weaver does not re-compute the prompt. It simply "branches" the tree from the existing memory node.
  • The Result: Massive efficiency gains for Agents that "think" in loops or multi-turn reasoning steps.
• The Hardware Reality (Ampere): SGLang utilizes the Marlin Kernel (--enable-marlin) for AWQ models. This is highly optimized for RTX 3090 architectures, often outperforming standard GEMM kernels.
• The Nuance: SGLang is strictly for NVIDIA. While vLLM attempts to support AMD/ROCm, SGLang focuses on CUDA purity.

III. The Titan (llama.cpp)

"The Burden of Atlas."

• Best For: Massive Models (MoE, 405B) that exceed a 48GB VRAM envelope, and Orchestration tasks where raw intelligence outweighs speed.
• The Mechanic (The Offload): The Titan accepts that the GPU is finite. It splits the model layer-by-layer. Layers 1-40 might live on the GPU (Fast), while layers 41-80 live in System RAM (Slow).
• The Flags of Power:
  • --n-gpu-layers: The slider of speed. Raise it until VRAM is nearly full.
  • --n-cpu-moe: A critical flag for Mixture-of-Experts (like Mixtral or DeepSeek). It allows the "Expert" layers to live in RAM while the attention heads stay on GPU.
• The Cost: Speed bleeds away the deeper the model reaches into System RAM. The PCIe bus becomes the bottleneck.
• The Solitude: The Titan is solitary. It generally processes one request at a time (Serial).

Router Specialization (llama.cpp)

llama.cpp is treated as a special runtime with two startup modes:

• Single Mode: starts with -m <model_path> and serves one model alias.
• Router Mode: starts without -m and uses --models-dir or --models-preset to load/unload models dynamically.

When startup_mode = "auto":

• if model_path is set -> single mode
• otherwise -> router mode

This allows a single Soulstone to expose a model catalog while still presenting one runtime endpoint to the dispatcher/binder at any given moment.

Mode/argument precedence is deterministic:

1. exec set explicitly in TOML → runtime adapter does not synthesize flags.
2. startup_mode set to single/router → forced mode.
3. startup_mode = "auto" → infer from model_path (single if set, else router).
4. extra_args → appended last, so users can override defaults without forking schema.

Capability State During Model Swaps

In router mode, a single llama.cpp Soulstone can serve different models over its lifetime without the container restarting. Each model load/unload transitions the Animator's capability state:

• Container boots → is_static=True for all capabilities the current model supports.
• Model swap triggered (via llama.cpp API) → old model's capabilities flip is_active=False.
• New model loads and warms → new model's capabilities flip is_active=True.
• The Orchestrator manages these transitions; no coven swap (Systemd restart) is required.

This means a single llama.cpp Soulstone can dynamically expose chat, vision, or embedding capability families as different models are loaded. The Dispatcher tracks is_active state and routes accordingly.

The Ritual of Compression (Quantization)

Models should not run in FP16 (Raw weight) unless H100-class hardware is available. The degradation in intelligence from 4-bit quantization is negligible compared to the massive gains in VRAM efficiency (allowing for larger context windows).

🤝 Coven Management (The Group Rule)

To manage finite VRAM, Soulstones declare their membership in Covens using the groups field.

• Inclusive Coexistence: If two Soulstones share at least one common group (e.g., groups = ["vision-state"]), they belong to the same Coven. Systemd allows them to run simultaneously.
• Exclusive Banishment: If two Soulstones share no common groups, they are mutually exclusive. The system generates a Conflicts= directive between them.

Example: A Vision Coven

# ~/.config/lychd/runes/animator/soulstones/sglang/vision_eye.toml
name = "eye"
description = "Reasoning and Vision engine."
image = "lmsysorg/sglang:latest"
runtime = "sglang"
groups = ["vision-ritual"]
port = 8780
model_path = "/models/qwen3-next-80b-awq"
tensor_parallel_size = 2
enable_marlin = true
# extra_args are appended after adapter defaults
extra_args = ["--reasoning-parser", "deepseek-r1"]

# ~/.config/lychd/runes/animator/soulstones/llamacpp/vision_scribe.toml
name = "scribe"
description = "Specialized OCR tool (Titan)."
image = "ghcr.io/ggerganov/llama.cpp:server"
runtime = "llamacpp"
groups = ["vision-ritual"] # Shares the group; will NOT be killed by 'eye'
port = 8781
model_path = "/models/moondream.gguf"
startup_mode = "single"
n_gpu_layers = 99

The Dispatcher later binds capability surfaces from the runtime connector exposed by these Soulstones. In this example those surfaces are model and tool capabilities, but the same placement law applies to non-model local services.

Podman Secret Hydration

Soulstones can reference Podman secrets directly with secret_env_files.

name = "private-runtime"
runtime = "vllm"
image = "vllm/vllm-openai:latest"
model_path = "/models/qwen-awq"

# ENV var -> Podman secret name
[secret_env_files]
HF_TOKEN_FILE = "hf_runtime_token"

At bind time:

1. LychD checks that each referenced secret exists in Podman.
2. Generated Quadlets emit Secret=hf_runtime_token.
3. The container env contains HF_TOKEN_FILE=/run/secrets/hf_runtime_token.

This keeps rune files reference-only while allowing runtime code to read credential files.

⚔️ The Law of Exclusivity

The Orchestrator uses these group definitions to manifest the machine's state.

1. The Intent: An Agent needs vision. The Orchestrator identifies the vision-ritual coven.
2. The Cleansing: Systemd automatically stops any active Soulstone services/Quadlet units that do not belong to vision-ritual (e.g., a heavy deep-think coven).
3. The Manifestation: All services tagged with vision-ritual are started in concert.