Skip to content

Agents

If your agent harness supports external endpoint providers, you can configure your agent to utilize the ALCF Inference Service endpoints as a backend.

Installation

Most agents have an installation script that you can run with one command to install in ~/.local/bin.

Agent Install Command
OpenCode1 curl -fsSL https://opencode.ai/install | bash
Pi1 curl -fsSL https://pi.dev/install.sh | sh
OpenAI Codex curl -fsSL https://chatgpt.com/codex/install.sh | sh
Claude Code curl -fsSL https://claude.ai/install.sh | bash

Most scripts will require you to reload your environment with source ~/.bashrc before the agent will be in your PATH.

Alternatively, you can also install via your system package manager (i.e. brew, apt, etc.).

Codex Version Requirement

Codex removed support for the Chat Completions API in 0.95. Use 0.94 or lower to utilize ALCF Inference Service endpoints without Responses API support. If you use Nix, you can use this command to pull the correct version into an ephemeral shell: nix-shell -p codex -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/cc4bd5f859cdf01153d999995c20c9a457045bd6.tar.gz.

Automatic Configuration w/ alcf-ai

You can quick-configure most agents to use the ALCF Inference Service endpoints with alcf-ai (installable with pip install alcf-ai). This also handles authentication and pulling an API key from the service. See alcf-ai CLI and SDK for installation details and additional options.

alcf-ai agent configure <agent>

Supported agents include:

  • opencode
  • pi
  • codex
  • claude
Option Description
--include-experimental Also configure non-whitelisted (experimental) models.
-m, --default-model Default model for the agent configuration (codex and claude). Default: muse-spark-bf16.
-c, --default-cluster Cluster serving the default model (codex and claude). Default: minerva.

Refreshing API Keys

alcf-ai agent configure <agent> is idempotent, so you can re-run it to reconfigure an agent at any time. When a token helper (alcf-tokens or alcf-ai) is on your PATH, the generated configuration refreshes access tokens automatically. Otherwise, alcf-ai embeds the current access token and warns you to re-run the command when it expires.

Configuration uses the published model capabilities, including context limits and supported input types. Update alcf-ai and re-run configuration when new models are added to its allowlist. OpenCode and Pi expose configured models in their model pickers; choose the appropriate cluster/provider, such as Minerva Direct API (alcf-inference-service-minerva-api).

Agent API selection Reasoning controls
OpenCode Chat Completions Variants from the model's published effort levels.
Pi Responses when advertised; otherwise Chat Completions Thinking levels mapped to the model's published effort levels.
Codex Responses with current Codex versions Generated catalogs retain conservative settings: OpenAI-specific reasoning summaries and built-in tools are disabled, with no effort selector.
Claude Code Messages Client-dependent effort controls; requests must use the model's accepted values.

Muse Spark remains the default. Codex preserves an existing model/provider selection and generates a separate alcf-<cluster>-<model> profile for each configured model. Select that profile to switch models, and regenerate its version-matched catalog after upgrading Codex. Claude Code's --default-model option sets its model and Opus/Sonnet/Haiku aliases; omitting it selects Muse. See API reasoning fields for protocol-specific controls.

Manual Configuration

OpenCode

After installing opencode, place the following in your ~/.config/opencode/opencode.jsonc.

{
    ...
    "provider": {
        "alcf-inference-service-sophia": {
            "name": "ALCF Inference Service (Sophia)",
            "npm": "@ai-sdk/openai-compatible",
            "options": {
                "baseURL": "https://inference-api.alcf.anl.gov/resource_server/sophia/vllm/v1",
                "apiKey": "{env:ALCF_AI_TOKEN}"
            },
            "models": {
                "openai/gpt-oss-120b": {
                    "name": "openai/gpt-oss-120b",
                    "timeout": false,
                    "limit": {
                        "context": 65536,
                        "output": 0
                    }
                }
            }
        }
    }
    ...
}

Before running opencode, a valid token needs to be stored in the ALCF_AI_TOKEN environment variable. You can either set a key manually (see API Access) or utilize alcf-tokens.

alcf-tokens login inference # follow interactive instructions to login
export ALCF_AI_TOKEN="$(alcf-tokens get-token inference)" # pull a token and store

opencode

Pi

Add the following provider to your ~/.pi/agent/models.json.

{
    ...
    "providers": {
        "alcf-minerva": {
            "baseUrl": "https://inference-api.alcf.anl.gov/resource_server/minerva/api/v1",
                "api": "openai-completions",
                "apiKey": "!alcf-tokens get-token inference",
                "compat": {
                    "supportsDeveloperRole": false,
                    "supportsReasoningEffort": false
                },
                "models": [
                    { "id": "muse-spark-bf16" },
                    { "id": "inkling-bf16" },
                    {
                        "id": "mistral-large-4-nvfp4",
                        "api": "openai-responses",
                        "contextWindow": 1048576,
                        "input": ["text", "image"],
                        "reasoning": true,
                        "thinkingLevelMap": {
                            "minimal": null,
                            "low": "low",
                            "medium": "medium",
                            "high": "high",
                            "xhigh": null,
                            "max": null
                        },
                        "compat": { "supportsReasoningEffort": true }
                    }
                ]
        }
    }
    ...
}

Codex

Choose a model and a Direct API cluster with Responses support from Available Models, then set these variables in your terminal:

export model="<model>"
export cluster="<cluster>"
export ALCF_AI_TOKEN="$(alcf-tokens get-token inference)"

In your ~/.codex/config.toml, merge the following template with your existing settings. Replace <model> and <cluster> with the values selected above; TOML does not substitute these placeholders. The cluster selects the URL, while the model ID is sent in requests, not appended to the URL.

model = "<model>"
model_provider = "alcf-inference-service-<cluster>-api"

[model_providers."alcf-inference-service-<cluster>-api"]
name = "ALCF Inference Service (<cluster>)"
base_url = "https://inference-api.alcf.anl.gov/resource_server/<cluster>/api/v1"
env_key = "ALCF_AI_TOKEN"
wire_api = "responses"

In addition, you need to add a model_catalog_json for Codex to understand the model's capabilities. This script uses the exported model and cluster variables and reads the context limit and input types from the selected model's published capabilities. Catalog filenames include the cluster, with / in model IDs replaced by -.

Model Config Generation Script 
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
#!/usr/bin/env bash
set -euo pipefail

: "${model:?Set model to the served model ID}"
: "${cluster:?Set cluster to the serving cluster}"
: "${ALCF_AI_TOKEN:?Set ALCF_AI_TOKEN to a valid inference access token}"

CODEX_VERSION="$(
  codex --version |
    awk '{print $NF}' |
    sed 's/^v//'
)"

CATALOG_DIR="$HOME/.codex/catalogs"
UPSTREAM_CATALOG="$CATALOG_DIR/models-${CODEX_VERSION}.upstream.json"
UPSTREAM_PROMPT="$CATALOG_DIR/prompt-${CODEX_VERSION}.md"
MODEL_CATALOG="$CATALOG_DIR/${cluster}-${model//\//-}.json"

capabilities="$(
  curl -fsS --get \
    -H "Authorization: Bearer $ALCF_AI_TOKEN" \
    --data-urlencode "model_id=$model" \
    "https://inference-api.alcf.anl.gov/resource_server/${cluster}/models" |
    jq -e --arg model "$model" '
      .[] | select(.id == $model and .framework == "api")
      | .capabilities | select(.api_protocols | index("responses"))
    '
)"
CONTEXT_WINDOW="$(jq -er '.context_window_tokens' <<< "$capabilities")"
INPUT_MODALITIES="$(jq -ce '
  [.input_modalities[] | select(. == "text" or . == "image" or . == "audio")]
  | select(length > 0)
' <<< "$capabilities")"

mkdir -p "$CATALOG_DIR"

curl -fsSL \
  "https://raw.githubusercontent.com/openai/codex/rust-v${CODEX_VERSION}/codex-rs/models-manager/models.json" \
  -o "$UPSTREAM_CATALOG"

curl -fsSL \
  "https://raw.githubusercontent.com/openai/codex/rust-v${CODEX_VERSION}/codex-rs/models-manager/prompt.md" \
  -o "$UPSTREAM_PROMPT"

jq \
  --arg model "$model" \
  --arg cluster "$cluster" \
  --arg version "$CODEX_VERSION" \
  --argjson context_window "$CONTEXT_WINDOW" \
  --argjson input_modalities "$INPUT_MODALITIES" \
  --rawfile prompt "$UPSTREAM_PROMPT" '
    ([.models[] | select(.slug == "gpt-5.4")][0] // .models[0])
    | .slug = $model
    | .display_name = $model
    | .description = ($model + " served through ALCF " + $cluster)
    | .visibility = "list"
    | .supported_in_api = true
    | .priority = 0
    | .minimal_client_version = $version
    | .availability_nux = null
    | .upgrade = null
    | .context_window = $context_window
    | .max_context_window = $context_window
    | .auto_compact_token_limit = null
    | .effective_context_window_percent = 90
    | .comp_hash = null
    | .model_messages.instructions_template = $prompt
    | .model_messages.instructions_variables = null
    | .base_instructions = $prompt
    | .default_reasoning_level = null
    | .supported_reasoning_levels = []
    | .supports_reasoning_summary_parameter = false
    | .supports_reasoning_summaries = false
    | .default_reasoning_summary = "none"
    | .reasoning_summary_format = "none"
    | .support_verbosity = false
    | .default_verbosity = null
    | .shell_type = "shell_command"
    | .apply_patch_tool_type = "freeform"
    | .supports_search_tool = false
    | .web_search_tool_type = "text"
    | .experimental_supported_tools = []
    | .input_modalities = $input_modalities
    | .supports_image_detail_original = false
    | .prefer_websockets = false
    | .use_responses_lite = false
    | .tool_mode = null
    | .multi_agent_version = null
    | .auto_review_model_override = null
    | .model_specialty = null
    | .service_tiers = []
    | .additional_speed_tiers = []
    | .default_service_tier = null
    | .include_skills_usage_instructions = true
    | .include_plugin_usage_instructions = false
    | .include_apps_usage_instructions = false
    | (if has("supports_parallel_tool_calls") then .supports_parallel_tool_calls = false else . end)
    | (if has("node_repl_disabled") then .node_repl_disabled = true else . end)
    | (if has("node_repl_auto_review_required") then .node_repl_auto_review_required = false else . end)
    | {models: [.]}
  ' "$UPSTREAM_CATALOG" > "$MODEL_CATALOG"

jq -e --arg model "$model" '
  .models
  | length == 1
    and .[0].slug == $model
' "$MODEL_CATALOG" >/dev/null

printf 'Created %s for Codex %s\n' \
  "$MODEL_CATALOG" \
  "$CODEX_VERSION"

Version Compatibility

The catalog fields depend on your Codex version. If you upgrade Codex, re-run this script to regenerate the catalog with the correct schema for that version.

Finally, add a profile named alcf-<cluster>-<model>.config.toml beside config.toml, replacing / in the model ID with - in the filename. Replace <path> below with the catalog's absolute path printed by the script, and replace the model and cluster placeholders as above.

model_provider = "alcf-inference-service-<cluster>-api"
model = "<model>"
model_catalog_json = "<path>"

To run Codex, use the same model and cluster variables to select the profile. The manual configuration reads ALCF_AI_TOKEN; refresh that variable with alcf-tokens get-token inference when it expires. The automatic configuration above retains its token-helper refresh mechanism.

codex --disable apps --strict-config --profile "alcf-${cluster}-${model//\//-}" -c 'web_search="disabled"'

Suggested Models

Muse Spark remains the default. Other models with published Responses and tool-calling support can be selected using their generated profiles. Direct API endpoints with native Responses support need no Harmony-format translation proxy. Do not assume that OpenAI-specific built-in tools or reasoning-summary controls are supported by every model.

Claude Code

Add the following to ~/.claude/settings.json.

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://inference-api.alcf.anl.gov/resource_server/minerva/api",
    "ANTHROPIC_AUTH_TOKEN": "<your-api-key-here>",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "muse-spark-bf16",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "muse-spark-bf16",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "muse-spark-bf16"
  }
}

Base URL

ANTHROPIC_BASE_URL intentionally omits the trailing /v1 used by the OpenAI-compatible endpoints. Claude Code automatically appends /v1/messages to the base URL.

Anthropic Messages Support

You need to use a Direct API-backed endpoint that supports the Anthropic Messages API for Claude Code to function without shims or proxies (i.e. Minerva or Tara).

Other 3rd-Party Agents

Other OpenAI-compatible agents can also be configured to use the service, albeit with varying degrees of support. After familiarizing yourself with your agent's configuration format, see Available Clusters for integration choices. The model can be specified by its fully qualified name (see Available Models).

Shims/Proxies

Some AI agents need to have their inference requests translated and sanitized prior to calling the ALCF Inference Service endpoints for correct behavior. Several community-developed shims and resources are available to aid the integration of these agents with the service:

  1. alcf-proxy - Hongwei Jin
  2. DIY w/ llm-rosetta - Sam Foreman

Responses API Streaming Support

Streaming support is currently disabled for the OpenAI Responses API on non-Direct API endpoints. This means agents that rely on the Responses API as their default wire protocol will fail to send requests to clusters like Sophia. Either configure your agent to utilize the OpenAI Chat Completions API or switch clusters for compatibility.

Metis Tool Calling

There is a known issue related to SambaNova's sanitization of tool calls, leading to strange error responses like this when processing requests. 

{\"error\":\"Model started a function call but did not complete it.\",\"error_code\":null,\"error_model_output\":\"\\u003c|channel|\\u003eanalysis\\u003c|message|\\u003eWe needto adjust imports and usage in main.rs.\\n\\nSearch for clear_lines usage. Already seen at line 327. Need to modify that block.\\n\\nOpen around lines 320-340.\\u003c|end|\\u003e\\u003c|start|\\u003eassistant\\u003c|channel|\\u003ecommentary to=functions.read\\u003c|message|\\u003e{\\\"filePath\\\":\\\"/Users/ewong/Documents/alcf/inference-service/vibecoding/rust_vibecoding/src/main.rs\\\",\\\"offset\\\":320,\\\"limit\\\":80}\",\"error_param\":null,\"error_type\":\"Invalid function calling output.\"}

  1. OpenCode and Pi explicitly support third-party/custom provider endpoints. ↩↩