Getting Started

This guide walks through the shortest path from an empty LLM Gateway configuration to a successful chat completion request.

Prerequisites

• Access to Axiom Cloud.
• Permission to manage LLM Gateway credentials.
• At least one provider account and API credential.
• An application or test client that can call an OpenAI-compatible API.

For production environments, also prepare:

• A credential rotation process.
• A fallback provider or secondary credential for critical workloads.
• Metrics scraping through Prometheus or VictoriaMetrics.
• Budget and usage review ownership.

Step 1: Open LLM Gateway

1. Sign in to Axiom Cloud.
2. Open LLM Gateway.
3. Review the Overview page to confirm the gateway is reachable.
4. Open Credentials to add your first provider key.

Step 2: Add a provider credential

In the Credentials tab:

1. Select Add Credential.
2. Choose the provider.
3. Enter a descriptive name, such as OpenAI Production or Anthropic Support Workloads.
4. Enter the provider API key or provider-specific configuration.
5. Set the default model when the provider requires it for routing.
6. Set the load balancing weight if this credential shares traffic with other credentials.
7. Save the credential.

Provider-specific credentials can include extra fields:

Step 3: Send a test request

Set your Axiom Cloud URL and session token:

export AXIOMCLOUD_URL="https://cloud.axiomstudio.ai"
export AXIOMCLOUD_SESSION="your-session-token"

Send a chat completion request:

curl -X POST "$AXIOMCLOUD_URL/rest/v1/llm-gateway/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Cookie: session=$AXIOMCLOUD_SESSION" \
  -d '{
    "model": "gpt-4",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Give me a one sentence status check."}
    ]
  }'

If you use the OpenAI SDK, point the client at the gateway base URL:

from openai import OpenAI

client = OpenAI(
    base_url="https://cloud.axiomstudio.ai/rest/v1/llm-gateway/v1/",
    api_key="your-session-token",
)

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello from Axiom LLM Gateway"}],
)

print(response.choices[0].message.content)

Step 4: Enable streaming when needed

Streaming uses Server-Sent Events and the same OpenAI-compatible endpoint:

curl -X POST "$AXIOMCLOUD_URL/rest/v1/llm-gateway/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Cookie: session=$AXIOMCLOUD_SESSION" \
  -N \
  -d '{
    "model": "gpt-4",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Write a short deployment checklist."}
    ]
  }'

Step 5: Review the request

After the request completes:

1. Open Overview for provider count, request rate, error rate, and latency percentiles.
2. Open Analytics to inspect provider, model, token, and request trends.
3. Open Audit Logs if you changed credentials or fallback settings.
4. Open FinOps when cost tracking is enabled for your providers.

Common first-run problems