AX Engine 7.5.x ships dense 27B past the DRAM ceiling.See the numbers
AX Engine.
Get started

AX ENGINE / DOCUMENTATION

Use the local API

Send a text request from curl or Python, or connect an OpenAI-compatible local client to a running AX Engine server.

Start a server and set your connection

The default server listens on 127.0.0.1:31418. In the GUI, start a model and use Copy API URL and Copy API key. In a CLI Terminal, the example below starts a local server with a key you choose. Replace the key placeholder with your own value. Keep the server running and use a second Terminal for requests.

Terminal
export AX_ENGINE_API_KEY="replace-with-your-own-local-key"
ax-engine serve "$HOME/Models/my-model" --port 31418 -- --api-key "$AX_ENGINE_API_KEY"

Configure the request Terminal

Set the base URL and key in your second Terminal. For a GUI-owned server, paste the copied URL and its current key instead of the example values. The base URL ends in /v1. CLI authentication is optional by default, but these examples use an authenticated server consistently. Keep keys private.

Terminal
export AX_ENGINE_BASE_URL="http://127.0.0.1:31418/v1"
export AX_ENGINE_API_KEY="replace-with-the-server-key"

Discover the model ID

List the models on this server, then replace the model placeholder below with one returned data[].id. A download alias or directory name is not necessarily the API model ID. Repeat discovery after changing the loaded models.

Terminal
curl -fsS "$AX_ENGINE_BASE_URL/models" \
  -H "Authorization: Bearer $AX_ENGINE_API_KEY"

Terminal
export AX_ENGINE_MODEL="replace-with-an-id-from-v1-models"

Example: a text chat with curl

After setting AX_ENGINE_MODEL to an advertised ID, send a short chat request. The response uses the OpenAI chat-completions shape. Read the assistant’s reply in choices[0].message.content. Start with max_tokens set low while checking the connection.

Terminal
curl -fsS "$AX_ENGINE_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $AX_ENGINE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d @- <<JSON
{
  "model": "$AX_ENGINE_MODEL",
  "messages": [{"role": "user", "content": "Explain a Python list in one sentence."}],
  "max_tokens": 128
}
JSON

Example: stream the reply

Use stream: true for server-sent events and curl -N to display chunks as they arrive. A client should consume the event stream until completion rather than parse the entire response as a single JSON document.

Terminal
curl -fsS -N "$AX_ENGINE_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $AX_ENGINE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d @- <<JSON
{
  "model": "$AX_ENGINE_MODEL",
  "messages": [{"role": "user", "content": "Write a short welcome message."}],
  "max_tokens": 128,
  "stream": true
}
JSON

Example: Python with no client dependency

Save this code as client.py, then run python3 client.py in the request Terminal after exporting the same base URL, key and model ID. It uses the standard library and prints the assistant’s reply. No AX Engine Python SDK is required to call the HTTP API.

Python
import json
import os
from urllib.request import Request, urlopen

payload = {
    "model": os.environ["AX_ENGINE_MODEL"],
    "messages": [{"role": "user", "content": "Say hello from my Mac."}],
    "max_tokens": 128,
}
request = Request(
    os.environ["AX_ENGINE_BASE_URL"].rstrip("/") + "/chat/completions",
    data=json.dumps(payload).encode("utf-8"),
    headers={
        "Content-Type": "application/json",
        "Authorization": "Bearer " + os.environ["AX_ENGINE_API_KEY"],
    },
)
with urlopen(request, timeout=120) as response:
    result = json.load(response)
print(result["choices"][0]["message"]["content"])

Configure an API-compatible client

Choose a custom OpenAI-compatible provider in your client. Set its base URL to the copied /v1 URL, enter the current server key, and select the advertised model ID. AX Engine implements a documented subset of the API; tools, structured output and media depend on the model and backend. Use the compatibility reference before enabling advanced features.

Detailed reference