Model instances (Compute API)

A model instance is a dedicated endpoint serving one model on GPUs reserved for you. With an API key you can launch one, follow it until it is running, read its server log and stop it. These are the same operations the console's Model Instances page performs.

Prerequisites and the shared rules (auth, rate limits, errors) are on the overview. Listing and reading work with Read-only Compute access; launching and stopping need full Compute access.

Endpoints

Method	Path	What it does
GET	/inference-instances	Your model instances, newest first
POST	/inference-instances	Launch one
GET	/inference-instances/{id}	One instance, and the credit held for it
GET	/inference-instances/{id}/deployments	Its deployment, with status
GET	/inference-instances/{id}/deployments/{deploymentId}/logs	The model server's recent output
DELETE	/inference-instances/{id}	Stop it and settle billing

Attaching fine-tuned adapters to a running instance is done in the console.

Launching from a template

The usual way to launch by API is from a model_instance template. The template already decides the image, command, GPUs and model:

1. GET /compute-templates/{id}. Check that template.target_kind is model_instance.
2. POST /compute-templates/quote with template_id. Note gpu_type and upfront_hold_usd.
3. POST /inference-instances with the fields of spec, gpu_type from the quote, template_id, and a name.

import time, requests

API, KEY = "https://api.ecohash.com", "eco_YOUR_KEY"
H = {"Authorization": f"Bearer {KEY}"}
TPL = 48

got = requests.get(f"{API}/compute-templates/{TPL}", headers=H).json()
assert got["template"]["target_kind"] == "model_instance"
spec = got["spec"]

q = requests.post(f"{API}/compute-templates/quote", headers=H, json={"template_id": TPL}).json()
print("hold now:", q["upfront_hold_usd"])

inst = requests.post(f"{API}/inference-instances", headers=H, json={
    **spec, "gpu_type": q["gpu_type"], "template_id": TPL, "name": "deepseek",
}).json()

while True:
    d = requests.get(f"{API}/inference-instances/{inst['id']}", headers=H).json()["instance"]
    if d["status"] == "running":
        break
    if d["status"] in ("failed", "stopped"):
        raise SystemExit(d["status"])
    time.sleep(15)          # large models take several minutes to load

print(f'call it as model "{d["name"]}:{d["id"]}"')

template_id on its own launches nothing. It records which template the launch came from, so send the spec's fields with it.

POST /inference-instances

The body is what a template's spec holds. The main fields:

Field	Type	Notes
name	string	Shown in lists. Together with the id, it is what you call the model by
gpu_type, gpu_count	string, int	The GPUs for the instance. gpu_type is required
mode	string	vllm (the platform's serving image) or custom (your container_image)
startup_command	string	Required: the command that starts the server (a template carries it)
container_image, service_port	string, int	For custom mode; container_image is required there
user_model_id	int	Serve a registered model: your own, or one another account published. It also fixes the region, which is the model's
platform_model_pvc	string	Serve a model from the platform catalogue
model_path, hf_token	string	Serve a HuggingFace repo; a gated one needs your token
openai_compatible, custom_api_path	bool, string	How requests reach the server; see Calling your endpoint
template_id	int	The template this launch came from

Returns 202 Accepted with the instance, status pending.

Billing: 24 hours of the hourly rate are held from your credit at launch and settled as it runs (Cost and lifecycle). If the launch can't be covered, nothing is created and you get 402. On a reservation nothing is held, and the instance runs on your reserved nodes.

Following an instance

• GET /inference-instances/{id} returns { "instance": {…}, "held_amount": 30.24 }.
• instance.status is one of:
  • pending: being created, or loading weights;
  • running: ready to call;
  • degraded;
  • stopped;
  • failed.
• Loading a large model takes several minutes. Poll every 15–30 seconds.
• GET …/deployments lists the deployment. Its status is pending, loading, running, failed or stopped, and its id is the one the logs call takes.
• GET …/deployments/{deploymentId}/logs returns { "logs": "…" }, the tail of the model server's output. Use it when a launch is slow or fails.

Calling it

Once running, call it like any platform model, with "<name>:<id>" as the model. For example, "deepseek:255" with POST /v1/chat/completions. See Calling your endpoint and OpenAI-compatible. A server that isn't OpenAI-compatible is reached at /inference-instances/{id}/proxy/<path>.

Stopping

DELETE /inference-instances/{id} stops the instance. Usage is settled to the moment it stops, and the rest of the hold is refunded. It returns 202 with the instance. On an instance that has already stopped or failed, the same call removes it from your list.

Related

• Templates API
• User inference overview
• GPU MCP server: launch_template, get_model_instance and terminate_model_instance