Registering a model

Before launching an inference instance, you register your model in EcoLink's Registry. This tells the platform where to find the weights. Two sources supported:

1. HuggingFace — pull from a public or gated HF repo at launch time
2. Shared filesystem — weights already living on one of your shared filesystems

From HuggingFace

Best for: standard off-the-shelf models and fine-tunes you've pushed to HF Hub.

Register

1. Console → Registry → Models → Add model.
2. Pick HuggingFace repo as the source.
3. Fill in:
   • Name — short friendly label (used as the model ID prefix in API calls)
   • HF repo — e.g., meta-llama/Llama-3.1-70B-Instruct
   • HF revision — branch / commit / tag (optional, defaults to main)
   • Category — chat / embedding / reranker / image / audio / video
   • HF token — only required for gated models (Llama, Gemma, some others). Your token stays on the EcoLink side and isn't visible to other users.

4. Save.

What happens at launch

When you launch an inference instance with an HF-backed model:

1. The first pod in each region pulls the model from HF into a PersistentVolume (~20 GB for 7B models, ~140 GB for 70B).
2. Subsequent replicas in the same region share the PVC — weights are downloaded once per region, not per pod.
3. Future launches of the same model reuse the PVC.

First-launch download time scales with model size and HF's bandwidth — typically 2–10 minutes for ≤13B models, 20–60 minutes for 70B+.

From a shared filesystem

Best for: fine-tunes you've trained on your own GPU instances, weights you want loaded from a fast local filesystem instead of HF, or custom model formats.

Put the weights in place

1. Launch a GPU instance with a shared filesystem attached at /shared.
2. Put the model in a known location: /shared/models/my-llama-finetune/ with config.json, tokenizer.json, .safetensors weights.
3. Terminate the instance (the filesystem persists).

Register

1. Console → Registry → Models → Add model.
2. Pick Shared filesystem as the source.
3. Fill in:
   • Name — short label
   • Shared filesystem — pick from your dropdown
   • Subfolder — the path inside the filesystem (e.g., models/my-llama-finetune). The full model path is <mount_path>/<subfolder>.
   • Category — same as above
   • Region — locked to the filesystem's region; the instance must launch in the same region

4. Save.

What happens at launch

The inference instance mounts the shared filesystem and loads weights directly from <mount_path>/<subfolder> — no download. Start-up is typically 30–90 seconds depending on model size and how quickly your container initializes.

Tradeoff: filesystem-backed models are pinned to a single region (the one their filesystem lives in). HF-backed models can run in any region.

Updating or re-registering

• Change HF revision to pull a newer version? Add a new registry entry with the new revision rather than editing in place — old inference instances keep using the old revision until restarted.
• Change FS subfolder? Same pattern: new registry entry, new inference instance.
• Delete a registered model from the Registry: it's safe as long as no inference instance is using it. If one is, the UI blocks deletion until you stop the instance.

Next step

Launch an inference instance with your registered model