AI & GPU

This part of the platform runs open-source AI models on rented GPUs, or on a server you already connected. You can take a model off Hugging Face and serve it behind an HTTP endpoint, give that model your own documents so it answers from them, fine-tune it on your own examples, or skip all of that and rent a GPU to run whatever you like.

One thing to know up front: models, knowledge bases and fine-tuning are the newest part of ClikDeploy. They are live, and they have test coverage, but they are not in the platform's general-availability set the way apps, domains and servers are. GPU compute is. Read a quote before you spend, and expect more rough edges here than elsewhere in these docs.

Every model action below has a CLI command. Knowledge bases, fine-tuning and raw GPU are driven from the dashboard or the REST API — there are no CLI commands for them yet.

The Gateway is one API in front of every model ClikDeploy serves. It speaks both the OpenAI and the Anthropic formats, so any OpenAI or Anthropic client works with a platform API key (Settings → Developer). Each model is listed once and served by the cheapest provider available. Calls are paid from your AI credit at the listed price.

The full reference — every endpoint with examples, model naming, provider choice, tags, idempotency, timeouts, headers and errors — is the Gateway API reference.

https://clikdeploy.com/v1

from openai import OpenAI

client = OpenAI(base_url="https://clikdeploy.com/v1", api_key="YOUR_API_KEY")
reply = client.chat.completions.create(model="auto", messages=[{"role": "user", "content": "Hello"}])

# Anthropic SDKs and Claude Code
export ANTHROPIC_BASE_URL=https://clikdeploy.com
export ANTHROPIC_API_KEY=YOUR_API_KEY

Endpoints, all under https://clikdeploy.com/v1:

POST /chat/completions and POST /responses (OpenAI) and POST /messages (Anthropic): streaming, tools and tool_choice, images, structured output (response_format), temperature, top_p, stop, seed and penalties. Reasoning effort is reasoning_effort (none to xhigh) or Anthropic's thinking, sent to every model known to reason; how closely a model follows it is up to its provider, and some ignore it. Reasoning streams back beside the answer. models: [...] names fallbacks tried in order.

GET /models and GET /models/{id}: every model with its context window, maximum output, capabilities, supported parameters, measured speed, and price per million tokens (full price, discount, and what you pay). auto lets the Gateway choose. POST /embeddings embeds text with the embedding models on the list.

GET /credits, PATCH /credits (automatic top-up on or off), POST /credits/checkout (buy credit), GET /usage?days=30 (spend and tokens per model), and GET /generations/{id} (one call's cost, tokens, latency and speed, for 30 days).

Speed: add :fast to a model name (glm-5:fast), or send "speed": "fast" or "service_tier": "priority", to be served by the fastest measured provider rather than the cheapest. Cost: every response's usage.cost is what the call cost you in USD. Headers: X-Generation-Id (the response's id, for /generations) and X-RateLimit-Remaining-Requests on every response. Send X-Session-Id to keep a conversation on one provider, so repeated prompts are cheaper through caching. Out of credit returns 402. Settings the Gateway cannot honour (n above 1, logprobs, audio and file input) are refused with 400, never silently ignored. ClikCode uses this same API.

Providers: both formats take OpenRouter's provider object to choose which of a model's providers serve it. only and ignore filter by provider id; order tries the providers named first, in that order; allow_fallbacks: false uses only those (or, without order, only the first provider); sort is price, throughput (measured tokens per second) or latency (measured time to first output), unmeasured providers last; require_parameters: true skips providers known not to take a setting you sent (tools, response_format, reasoning effort, temperature); max_price: { prompt, completion } (USD per million tokens, the price you are charged) skips any provider priced above it. Model suffixes: :floor sorts by price, :free uses free providers only. data_collection: "deny" and zdr: true are refused: the Gateway has no record of providers' retention or training policies. An unknown key or provider id is a 400 that lists the valid ids; when no provider matches, a 404 (no_provider_matches) says which setting ruled them all out.

{
  "model": "glm-5",
  "messages": [{ "role": "user", "content": "Hello" }],
  "provider": { "order": ["groq", "together"], "allow_fallbacks": false, "max_price": { "prompt": 1, "completion": 3 } }
}

A model deployment is one open-source model, loaded onto a GPU, serving an OpenAI-compatible HTTP endpoint. "Inference" is the act of asking that loaded model a question and getting an answer back; an "endpoint" is the URL you send that question to.

1. Find a model

Search Hugging Face by name. Ollama, Kaggle and a plain download URL also work as sources, via --source.

clikdeploy model search qwen2.5

2. Read the quote

Before renting anything, the platform works out what the model needs. This is the quote, and it is a plain GET you can call yourself. It comes back with: the model manifest, the available quantizations, the serving runtime it matched, an estimated VRAM requirement, the GPU it picked, the disk space the weights need, a price per hour, and any warnings.

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://clikdeploy.com/api/models/resolve?modelId=Qwen/Qwen2.5-7B-Instruct"

Two terms in that answer. "VRAM" is the memory on the GPU itself — the model's weights and its working state have to fit in it, which is what decides how big a GPU you need. "Quantization" is a smaller, less precise copy of the same model: a Q4_K_M build is roughly a quarter the size of the original and needs a much cheaper GPU, at some cost in answer quality. The quote defaults to a modest 4k context and one request at a time, so it sizes for practical serving rather than for the model's advertised maximum.

3. Deploy it

clikdeploy model deploy Qwen/Qwen2.5-7B-Instruct --quant Q4_K_M

With no --gpu, the platform picks the GPU from the quote. Add --server <serverId> to run the model on a machine you already connected instead of renting one. Weights are cached on a network volume, so a redeploy of the same model does not download them again.

Gated models — the ones Hugging Face makes you accept a licence for — need your own Hugging Face token connected first, or the weights download will fail with a 401.

The endpoint speaks the OpenAI chat API, so anything that can talk to OpenAI can talk to it. The base URL is your deployment id with /v1 on the end:

https://clikdeploy.com/api/models/deployments/DEPLOYMENT_ID/v1

Authenticate with either a platform API key or a scoped endpoint key. An endpoint key starts with mk_live_, works only for that one deployment, and is shown to you once when it is minted — which makes it the right thing to paste into your own app.

curl -X POST https://clikdeploy.com/api/models/deployments/DEPLOYMENT_ID/keys \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"my-app"}'

Then call the model:

curl -X POST https://clikdeploy.com/api/models/deployments/DEPLOYMENT_ID/v1/chat/completions \
  -H "Authorization: Bearer mk_live_YOUR_ENDPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}]}'

The reply is an OpenAI chat completion. Set "stream": true and the response is server-sent events, passed through byte for byte. Other /v1 paths are forwarded to the runtime unchanged.

Two per-deployment settings are worth knowing, both set with a PATCH to /api/models/deployments/<id> or from the dashboard: systemPrompt, which is prepended to every chat, and idleSleepMinutes, which stops the GPU after that many idle minutes so you stop paying for it.

A knowledge base is a store of your own documents that a model can look things up in before it answers. This is what "RAG" means — retrieval-augmented generation: find the relevant passages first, put them in front of the model, then let it answer. It is how you get a general model to answer about your handbook, your changelog or your support history without retraining it.

Creating one provisions a Qdrant vector database on the server you name. A "vector database" stores each passage next to an "embedding" — a list of numbers representing its meaning, which is what makes similarity search possible.

curl -X POST https://clikdeploy.com/api/knowledge \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Handbook","serverId":"YOUR_SERVER_ID"}'

• Add sources with POST /api/knowledge/<id>/sources: a URL, a block of text, or a file upload of up to 50MB. For anything larger, point it at a URL.
• Sync with POST /api/knowledge/<id>/sync. That splits each source into chunks, embeds them, and writes them to the store. It runs in the background; GET the same path for progress and per-source status.
• Attach it to a model with PATCH /api/models/deployments/<id> and {"knowledgeBaseId": "..."}. Passing null detaches it.

Once attached, chat requests through the endpoint have the matching passages added automatically. If retrieval fails, the request still goes through un-augmented rather than erroring — so a knowledge base that is misbehaving looks like a model that has forgotten your documents, not like an outage. You can also have the platform redact emails, phone numbers and card numbers before embedding, and you can embed using your own deployed embedding model instead of the default.

Fine-tuning teaches an existing model your own examples, producing a new model. What is supported is LoRA fine-tuning of text models, run with Axolotl on a one-shot GPU pod. "LoRA" trains a small set of extra weights alongside the frozen original instead of retraining the whole thing — far cheaper, and enough for tone, format and domain knowledge.

• Dataset. Upload a JSONL or CSV file (up to 50MB) or register one by URL at POST /api/datasets.
• Quote. POST /api/training/quote with a base model and a dataset returns sensible hyperparameter defaults, the GPU it would use, and an honest cost and time band — something like "roughly $3–$9, about 1–3 hours".
• Run. POST /api/training/jobs starts it. You must echo back the quote you were shown as acceptedQuoteMicroUsd; without it the platform refuses to rent a GPU. You can also set your own spendCeilingMicroUsd, which otherwise defaults to twice the accepted quote.
• Watch. GET /api/training/jobs/<id> reports progress parsed from the training logs. DELETE cancels and terminates the pod.

Checkpoints are written to your network volume as training runs, and the volume outlives the pod. If a job is cancelled, hits its ceiling or the pod dies, the checkpoints stay put and the job can be resumed rather than restarted.

When a run finishes, the merged weights sit on that volume and the job hands back a reference to them. Deploy that reference as a model like any other, using the volume source instead of huggingface — from there it is an endpoint with the same behaviour as the section above.

If you would rather run your own thing on a GPU or a CPU machine, there are two shapes.

• Pods are a GPU machine that stays up until you stop it, running an image from the GPU app catalogue, optionally with a network volume attached for model cache. Start, stop and list them at /api/gpu/pods.
• Jobs are one-off: you say how big a machine the work needs and the command to run, it runs to completion, and you pay only while it runs. Create one with POST /api/compute/runs, then follow it with GET /api/compute/runs/{id}, read its output with GET /api/compute/runs/{id}/logs, or stop it with POST /api/compute/runs/{id}/cancel.

A job is described by its size, not by a machine type: vcpu and ramGb for CPU, vramGb (and gpuCount) for GPU. Price it first with POST /api/compute/runs/quote, which answers whether such a machine is available, the likely price per hour, and a suggested maxUsdPerHour to allow. That limit is your price: a job never runs on a machine that costs you more per hour.

POST /api/compute/runs
{
  "hardware": "gpu",
  "workload": { "kind": "capacity", "id": "capacity" },
  "capacity": {
    "shape": { "hardware": "gpu", "vramGb": 24, "maxUsdPerHour": 1.5 },
    "workload": { "mode": "task", "command": "python train.py", "image": "pytorch/pytorch:latest" }
  },
  "input": {},
  "stateClass": "stateless",
  "maxRuntimeMs": 3600000
}

From the CLI the same job is clikdeploy compute runs create --hardware gpu --workload capacity --gpu-memory 24 --task --command 'python train.py' --max-usd-per-hour 1.50, and clikdeploy compute runs logs <id> -f follows its output. The dashboard runs one from Compute › Runs.

A GPU app (ComfyUI, Ollama, Whisper…) runs the same way, kept running until you stop it: POST /api/compute/apps { "app": "comfyui", "vramGb": 24 } (a JupyterLab GPU pod: POST /api/compute/pods { "hardware": "gpu" }). Price a size first with POST /api/compute/runs/quote; list and stop them with GET /api/compute/services and DELETE /api/compute/services/{id}.

You pay for compute time from your credit, at the price shown before you start. That price is computed in one place, so the number in the quote is the number you pay: the quote and the meter cannot disagree.

• You are billed for GPU time, so a model that is stopped or asleep is not costing GPU. Set an idle timeout and it stops itself.
• Serving a model on a server you already own rents no GPU at all — you are just using your own machine.
• Requests through a model endpoint have their token counts recorded for the usage view. Nothing is charged per token.
• A fine-tune cannot start without an accepted quote, and stops at its spend ceiling with checkpoints intact.

Deploying models and using GPU are available on every plan, including the free one. What your plan sets is how many models may be running at once, and how many endpoint keys and evaluation suites you can have. See pricing for the current limits.