AI & GPU

This part of the platform runs open-source AI models on rented GPUs, or on a server you already connected. You can take a model off Hugging Face and serve it behind an HTTP endpoint, give that model your own documents so it answers from them, fine-tune it on your own examples, or skip all of that and rent a GPU to run whatever you like.

One thing to know up front: models, knowledge bases and fine-tuning are the newest part of ClikDeploy. They are live, and they have test coverage, but they are not in the platform's general-availability set the way apps, domains and servers are. GPU compute is. Read a quote before you spend, and expect more rough edges here than elsewhere in these docs.

Every model action below has a CLI command. Knowledge bases, fine-tuning and raw GPU are driven from the dashboard or the REST API — there are no CLI commands for them yet.

A model deployment is one open-source model, loaded onto a GPU, serving an OpenAI-compatible HTTP endpoint. "Inference" is the act of asking that loaded model a question and getting an answer back; an "endpoint" is the URL you send that question to.

1. Find a model

Search Hugging Face by name. Ollama, Kaggle and a plain download URL also work as sources, via --source.

clikdeploy model search qwen2.5

2. Read the quote

Before renting anything, the platform works out what the model needs. This is the quote, and it is a plain GET you can call yourself. It comes back with: the model manifest, the available quantizations, the serving runtime it matched, an estimated VRAM requirement, the GPU it picked, the disk space the weights need, a price per hour, and any warnings.

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://clikdeploy.com/api/models/resolve?modelId=Qwen/Qwen2.5-7B-Instruct"

Two terms in that answer. "VRAM" is the memory on the GPU itself — the model's weights and its working state have to fit in it, which is what decides how big a GPU you need. "Quantization" is a smaller, less precise copy of the same model: a Q4_K_M build is roughly a quarter the size of the original and needs a much cheaper GPU, at some cost in answer quality. The quote defaults to a modest 4k context and one request at a time, so it sizes for practical serving rather than for the model's advertised maximum.

3. Deploy it

clikdeploy model deploy Qwen/Qwen2.5-7B-Instruct --quant Q4_K_M

With no --gpu, the platform picks the GPU from the quote. Add --server <serverId> to run the model on a machine you already connected instead of renting one. Weights are cached on a network volume, so a redeploy of the same model does not download them again.

Gated models — the ones Hugging Face makes you accept a licence for — need your own Hugging Face token connected first, or the weights download will fail with a 401.

The endpoint speaks the OpenAI chat API, so anything that can talk to OpenAI can talk to it. The base URL is your deployment id with /v1 on the end:

https://clikdeploy.com/api/models/deployments/DEPLOYMENT_ID/v1

Authenticate with either a platform API key or a scoped endpoint key. An endpoint key starts with mk_live_, works only for that one deployment, and is shown to you once when it is minted — which makes it the right thing to paste into your own app.

curl -X POST https://clikdeploy.com/api/models/deployments/DEPLOYMENT_ID/keys \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"my-app"}'

Then call the model:

curl -X POST https://clikdeploy.com/api/models/deployments/DEPLOYMENT_ID/v1/chat/completions \
  -H "Authorization: Bearer mk_live_YOUR_ENDPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}]}'

The reply is an OpenAI chat completion. Set "stream": true and the response is server-sent events, passed through byte for byte. Other /v1 paths are forwarded to the runtime unchanged.

Two per-deployment settings are worth knowing, both set with a PATCH to /api/models/deployments/<id> or from the dashboard: systemPrompt, which is prepended to every chat, and idleSleepMinutes, which stops the GPU after that many idle minutes so you stop paying for it.

A knowledge base is a store of your own documents that a model can look things up in before it answers. This is what "RAG" means — retrieval-augmented generation: find the relevant passages first, put them in front of the model, then let it answer. It is how you get a general model to answer about your handbook, your changelog or your support history without retraining it.

Creating one provisions a Qdrant vector database on the server you name. A "vector database" stores each passage next to an "embedding" — a list of numbers representing its meaning, which is what makes similarity search possible.

curl -X POST https://clikdeploy.com/api/knowledge \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Handbook","serverId":"YOUR_SERVER_ID"}'

• Add sources with POST /api/knowledge/<id>/sources: a URL, a block of text, or a file upload of up to 50MB. For anything larger, point it at a URL.
• Sync with POST /api/knowledge/<id>/sync. That splits each source into chunks, embeds them, and writes them to the store. It runs in the background; GET the same path for progress and per-source status.
• Attach it to a model with PATCH /api/models/deployments/<id> and {"knowledgeBaseId": "..."}. Passing null detaches it.

Once attached, chat requests through the endpoint have the matching passages added automatically. If retrieval fails, the request still goes through un-augmented rather than erroring — so a knowledge base that is misbehaving looks like a model that has forgotten your documents, not like an outage. You can also have the platform redact emails, phone numbers and card numbers before embedding, and you can embed using your own deployed embedding model instead of the default.

Fine-tuning teaches an existing model your own examples, producing a new model. What is supported is LoRA fine-tuning of text models, run with Axolotl on a one-shot GPU pod. "LoRA" trains a small set of extra weights alongside the frozen original instead of retraining the whole thing — far cheaper, and enough for tone, format and domain knowledge.

• Dataset. Upload a JSONL or CSV file (up to 50MB) or register one by URL at POST /api/datasets.
• Quote. POST /api/training/quote with a base model and a dataset returns sensible hyperparameter defaults, the GPU it would use, and an honest cost and time band — something like "roughly $3–$9, about 1–3 hours".
• Run. POST /api/training/jobs starts it. You must echo back the quote you were shown as acceptedQuoteMicroUsd; without it the platform refuses to rent a GPU. You can also set your own spendCeilingMicroUsd, which otherwise defaults to twice the accepted quote.
• Watch. GET /api/training/jobs/<id> reports progress parsed from the training logs. DELETE cancels and terminates the pod.

Checkpoints are written to your network volume as training runs, and the volume outlives the pod. If a job is cancelled, hits its ceiling or the pod dies, the checkpoints stay put and the job can be resumed rather than restarted.

When a run finishes, the merged weights sit on that volume and the job hands back a reference to them. Deploy that reference as a model like any other, using the volume source instead of huggingface — from there it is an endpoint with the same behaviour as the section above.

If you would rather run your own thing on a GPU, there are two shapes. Both are managed through /api/gpu and the dashboard.

• Pods are a GPU machine that stays up until you stop it, running an image from the GPU app catalogue. They are provisioned on RunPod, optionally with a network volume attached for model cache. Start, stop and list them at /api/gpu/pods.
• Serverless jobs are one-shot: you submit an input, it runs, you get a result, you pay for the run. POST /api/gpu/serverless, either awaiting the result inline or polling for it.

Serverless jobs can go to RunPod, Modal, Replicate, Fal, Vast, Lambda Labs or Cloudflare Workers. Which of those you can actually pick depends on which have credentials configured on the instance you are using — GET /api/gpu/serverless/providers lists every provider with a configured flag, and an unconfigured one returns a 503 rather than failing halfway through.

GET /api/gpu/skus lists the GPUs you can pick from with their hourly rate. Pass cloudType=COMMUNITY for cheaper community capacity instead of the default secure capacity.

GPU time is rented from a provider and passed through to you at that provider's cost plus a platform fee. The fee is a single percentage applied on the server, on top of the raw cost — 25% for GPU at the shipped default, though an operator can retune it. That means the number in the quote is the number you pay: there is one place the price is computed, and the quote and the meter cannot disagree.

• You are billed for GPU time, so a model that is stopped or asleep is not costing GPU. Set an idle timeout and it stops itself.
• Serving a model on a server you already own rents no GPU at all — you are just using your own machine.
• Requests through a model endpoint have their token counts recorded for the usage view. Nothing is charged per token.
• A fine-tune cannot start without an accepted quote, and stops at its spend ceiling with checkpoints intact.

Deploying models and using GPU are available on every plan, including the free one. What your plan sets is how many models may be running at once, and how many endpoint keys and evaluation suites you can have. See pricing for the current limits.