> ## Documentation Index
> Fetch the complete documentation index at: https://runpod-b18f5ded-lg-modelrepotest.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Model Repo testing
> Upload a model to Model Repo and deploy it to a Serverless endpoint.
Model Repo is currently in beta and is available on Mac and Linux only. Windows support is coming soon.
## Why use Model Repo
Model Repo lets you upload your own models to private storage on Runpod and attach them directly to Serverless endpoints. Key benefits:
* **Faster cold starts**: Models are pre-cached on worker hosts when available, so cold starts are typically faster. In some cases a model may still need to be fetched from storage.
* **No HuggingFace dependency**: Your models are stored in Runpod's infrastructure, so endpoints don't require an outbound download on every cold start.
* **Private storage**: Models are stored in your account and are not accessible to other users.
***
## Prerequisites
* Your email is feature-flagged for Model Repo access.
* `jq` is installed for parsing JSON output.
***
## Step 1: Set environment variables
Export the following before running any commands. **Make sure to set your actual API key — missing this is the most common source of auth errors later.**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export RUNPOD_API_URL="https://rest.runpod.io/v1"
export RUNPOD_GRAPHQL_URL="https://api.runpod.io/graphql"
export RUNPOD_API_KEY="your-api-key" # replace with your actual API key
export MODEL_NAME="my-model" # name to register your model under
export MODEL_PATH="/path/to/model" # local path to the model files you want to upload
```
Each upload under the same `MODEL_NAME` creates a new version of that model, not a new model. Use a different name if you want a separate model entry.
***
## Step 2: Install runpodctl
Already have runpodctl installed? Update it first:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
runpodctl update
```
**Option A: Install via Homebrew (recommended)**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
brew install runpod/runpodctl/runpodctl
```
**Option B: Build from source**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
brew install go # install Go, required to build runpodctl
git clone git@github.com:runpod/runpodctl.git
cd runpodctl
make # builds the binary to ./bin/runpodctl
```
If you build from source, the binary is at `./bin/runpodctl`. Either run it with that path, or add `./bin` to your `PATH`. The steps below use `runpodctl` — adjust accordingly.
***
## Step 3: Upload the model
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# --name: the name to register the model under in your repo
# --model-path: local path to the model files
# --create-upload: creates the upload session and transfers files
# --wait-for-hash: waits for the background hashing process to complete
runpodctl model add \
--name "$MODEL_NAME" \
--model-path "$MODEL_PATH" \
--create-upload \
--wait-for-hash
```
This outputs a JSON string listing all uploaded files.
***
## Step 4: Get your user ID and model hash
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export USER_ID="$(runpodctl user | jq -r '.id')" # your Runpod user ID
export MODEL_HASH="$(runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash')" # the model version hash
```
***
## Step 5: Deploy a Serverless endpoint with the model attached
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# --name: name for the endpoint
# --hub-id: Hub template to deploy
# --gpu-ids: comma-separated list of GPU pools to target (up to 3)
# --workers-max: maximum number of active workers
# --workers-min: minimum number of workers kept warm
# --model-reference: attaches your model to the endpoint
# --env MODEL_NAME: sets the model path on the worker
# --min-cuda-version: works around a bug in runpodctl
runpodctl serverless create \
--name "my_worker" \
--hub-id "cm8h09d9n000008jvh2rqdsmb" \
--gpu-ids "AMPERE_24,AMPERE_48,ADA_24" \
--workers-max 3 \
--workers-min 1 \
--model-reference "https://local/$USER_ID/$MODEL_NAME:$MODEL_HASH" \
--env MODEL_NAME="/runpod/model-store/modelrepo-local/models/$USER_ID/$MODEL_NAME/$MODEL_HASH" \
--min-cuda-version "13.0"
```
`--model-reference` is only supported with `--hub-id` and GPU endpoints. It is repeatable if you need to attach multiple models to the same endpoint. The `--env MODEL_NAME` flag passes the full local path to the worker so your handler knows where to find the model files.
You can target up to 3 GPU pools by passing a comma-separated list to `--gpu-ids`. Runpod schedules workers on whichever pool has availability.
**What "pre-cached" means:** When a worker starts, Runpod copies your model files from storage onto the worker host's local disk before the container boots. This eliminates the external download on cold start. Files stay on disk for as long as the worker is running; if the worker is replaced, the next worker pre-caches the files again on startup.
***
## Step 6: Verify the model is working
Send a test request to confirm the endpoint is live and the model is accessible. Replace `ENDPOINT_ID` with the ID returned in the previous step:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -s -X POST "https://api.runpod.ai/v2/${ENDPOINT_ID}/runsync" \
-H "Authorization: Bearer $RUNPOD_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": {"prompt": "hello"}}' | jq
```
A successful response confirms the endpoint is running and the model is attached. If the request fails with an auth error, verify that `RUNPOD_API_KEY` is set correctly.
You can also send requests from the web UI if you prefer a graphical interface.
***
## Step 7: Clean up
Delete the endpoint after testing to stop accruing spend. Use the web UI or:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
runpodctl serverless delete
```
***
## Removing a model
To remove a model and all its versions from your repo:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
runpodctl model remove --name "$MODEL_NAME"
```
This removes the entire model and all its versions. There is currently no way to remove a single version. Make sure no active endpoints reference this model before removing it.