Skip to content
LayerScale

Getting started

1. Get a license key

Request a free key at layerscale.ai/get-license. It starts with LSK-. The engine refuses to start without one, and the same key is the API bearer token every client sends.

The free key allows one GPU, one live session and a context of up to 32,768 tokens, and is valid for one year. Pro licenses lift those limits.

2. Check the host

  • An NVIDIA GPU of the Hopper or Blackwell generation.
  • An NVIDIA driver that supports CUDA 13.3. The image carries its own CUDA runtime libraries; only the driver comes from the host.
  • Docker with the NVIDIA Container Toolkit, so docker run --gpus all works.

3. Get a model

LayerScale loads a checkpoint from a local directory of Hugging Face safetensors. It does not download from the Hub. Fetch the model first, for example with the Hugging Face CLI:

Terminal window
hf download mistralai/Devstral-Small-2507 --exclude 'consolidated*' --local-dir /models/Devstral-Small-2507

The directory needs config.json, a tokenizer (tokenizer.json or tekken.json) and the weights (model.safetensors, or model.safetensors.index.json with its shards). See Models for the directory layout and quantized checkpoints.

4. Start the server

Terminal window
docker run --gpus all -p 8080:8080 \
  -e LAYERSCALE_LICENSE_KEY=LSK-... \
  -v /models:/models:ro \
  layerscale/layerscale:latest \
  serve 0.0.0.0:8080 --model /models/Devstral-Small-2507

Everything after the image name is the engine’s command line. Left out, the image runs serve 0.0.0.0:8080 --model /models, so mounting a checkpoint directly at /models also works; its model id on the API is then models, because the id is the model directory’s name in lower case.

The engine logs one JSON object per line on stdout. It verifies the key, loads the weights, runs a warm-up and only then binds the port. The ready line carries the bound address. Until then the port refuses connections.

5. Send a request

/health needs no key:

Terminal window
curl http://127.0.0.1:8080/health

Everything else does. Send it as a bearer token:

Terminal window
curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $LAYERSCALE_LICENSE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is a KV cache?"}],
    "max_tokens": 128
  }'

The reply is an OpenAI chat.completion object. A model field in the request is accepted and ignored; the response carries the server’s model name.

Stream tokens with "stream": true:

Terminal window
curl -N http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $LAYERSCALE_LICENSE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Count to ten."}], "stream": true}'

Next steps

  • Configuration: context length, memory budget, multi-GPU, speculative decoding.
  • API reference: every route, parameter and error.
  • Sessions: keep a conversation’s KV resident between requests.