Troubleshooting

The engine logs one JSON object per line on stdout. A startup failure is a single startup_failed line whose message is one of those below. In the Docker image the loader verifies the license before the engine starts and reports its own failures the same way, as a loader_failed line: a missing key there reads a license key is required: set LAYERSCALE_LICENSE_KEY or pass --license-key <LSK-...>, and a key that does not verify carries the license messages below behind a license error: prefix. Request errors come back in the route’s JSON envelope with the status shown.

Startup

Message	Cause and fix
a license key is required: pass --license-key <LSK-...> or set LAYERSCALE_LICENSE_KEY	No key. Get one at layerscale.ai/get-license.
license key must start with LSK-	The value is not a LayerScale key. Check for a truncated or wrong variable.
license key contains invalid characters, license key decodes to N bytes, expected 112	The key was mangled when copied.
license key signature is invalid	The key was not issued for this engine, or has been edited.
license expired at unix ...	Past the key’s expiry and its two-day grace. Renew at layerscale.ai/pricing.
the free license runs on one GPU; tensor parallelism needs a Pro license (M requested; pass --tp 1)	A free key with --tp above 1, or on a multi-GPU host with no --tp, which means every GPU. Pass --tp 1, or upgrade to Pro.
license permits at most N GPU(s) (M requested; pass --tp N)	A Pro key asked for more GPUs than it permits, with --tp or because the host shows more and no --tp was given. Pass --tp N.
--model <DIR|ORG/NAME> is required; run --help for usage	Every command needs --model.
--model ORG/NAME: ORG/NAME is a gated model: ... no token is set ...	The repo is gated. Accept its license on its Hugging Face page, create a read token at huggingface.co/settings/tokens and set HF_TOKEN (in Docker, -e HF_TOKEN=hf_...).
--model ORG/NAME: ... is a gated model, and Hugging Face rejected the token in HF_TOKEN as invalid or expired ...	The token is wrong or revoked. Create a new one.
--model ORG/NAME: ... is a gated model, and the account that owns the token in HF_TOKEN has not been granted access to it ...	Accept the license on the model’s page; a manually reviewed request can take a while. A fine-grained token also needs read access to public gated repos.
--model ORG/NAME: no model ORG/NAME on Hugging Face ...	The repo id is misspelled, or the repo is private and HF_TOKEN cannot read it.
--model ORG/NAME: downloading from Hugging Face needs curl, which is not installed ...	A build run outside the image on a host without curl. Install curl and ca-certificates, or pass a directory.
--model ORG/NAME: could not resolve huggingface.co ..., could not reach huggingface.co ...	No network. Download the model elsewhere and pass its directory, or mount a Hugging Face cache that has it.
--model ORG/NAME: TLS with huggingface.co failed ...	curl cannot verify the Hub’s certificate; install ca-certificates.
--model ORG/NAME: could not write ... into the Hugging Face cache ...	The disk is full or the cache directory is read-only. Point HF_HOME or HF_HUB_CACHE at a writable volume.
--model ORG/NAME: not in the Hugging Face cache at ..., and HF_HUB_OFFLINE is set ...	Downloads are off. Unset HF_HUB_OFFLINE or fetch the model first.
unknown command '...'; run --help for usage	A flag was mistyped and landed as the command, or the command was misspelled.
config.json: architecture '...' is not implemented	The model family is not supported. The message lists the families this build knows.
layer N needs ..., which the GPU backend does not implement	The checkpoint uses a layer feature outside the supported set: a chunked attention mask that is not a whole number of KV pages, a sliding window at a head_dim other than 64 or 128, a scaled routed expert, or routers that differ between layers. A latent-attention or Gated DeltaNet shape the kernels are not instantiated for is refused in the same words, without the layer N prefix.
no tokenizer.json or tekken.json in ...	The directory has no tokenizer. Check the download completed.
model.safetensors.index.json: missing 'weight_map'	The shard index is damaged; download the checkpoint again.
config.json has no max_position_embeddings; pass --ctx	The checkpoint does not state a context length.
the KV pool is N tokens but the context is M (...); one full context must fit	Not enough memory for one context after the weights. Raise --gpu-memory-utilization or --max-total-tokens, lower --ctx, use --kv-cache-dtype fp8, or add GPUs with --tp (a Pro license). The parenthesis says where the context length came from: --ctx, --ctx N, capped to the license's context ceiling, the license's context ceiling, or the model's default context.
no accelerator: the CUDA runtime reports "..."; ... / no accelerator: the CUDA runtime sees no device; ...	The GPU build found no device before loading anything. The line carries the runtime’s own words and the cause the probe points to: no driver visible (the container was started without --gpus), a host driver older than the image’s toolkit (CUDA 13.x needs a driver that reports 13.0 or newer, the R580 branch), or CUDA_VISIBLE_DEVICES hiding every device. --device auto runs on the CPU instead and logs device_fallback.
--tp N needs N GPUs, this host shows M	More ranks were asked for than the process can see. Pass --tp M, or give the container more devices.
a device among the first N reports 0 bytes of memory	A visible card answered the memory query with nothing; check it with the vendor’s smi tool.
--gpu-memory-utilization X is not in (0, 1]	The value must be a fraction above 0 and at most 1.
this checkpoint's tokenizer has no ... token, so it was not trained on the ... chat template this engine renders for its architecture	The checkpoint uses a template variant the engine does not render for this architecture.
--kv-cache-dtype fp8 needs head_dim 128 and this model's is ..., --kv-cache-dtype fp8 the sliding-window attention kernels are built over a bf16 cache only	fp8 KV needs head_dim 128 and a checkpoint with no sliding-window layers. Use bfloat16, or auto.
config.json quantization_config: quant_method '...' is not implemented	An unsupported quantization format; see Models.
--device cuda: this build is CPU-only (built with --no-default-features --features cpu)	The binary was built without a GPU backend; the published images carry one.
cannot bind ADDR: ...	The port is taken or the address is not local.
the warm-up failed: ...	A request of the warm-up the engine runs before it binds failed, so a client’s would have: the rest of the line is that request’s error, as in the Requests table below. The process exits non-zero and the port never opens.

GPU and container

Symptom	Cause and fix
no kernel image is available for execution on the device	The GPU is older than Hopper. The image supports Hopper and Blackwell only.
The container exits at once with no accelerator: ...	The host driver is too old for the image’s CUDA 13.3 runtime (it needs the R580 branch, a driver reporting CUDA 13.0 or newer), or the container was started without --gpus. The line says which.
The port refuses connections after start	Expected: the engine binds after loading and warming up. Follow the logs for ready.
/health answers 503	The engine thread stopped or stalled. Restart the container and check the last error line in the log. After a fault the engine cannot recover from (an engine_fatal line) the process exits non-zero by itself, for a supervisor to restart.

Requests

Status and message	Meaning
401 missing or invalid license key; send it as 'Authorization: Bearer <LSK-...>'	The server was started with --api-key: send the key it was started with, as a bearer token or x-api-key.
400 prompt has N tokens but the context limit is M (raise --ctx)	The conversation no longer fits. Trim it, or restart with a larger --ctx (a free license caps it at 32,768).
400 '...' is not supported	A request parameter the engine refuses by name. The API reference lists what each route accepts.
400 'max_tokens' is required	/v1/messages requires it, as Anthropic’s API does.
400 'stop' takes at most 4 sequences	Send at most 4 stop strings of at most 64 bytes each.
404 no such session '...'	Sessions must be created with POST /v1/sessions first.
409 session '...' has a request in flight	One request per session at a time. Wait for it, or delete the session.
409 session '...' already exists	Pick another id, or delete the old session.
403 the free license permits at most 1 live session(s); delete one first	The free tier’s session limit. Delete a session or upgrade to Pro.
408 request took too long to arrive	The full request did not arrive within 60 seconds.
413 body too large	Bodies are limited to 32 MiB.
503 server is holding too many request bodies	More than 256 MiB of request bodies are buffered at once. Retry.
500 engine stopped	The engine thread has died. /health answers 503 too; restart.
500 backend error: ... the engine must restart	At --tp above 1 a GPU never arrived at a step and the others’ wait for it ran out (ten seconds). The requests in flight get this error, the log carries engine_fatal, and the process exits non-zero by itself; the supervisor’s restart is the recovery.
500 backend error: the engine stopped on a fault it cannot recover from	The request arrived while the engine was stopping on such a fault. Retry once the process is back.

A streaming response that fails after it has started carries the error as its last data: frame (or an error event on /v1/messages), because the status line is already 200.

Output quality

• Sampling defaults are temperature 0.6 and top_p 0.9. Send "temperature": 0 for greedy output.
• Prompt-lookup speculative decoding is on by default and is not bit-exact against a plain decode. Start with --speculative-algorithm none when comparing outputs across runs or engines.
• A tool call the model wrote but the engine could not parse is returned as plain content. The tool_call_unparseable log line and the layerscale_tool_calls_malformed_total metric show when this happens.
• A reply cut off at max_tokens has finish_reason: "length"; raise it.

Performance figures for supported models and GPUs are at layerscale.ai/benchmarks.