REST API DOCUMENTATION

VRAM Calculator API

Integrate programmatic LLM model requirements checks into your Dashboard, CI/CD workflows, or custom deployment scripts.

Get API Key MCP Server Docs

API Reference

Authentication

Calculations require an API token created in settings. Include it as a Bearer credential in the request header.

Authorization: Bearer YOUR_API_KEY_HERE

Never expose your API key in client-side applications. Use backend proxies when calling from browsers.

Generate Token

POST

/api/v1/vram-calculation/inference

Computes required GPU memory allocation breakdown and performance metrics specifically for LLM inference workloads.

Request JSON Body Parameters: Inference

Field	Type	Description & Enum Choices
Core Model and Hardware Parameters
`gpu_key` *	string	GPU profile key, such as `rtx_4090` or `h100_80`. See GET /gpus for a full list.
`llm_key` *	string	Model variant slug, such as `llama-3-8b-instruct` or `custom`. See GET /llms for a full list.
`quantization`	string	Weights quantization target. Choices: `fp32`, `fp16`, `fp8`, `int8`, `int4`, `q8`, `q4_k_m`, etc. Default is `fp16`.
`custom_vram`	number	Custom GPU memory capacity in GB. Required if gpu_key is `custom_discrete` or `custom_apple_silicon`.
`custom_bandwidth`	number	Custom GPU memory bandwidth in GB/s. Required if gpu_key is `custom_discrete` or `custom_apple_silicon`.
`custom_tflops`	number	Custom GPU FP16 compute performance in TFLOPS. Required if gpu_key is `custom_discrete` or `custom_apple_silicon`.
`framework_preset`	string	Runtime the estimate models: `auto`, `vllm`, `tensorrt_llm`, `llama_cpp`, `ollama`, `hf_transformers`, `diffusers` or `generic`. Default is `auto`, which picks `llama_cpp` for GGUF quantizations, `diffusers` for diffusion architectures, and `vllm` otherwise. `generic` reproduces the framework-agnostic estimate of earlier API versions with `legacy_auto` accepted as an alias. The response's `framework_preset` field names the runtime used.
Language Model Workload Parameters
`batch_size`	integer	Batch size. Default is 1.
`seq_length`	integer	Context sequence length in tokens. Default is 2048.
`concurrent_users`	integer	Number of concurrent users. Default is 1.
`kv_cache_quantization`	string	KV cache sharding/precision. Choices: `fp16`, `fp8`, `int8`, `int4`. Default is `fp16`.
`enable_prefix_caching`	boolean	Enable KV cache prefix caching. Defaults to the framework preset's behavior, which is on for `vllm` and `tensorrt_llm`. Takes effect with `shared_prefix_ratio` above 0.
`shared_prefix_ratio`	number	Fraction of prompt sequence that is a shared prefix. Range: 0.0 to 1.0. Default is 0.0.
`enable_continuous_batching`	boolean	Enable continuous batching. Defaults to the framework preset's behavior.
`inference_parallelism`	string	Parallelism strategy. Choices: `pipeline`, `tensor`. Default is `pipeline`.
`vocab_size`	integer	Custom vocabulary size, such as `128256` for Llama 3 or `256000` for Gemma. Default is 32000.
`ffn_intermediate_size`	integer	Custom MLP or SwiGLU intermediate dimension size, such as `14336` for Llama 3 8B.
`kv_source_layer_count`	integer	Number of layers generating shared KV cache states for compressed or hybrid architectures like CSA2.
`kv_latent_dim`	integer	Latent compressed KV cache dimension for architectures like MLA or CSA2.
`indexer_dim`	integer	Sparse routing indexer key dimension for architectures like DSA or CSA2.
Diffusion and DiT Workload Parameters
`image_width`	integer	Target image width in pixels for diffusion generation. Defaults to 1024 for images and 1280 for video.
`image_height`	integer	Target image height in pixels for diffusion generation. Defaults to 1024 for images and 720 for video.
`num_frames`	integer	Total frames generated for video diffusion models. Defaults to 1 for images and 49 for video.
`inference_steps`	integer	Denoising inference step count for diffusion generation. Default is 20.
`guidance_scale`	number	Classifier-free guidance scale value. Default is 1.0 for Schnell and 3.5 for Dev.
`encoder_offload`	boolean	Sequentially offload text encoders like T5 or CLIP to host CPU memory during denoising steps. Default is true.
`vae_tiling`	boolean	Enable spatial and temporal tiled VAE decoding to bound peak activation VRAM. Default is false for images and true for video.
Distributed Hardware and Topology Parameters
`num_gpus`	integer	Number of GPUs distributed on a single node. Defaults to 1 when cluster parameters are not used; mutually exclusive with `num_nodes` and `gpus_per_node`.
`num_nodes`	integer	Number of physical server chassis/nodes in the cluster. Required when using cluster configuration; mutually exclusive with `num_gpus`.
`gpus_per_node`	integer	Number of GPUs populated per server node in the cluster. Required when using cluster configuration; mutually exclusive with `num_gpus`.
`tp_degree`	integer	Tensor Parallelism degree shards to partition weights across GPUs. Default is 1.
`pp_degree`	integer	Pipeline Parallelism degree shards to distribute model layers sequentially across nodes or devices. Default is 1.
`interconnect_type`	string	Interconnect type for multi-GPU setups. Supported values include PCIe types `pcie3`, `pcie4`, `pcie5`; NVLink types `nvlink_bridge`, `nvlink_gen2` to `nvlink_gen6`; InfiniBand types `infiniband_hdr`, `infiniband_edr`; and Ethernet types `ethernet10g` to `ethernet400g`. Default is `pcie4`.
`inter_node_interconnect`	string	Network connection interconnect between different nodes in the cluster. E.g., `ethernet100g`, `ethernet400g`, `infiniband_hdr`.
Host and Storage Offloading Parameters
`enable_offloading`	boolean	Enable GPU offloading. Default is `false`.
`offload_target`	string	Storage target for offloaded layers. Choices: `cpu_ram`, `nvme`.
`num_offload_layers`	integer	Number of model layers to offload.
`percentage_offload`	number	Percentage of layers to offload, from 0 to 100.
`offload_kv_cache`	boolean	Offload KV Cache to storage target. Default is `false`.

cURL Example: Inference

Shell Request

curl -X POST https://apxml.com/api/v1/vram-calculation/inference \
  -H "Authorization: Bearer YOUR_API_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "gpu_key": "rtx_4090",
    "llm_key": "llama-3-8b-instruct",
    "quantization": "q4_k_m",
    "batch_size": 1,
    "seq_length": 2048
  }'

Python Example: Inference

Python Script

import requests

url = "https://apxml.com/api/v1/vram-calculation/inference"
headers = {
    "Authorization": "Bearer YOUR_API_KEY_HERE",
    "Content-Type": "application/json"
}
payload = {
    "gpu_key": "rtx_4090",
    "llm_key": "llama-3-8b-instruct",
    "quantization": "q4_k_m",
    "batch_size": 1,
    "seq_length": 2048
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

cURL Example: Diffusion Inference

Shell Request

curl -X POST https://apxml.com/api/v1/vram-calculation/inference \
  -H "Authorization: Bearer YOUR_API_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "gpu_key": "rtx_4090",
    "llm_key": "flux-1-schnell",
    "quantization": "fp16",
    "image_width": 1024,
    "image_height": 1024,
    "encoder_offload": true
  }'

Python Example: Diffusion Inference

Python Script

import requests

url = "https://apxml.com/api/v1/vram-calculation/inference"
headers = {
    "Authorization": "Bearer YOUR_API_KEY_HERE",
    "Content-Type": "application/json"
}
payload = {
    "gpu_key": "rtx_4090",
    "llm_key": "flux-1-schnell",
    "quantization": "fp16",
    "image_width": 1024,
    "image_height": 1024,
    "encoder_offload": True
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

POST

/api/v1/vram-calculation/finetuning

Computes required GPU memory allocation breakdown, optimization impacts, and dataset training speeds for LLM fine-tuning and training workloads.

Request JSON Body Parameters: Fine-tuning

Field	Type	Description & Enum Choices
`gpu_key` *	string	GPU profile key, such as `rtx_4090` or `h100_80`. See GET /gpus for a full list.
`llm_key` *	string	Model variant slug, such as `llama-3-8b-instruct` or `custom`. See GET /llms for a full list.
`finetuning_method`	string	Finetuning method. Choices: `full`, `lora`, `qlora`. Default is `lora`.
`fine_tuning_quantization`	string	Base model precision for full finetuning. Used when method is `full`. Choices: `fp32`, `fp16`, `fp8`. Default is `fp16`.
`lora_rank`	integer	LoRA rank size. Used when method is `lora` or `qlora`. Default is 16.
`gradient_accumulation_steps`	integer	Gradient accumulation steps. Default is 1.
`batch_size`	integer	Global batch size. Default is 1.
`seq_length`	integer	Context sequence length in tokens. Default is 2048.
`num_samples`	integer	Dataset size in total samples.
`tokens_per_sample`	integer	Average tokens per sample. Defaults to seq_length.
`num_epochs`	integer	Number of training epochs. Default is 3.
`framework_preset`	string	Training framework the estimate models: `auto`, `hf_trainer`, `unsloth`, `torchtune` or `generic`. Default is `auto`, which picks `hf_trainer`. `generic` reproduces the framework-agnostic estimate of earlier API versions with `legacy_auto` accepted as an alias. The response's `framework_preset` field names the framework used.
`optimization_config`	object	Advanced optimization flags object. Contains boolean flags like `flash_attention`, `gradient_checkpointing`, `use_8bit_optimizer`, `use_paged_optimizer`, `use_fused_kernels`, `activation_offloading`, etc., and integer zero_stage.
`custom_vram`	number	Custom GPU memory capacity in GB. Required if gpu_key is `custom_discrete` or `custom_apple_silicon`.
`custom_bandwidth`	number	Custom GPU memory bandwidth in GB/s. Required if gpu_key is `custom_discrete` or `custom_apple_silicon`.
`custom_tflops`	number	Custom GPU FP16 compute performance in TFLOPS. Required if gpu_key is `custom_discrete` or `custom_apple_silicon`.
`num_gpus`	integer	Number of GPUs distributed on a single node. Defaults to 1 when cluster parameters are not used; mutually exclusive with `num_nodes` and `gpus_per_node`.
`num_nodes`	integer	Number of physical server chassis/nodes in the cluster. Required when using cluster configuration; mutually exclusive with `num_gpus`.
`gpus_per_node`	integer	Number of GPUs populated per server node in the cluster. Required when using cluster configuration; mutually exclusive with `num_gpus`.
`enable_offloading`	boolean	Enable offloading to host system. Default is `false`.
`offload_target`	string	Storage target for offloaded layers. Choices: `cpu_ram`, `nvme`.
`interconnect_type`	string	Interconnect type for multi-GPU setups. Supported values include PCIe types `pcie3`, `pcie4`, `pcie5`; NVLink types `nvlink_bridge`, `nvlink_gen2` to `nvlink_gen6`; InfiniBand types `infiniband_hdr`, `infiniband_edr`; and Ethernet types `ethernet10g` to `ethernet400g`. Default is `pcie4`.
`tp_degree`	integer	Tensor Parallelism degree shards to partition weights across GPUs. Default is 1.
`pp_degree`	integer	Pipeline Parallelism degree shards to distribute model layers sequentially across nodes or devices. Default is 1.
`inter_node_interconnect`	string	Network connection interconnect between different nodes in the cluster. E.g., `ethernet100g`, `ethernet400g`, `infiniband_hdr`.

cURL Example: Fine-tuning

Shell Request

curl -X POST https://apxml.com/api/v1/vram-calculation/finetuning \
  -H "Authorization: Bearer YOUR_API_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "gpu_key": "rtx_4090",
    "llm_key": "llama-3-8b-instruct",
    "finetuning_method": "lora",
    "lora_rank": 16,
    "batch_size": 4,
    "seq_length": 2048
  }'

Python Example: Fine-tuning

Python Script

import requests

url = "https://apxml.com/api/v1/vram-calculation/finetuning"
headers = {
    "Authorization": "Bearer YOUR_API_KEY_HERE",
    "Content-Type": "application/json"
}
payload = {
    "gpu_key": "rtx_4090",
    "llm_key": "llama-3-8b-instruct",
    "finetuning_method": "lora",
    "lora_rank": 16,
    "batch_size": 4,
    "seq_length": 2048
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

Response Body: Inference Example

Inference Output

{
  "vram_usage": 6.84,
  "vram_percentage": 28.5,
  "actual_vram_percentage": 28.5,
  "memory_status": "okay",
  "static_shared_memory": 5.43,
  "per_user_memory": 0.52,
  "estimated_latency_tps": 45.2,
  "estimated_throughput_tps": 45.2,
  "per_user_tps": 45.2,
  "ms_per_token": 22.1,
  "ttft": 120,
  "ttft_ms": 120,
  "estimated_system_ram_required": 16.0,
  "offloaded_memory": 0.0,
  "estimated_power_draw": 320,
  "memory_details": {
    "Base Model Weights": 5.43,
    "KV Cache": 0.52,
    "Activations": 0.35,
    "Framework Overhead": 0.54
  },
  "memory_breakdown": [
    {
      "label": "Base Model Weights",
      "value": 79.4,
      "size_gb": 5.43
    },
    {
      "label": "KV Cache",
      "value": 7.6,
      "size_gb": 0.52
    }
  ],
  "runtime_advisories": [
    {
      "code": "PREFILL_STARVATION_RISK",
      "severity": "warning",
      "message": "Full-prompt prefill on long contexts stalls continuous batching token generation, causing high P99 latency jitter."
    }
  ]
};

Response Body: Fine-tuning Example

Fine-tuning Output

{
  "vram_usage": 32.4,
  "vram_percentage": 40.5,
  "actual_vram_percentage": 40.5,
  "memory_status": "okay",
  "static_shared_memory": 24.5,
  "training_tps": 12.0,
  "samples_per_second": 1.5,
  "steps_per_second": 0.18,
  "total_tokens": 10000000.0,
  "total_training_time_hours": 4.5,
  "estimated_system_ram_required": 32.0,
  "offloaded_memory": 0.0,
  "estimated_power_draw": 650,
  "active_optimizations": [
    "gradient_checkpointing",
    "flash_attention"
  ],
  "optimization_preset": "custom",
  "memory_details": {
    "Base Model Weights": 24.5,
    "Optimizer States": 29.4,
    "Gradients": 10.8
  },
  "memory_breakdown": [
    {
      "label": "Base Model Weights",
      "value": 38.2,
      "size_gb": 24.5
    },
    {
      "label": "Optimizer States",
      "value": 45.1,
      "size_gb": 29.4
    },
    {
      "label": "Gradients",
      "value": 16.7,
      "size_gb": 10.8
    }
  ]
}

Memory Status Enum Options

The memory_status field returns one of the following lowercase enum keys:

Enum Value	Description
`okay`	Fits with safe margin representing 50% or less of VRAM
`moderate`	Fits with moderate usage representing 75% or less of VRAM
`high`	Tight fit representing 90% or less of VRAM
`very_high`	Very close to capacity, exceeding 90% of VRAM
`insufficient`	Exceeds available VRAM capacity
`error`	GPU configuration or sizing calculation error

Runtime diagnostics and cluster metrics

The response includes diagnostic advisories and multi-node topology metrics when applicable:

Field	Type	Description
`runtime_advisories`	array of objects	Array of advisory objects containing code, severity, and message, evaluating conditions such as memory allocator churn, KV cache swapping penalties, and hardware bottlenecks.
`pipeline_asymmetry`	optional object	Stage memory imbalance metrics including stage_0_vram_gb, final_stage_vram_gb, and outer_stage_imbalance_ratio returned when pipeline parallelism pp_degree is greater than 1.
`disaggregated_network_budget`	optional object	Cross-node KV transfer requirements including kv_transfer_size_per_request_gb, required_bandwidth_gbs, and min_nic_line_rate returned for multi-node deployments when num_nodes is greater than 1.

Runtime advisory codes

The runtime_advisories array returns structured performance bottlenecks and stability advisories:

Advisory Code	Severity	Trigger Condition and Operational Impact
Language Model Advisories
`PCIE_KV_SWAP_HAZARD`	warning	Host memory KV cache swapping over PCIe bus adds significant per-token latency.
`PREFILL_STARVATION_RISK`	warning	Full prompt prefill on sequence lengths at or above 4096 tokens stalls continuous batching token generation.
`PREFIX_CACHE_ALIGNMENT`	advisory	Sequence length is not aligned to the runtime KV block size. Unaligned tokens cannot be shared.
`SPECULATIVE_KV_REDUCTION`	advisory	Draft model VRAM footprint reduces remaining KV cache capacity and lowers concurrency saturation knee.
Diffusion and Video Advisories
`VAE_UNTILED_OOM_HAZARD`	warning	Running video generation without VAE tiling causes high memory fragmentation during the final decode step. Enable VAE tiling to prevent out of memory errors.
Distributed and Parallelism Advisories
`PIPELINE_STAGE_ASYMMETRY`	warning	Outer stages require more memory due to embedding and projection layers. Sizing must account for stage memory imbalance.
`INTER_NODE_TP_BOTTLENECK`	warning	Tensor parallelism degree exceeds GPUs per node. All-reduce synchronization across inter-node network fabric introduces latency penalties.

Hardware & Model Listing Routes

GET

/api/v1/vram-calculation/gpus

Retrieve valid gpu_key parameters, hardware models, and VRAM capacities.

curl https://apxml.com/api/v1/vram-calculation/gpus \
  -H "Authorization: Bearer YOUR_API_KEY_HERE"

JSON Response

[
  {
    "key": "rtx_4090",
    "label": "NVIDIA GeForce RTX 4090",
    "memory": 24
  },
  {
    "key": "h100_80",
    "label": "NVIDIA H100 80GB",
    "memory": 80
  }
]

GET

/api/v1/vram-calculation/llms

Retrieve supported llm_key parameters mapped to model variant configurations. Supports query pagination parameters such as page and limit, for example ?page=1&limit=100.

curl https://apxml.com/api/v1/vram-calculation/llms \
  -H "Authorization: Bearer YOUR_API_KEY_HERE"

JSON Response

[
  {
    "key": "llama-3-8b-instruct",
    "name": "Llama 3 8B Instruct"
  },
  {
    "key": "llama-3.1-70b-instruct",
    "name": "Llama 3.1 70B Instruct"
  }
]

About Pricing Contact Terms of Use Privacy Policy