Python toolkit for building production-ready LLM applications. Modular utilities for prompts, RAG, agents, structured outputs, and multi-provider support.
Was this section helpful?
transformers library, a standard for loading and using various models, including quantized ones.bitsandbytes, a library enabling efficient 8-bit and 4-bit quantization for PyTorch models, integrated with transformers.