InferLite is a research-oriented LLM inference and optimization platform focused on making large language models faster, more memory-efficient, and more practical to deploy on constrained infrastructure.

I designed the project to benchmark different inference and quantization strategies across model sizes and deployment environments. It explores techniques including FP16, INT8, GPTQ, AWQ, and GGUF, with an emphasis on understanding the trade-offs between model quality, latency, throughput, memory consumption, and deployment cost.

The platform uses PyTorch and ONNX Runtime for model execution and optimization, with FastAPI providing the inference layer. Benchmarking workflows measure metrics such as time-to-first-token (TTFT), end-to-end latency, tokens/second, memory utilization, and resource efficiency. Docker-based deployment and observability components make the system easier to reproduce and monitor.

The project demonstrates practical experience in:

LLM inference optimization

Model quantization and compression

ONNX Runtime optimization

CPU/GPU inference benchmarking

Performance profiling

Memory and latency optimization

Production-oriented API design

Reproducible AI deployment

The goal is not simply to run an LLM, but to systematically understand how model and runtime optimizations affect real-world inference performance and deployment economics.