https://pagrim.github.io/tags/inference/ Recent content in Inference on Peter Grimshaw's Site Hugo -- gohugo.io en-gb Mon, 29 Sep 2025 08:05:01 +0100 https://pagrim.github.io/post/bentoml-chronos/ Mon, 29 Sep 2025 08:05:01 +0100 https://pagrim.github.io/post/bentoml-chronos/ When it comes to deploying machine learning models into production, there’s no shortage of tools available. I’ve been exploring the landscape of ML inference frameworks, trying to understand the trade-offs and strengths of different options. I spent a bit of time investigating BentoML a while back, and really liked user-friendly design and focus on model serving. How Widely used is BentoML? For fun—and a bit of insight—I compared three well-known ML serving tools using Google Trends: BentoML, NVIDIA’s Triton Inference Server, and KServe.