InferTap is cache analytics for production inference
InferTap is cache analytics for production inference. A hit rate tells you how often your cache helped. It cannot tell you what the misses cost, or how much compute and memory your fleet spends rebuilding and holding the same content. InferTap measures both, fleet-wide, outside the request path.