Package com.gencior.triton.core.pojo

Class TritonInferStatistics

java.lang.Object

com.gencior.triton.core.pojo.TritonInferStatistics

public final class TritonInferStatistics
extends Object

Encapsulates comprehensive inference statistics for a model.

This class aggregates timing and success/failure metrics for all inference requests, including queue times, compute times, and cache hit/miss statistics. It provides detailed insights into model performance and inference processing stages.

This is an immutable object that wraps the gRPC message InferStatistics.

Since:

1.0.0

Author:

sachachoumiloff

Method Summary

Modifier and Type	Method	Description
static TritonInferStatistics	fromProto(GrpcService.InferStatistics proto)
TritonStatisticDuration	getCacheHit()	Returns statistics for inferences that hit the result cache.
TritonStatisticDuration	getCacheMiss()	Returns statistics for inferences that missed the result cache.
TritonStatisticDuration	getComputeInfer()	Returns statistics for time spent in inference computation.
TritonStatisticDuration	getComputeInput()	Returns statistics for time spent preparing input tensors.
TritonStatisticDuration	getComputeOutput()	Returns statistics for time spent in model inference computation.
TritonStatisticDuration	getFail()	Returns statistics for failed inferences.
TritonStatisticDuration	getQueue()	Returns statistics for time spent in the request queue.
TritonStatisticDuration	getSuccess()	Returns statistics for successfully completed inferences.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

fromProto

public static TritonInferStatistics fromProto(GrpcService.InferStatistics proto)

getSuccess

public TritonStatisticDuration getSuccess()

Returns statistics for successfully completed inferences.

getFail

public TritonStatisticDuration getFail()

Returns statistics for failed inferences.

getQueue

public TritonStatisticDuration getQueue()

Returns statistics for time spent in the request queue.

getComputeInput

public TritonStatisticDuration getComputeInput()

Returns statistics for time spent preparing input tensors.

getComputeOutput

public TritonStatisticDuration getComputeOutput()

Returns statistics for time spent in model inference computation.

getComputeInfer

public TritonStatisticDuration getComputeInfer()

Returns statistics for time spent in inference computation.

getCacheHit

public TritonStatisticDuration getCacheHit()

Returns statistics for inferences that hit the result cache.

getCacheMiss

public TritonStatisticDuration getCacheMiss()

Returns statistics for inferences that missed the result cache.