Package inference

Interface GrpcService.InferBatchStatisticsOrBuilder

All Superinterfaces:
com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder
All Known Implementing Classes:
GrpcService.InferBatchStatistics, GrpcService.InferBatchStatistics.Builder
Enclosing class:
GrpcService
public static interface GrpcService.InferBatchStatisticsOrBuilder extends com.google.protobuf.MessageOrBuilder

  • Method Details

    • getBatchSize

      long getBatchSize()
      @@  .. cpp:var:: uint64 batch_size
@@
@@     The size of the batch.
@@
      uint64 batch_size = 1;
      Returns:
      The batchSize.

    • hasComputeInput

      boolean hasComputeInput()
      @@  .. cpp:var:: StatisticDuration compute_input
@@
@@     The count and cumulative duration to prepare input tensor data as
@@     required by the model framework / backend with the given batch size.
@@     For example, this duration should include the time to copy input
@@     tensor data to the GPU.
@@
      .inference.StatisticDuration compute_input = 2;
      Returns:
      Whether the computeInput field is set.

    • getComputeInput

      GrpcService.StatisticDuration getComputeInput()
      @@  .. cpp:var:: StatisticDuration compute_input
@@
@@     The count and cumulative duration to prepare input tensor data as
@@     required by the model framework / backend with the given batch size.
@@     For example, this duration should include the time to copy input
@@     tensor data to the GPU.
@@
      .inference.StatisticDuration compute_input = 2;
      Returns:
      The computeInput.

    • getComputeInputOrBuilder

      GrpcService.StatisticDurationOrBuilder getComputeInputOrBuilder()
      @@  .. cpp:var:: StatisticDuration compute_input
@@
@@     The count and cumulative duration to prepare input tensor data as
@@     required by the model framework / backend with the given batch size.
@@     For example, this duration should include the time to copy input
@@     tensor data to the GPU.
@@
      .inference.StatisticDuration compute_input = 2;

    • hasComputeInfer

      boolean hasComputeInfer()
      @@  .. cpp:var:: StatisticDuration compute_infer
@@
@@     The count and cumulative duration to execute the model with the given
@@     batch size.
@@
      .inference.StatisticDuration compute_infer = 3;
      Returns:
      Whether the computeInfer field is set.

    • getComputeInfer

      GrpcService.StatisticDuration getComputeInfer()
      @@  .. cpp:var:: StatisticDuration compute_infer
@@
@@     The count and cumulative duration to execute the model with the given
@@     batch size.
@@
      .inference.StatisticDuration compute_infer = 3;
      Returns:
      The computeInfer.

    • getComputeInferOrBuilder

      GrpcService.StatisticDurationOrBuilder getComputeInferOrBuilder()
      @@  .. cpp:var:: StatisticDuration compute_infer
@@
@@     The count and cumulative duration to execute the model with the given
@@     batch size.
@@
      .inference.StatisticDuration compute_infer = 3;

    • hasComputeOutput

      boolean hasComputeOutput()
      @@  .. cpp:var:: StatisticDuration compute_output
@@
@@     The count and cumulative duration to extract output tensor data
@@     produced by the model framework / backend with the given batch size.
@@     For example, this duration should include the time to copy output
@@     tensor data from the GPU.
@@
      .inference.StatisticDuration compute_output = 4;
      Returns:
      Whether the computeOutput field is set.

    • getComputeOutput

      GrpcService.StatisticDuration getComputeOutput()
      @@  .. cpp:var:: StatisticDuration compute_output
@@
@@     The count and cumulative duration to extract output tensor data
@@     produced by the model framework / backend with the given batch size.
@@     For example, this duration should include the time to copy output
@@     tensor data from the GPU.
@@
      .inference.StatisticDuration compute_output = 4;
      Returns:
      The computeOutput.

    • getComputeOutputOrBuilder

      GrpcService.StatisticDurationOrBuilder getComputeOutputOrBuilder()
      @@  .. cpp:var:: StatisticDuration compute_output
@@
@@     The count and cumulative duration to extract output tensor data
@@     produced by the model framework / backend with the given batch size.
@@     For example, this duration should include the time to copy output
@@     tensor data from the GPU.
@@
      .inference.StatisticDuration compute_output = 4;