Package inference

Interface ModelConfigOuterClass.ModelOptimizationPolicyOrBuilder

All Superinterfaces:
com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder
All Known Implementing Classes:
ModelConfigOuterClass.ModelOptimizationPolicy, ModelConfigOuterClass.ModelOptimizationPolicy.Builder
Enclosing class:
ModelConfigOuterClass
public static interface ModelConfigOuterClass.ModelOptimizationPolicyOrBuilder extends com.google.protobuf.MessageOrBuilder

  • Method Details

    • hasGraph

      boolean hasGraph()
      @@  .. cpp:var:: Graph graph
@@
@@     The graph optimization setting for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.Graph graph = 1;
      Returns:
      Whether the graph field is set.

    • getGraph

      ModelConfigOuterClass.ModelOptimizationPolicy.Graph getGraph()
      @@  .. cpp:var:: Graph graph
@@
@@     The graph optimization setting for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.Graph graph = 1;
      Returns:
      The graph.

    • getGraphOrBuilder

      ModelConfigOuterClass.ModelOptimizationPolicy.GraphOrBuilder getGraphOrBuilder()
      @@  .. cpp:var:: Graph graph
@@
@@     The graph optimization setting for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.Graph graph = 1;

    • getPriorityValue

      int getPriorityValue()
      @@  .. cpp:var:: ModelPriority priority
@@
@@     The priority setting for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.ModelPriority priority = 2;
      Returns:
      The enum numeric value on the wire for priority.

    • getPriority

      ModelConfigOuterClass.ModelOptimizationPolicy.ModelPriority getPriority()
      @@  .. cpp:var:: ModelPriority priority
@@
@@     The priority setting for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.ModelPriority priority = 2;
      Returns:
      The priority.

    • hasCuda

      boolean hasCuda()
      @@  .. cpp:var:: Cuda cuda
@@
@@     CUDA-specific optimization settings. Optional.
@@
      .inference.ModelOptimizationPolicy.Cuda cuda = 3;
      Returns:
      Whether the cuda field is set.

    • getCuda

      ModelConfigOuterClass.ModelOptimizationPolicy.Cuda getCuda()
      @@  .. cpp:var:: Cuda cuda
@@
@@     CUDA-specific optimization settings. Optional.
@@
      .inference.ModelOptimizationPolicy.Cuda cuda = 3;
      Returns:
      The cuda.

    • getCudaOrBuilder

      ModelConfigOuterClass.ModelOptimizationPolicy.CudaOrBuilder getCudaOrBuilder()
      @@  .. cpp:var:: Cuda cuda
@@
@@     CUDA-specific optimization settings. Optional.
@@
      .inference.ModelOptimizationPolicy.Cuda cuda = 3;

    • hasExecutionAccelerators

      boolean hasExecutionAccelerators()
      @@  .. cpp:var:: ExecutionAccelerators execution_accelerators
@@
@@     The accelerators used for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.ExecutionAccelerators execution_accelerators = 4;
      Returns:
      Whether the executionAccelerators field is set.

    • getExecutionAccelerators

      ModelConfigOuterClass.ModelOptimizationPolicy.ExecutionAccelerators getExecutionAccelerators()
      @@  .. cpp:var:: ExecutionAccelerators execution_accelerators
@@
@@     The accelerators used for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.ExecutionAccelerators execution_accelerators = 4;
      Returns:
      The executionAccelerators.

    • getExecutionAcceleratorsOrBuilder

      ModelConfigOuterClass.ModelOptimizationPolicy.ExecutionAcceleratorsOrBuilder getExecutionAcceleratorsOrBuilder()
      @@  .. cpp:var:: ExecutionAccelerators execution_accelerators
@@
@@     The accelerators used for the model. Optional.
@@
      .inference.ModelOptimizationPolicy.ExecutionAccelerators execution_accelerators = 4;

    • hasInputPinnedMemory

      boolean hasInputPinnedMemory()
      @@  .. cpp:var:: PinnedMemoryBuffer input_pinned_memory
@@
@@     Use pinned memory buffer when the data transfer for inputs
@@     is between GPU memory and non-pinned system memory.
@@     Default is true.
@@
      .inference.ModelOptimizationPolicy.PinnedMemoryBuffer input_pinned_memory = 5;
      Returns:
      Whether the inputPinnedMemory field is set.

    • getInputPinnedMemory

      ModelConfigOuterClass.ModelOptimizationPolicy.PinnedMemoryBuffer getInputPinnedMemory()
      @@  .. cpp:var:: PinnedMemoryBuffer input_pinned_memory
@@
@@     Use pinned memory buffer when the data transfer for inputs
@@     is between GPU memory and non-pinned system memory.
@@     Default is true.
@@
      .inference.ModelOptimizationPolicy.PinnedMemoryBuffer input_pinned_memory = 5;
      Returns:
      The inputPinnedMemory.

    • getInputPinnedMemoryOrBuilder

      ModelConfigOuterClass.ModelOptimizationPolicy.PinnedMemoryBufferOrBuilder getInputPinnedMemoryOrBuilder()
      @@  .. cpp:var:: PinnedMemoryBuffer input_pinned_memory
@@
@@     Use pinned memory buffer when the data transfer for inputs
@@     is between GPU memory and non-pinned system memory.
@@     Default is true.
@@
      .inference.ModelOptimizationPolicy.PinnedMemoryBuffer input_pinned_memory = 5;

    • hasOutputPinnedMemory

      boolean hasOutputPinnedMemory()
      @@  .. cpp:var:: PinnedMemoryBuffer output_pinned_memory
@@
@@     Use pinned memory buffer when the data transfer for outputs
@@     is between GPU memory and non-pinned system memory.
@@     Default is true.
@@
      .inference.ModelOptimizationPolicy.PinnedMemoryBuffer output_pinned_memory = 6;
      Returns:
      Whether the outputPinnedMemory field is set.

    • getOutputPinnedMemory

      ModelConfigOuterClass.ModelOptimizationPolicy.PinnedMemoryBuffer getOutputPinnedMemory()
      @@  .. cpp:var:: PinnedMemoryBuffer output_pinned_memory
@@
@@     Use pinned memory buffer when the data transfer for outputs
@@     is between GPU memory and non-pinned system memory.
@@     Default is true.
@@
      .inference.ModelOptimizationPolicy.PinnedMemoryBuffer output_pinned_memory = 6;
      Returns:
      The outputPinnedMemory.

    • getOutputPinnedMemoryOrBuilder

      ModelConfigOuterClass.ModelOptimizationPolicy.PinnedMemoryBufferOrBuilder getOutputPinnedMemoryOrBuilder()
      @@  .. cpp:var:: PinnedMemoryBuffer output_pinned_memory
@@
@@     Use pinned memory buffer when the data transfer for outputs
@@     is between GPU memory and non-pinned system memory.
@@     Default is true.
@@
      .inference.ModelOptimizationPolicy.PinnedMemoryBuffer output_pinned_memory = 6;

    • getGatherKernelBufferThreshold

      int getGatherKernelBufferThreshold()
      @@  .. cpp:var:: uint32 gather_kernel_buffer_threshold
@@
@@     The backend may use a gather kernel to gather input data if the
@@     device has direct access to the source buffer and the destination
@@     buffer. In such case, the gather kernel will be used only if the
@@     number of buffers to be gathered is greater or equal to
@@     the specified value. If 0, the gather kernel will be disabled.
@@     Default value is 0.
@@     Currently only recognized by TensorRT backend.
@@
      uint32 gather_kernel_buffer_threshold = 7;
      Returns:
      The gatherKernelBufferThreshold.

    • getEagerBatching

      boolean getEagerBatching()
      @@  .. cpp:var:: bool eager_batching
@@
@@     Start preparing the next batch before the model instance is ready
@@     for the next inference. This option can be used to overlap the
@@     batch preparation with model execution, with the trade-off that
@@     the next batch might be smaller than what it could have been.
@@     Default value is false.
@@     Currently only recognized by TensorRT backend.
@@
      bool eager_batching = 8;
      Returns:
      The eagerBatching.