Package inference

Interface ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpecOrBuilder

All Superinterfaces:
com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder
All Known Implementing Classes:
ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec, ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.Builder
Enclosing class:
ModelConfigOuterClass.ModelOptimizationPolicy.Cuda
public static interface ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpecOrBuilder extends com.google.protobuf.MessageOrBuilder

  • Method Details

    • getBatchSize

      int getBatchSize()
      @@      .. cpp:var:: int32 batch_size
@@
@@         The batch size of the CUDA graph. If 'max_batch_size' is 0,
@@         'batch_size' must be set to 0. Otherwise, 'batch_size' must
@@         be set to value between 1 and 'max_batch_size'.
@@
      int32 batch_size = 1;
      Returns:
      The batchSize.

    • getInputCount

      int getInputCount()
      @@      .. cpp:var:: map<string, Shape> input
@@
@@         The specification of the inputs. 'Shape' is the shape of the
@@         input without batching dimension.
@@
      map<string, .inference.ModelOptimizationPolicy.Cuda.GraphSpec.Shape> input = 2;

    • containsInput

      boolean containsInput(String key)
      @@      .. cpp:var:: map<string, Shape> input
@@
@@         The specification of the inputs. 'Shape' is the shape of the
@@         input without batching dimension.
@@
      map<string, .inference.ModelOptimizationPolicy.Cuda.GraphSpec.Shape> input = 2;

    • getInput

      @Deprecated Map<String,ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.Shape> getInput()
      Deprecated.
      Use getInputMap() instead.

    • getInputMap

      Map<String,ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.Shape> getInputMap()
      @@      .. cpp:var:: map<string, Shape> input
@@
@@         The specification of the inputs. 'Shape' is the shape of the
@@         input without batching dimension.
@@
      map<string, .inference.ModelOptimizationPolicy.Cuda.GraphSpec.Shape> input = 2;

    • getInputOrDefault

      ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.Shape getInputOrDefault(String key, ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.Shape defaultValue)
      @@      .. cpp:var:: map<string, Shape> input
@@
@@         The specification of the inputs. 'Shape' is the shape of the
@@         input without batching dimension.
@@
      map<string, .inference.ModelOptimizationPolicy.Cuda.GraphSpec.Shape> input = 2;

    • getInputOrThrow

      ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.Shape getInputOrThrow(String key)
      @@      .. cpp:var:: map<string, Shape> input
@@
@@         The specification of the inputs. 'Shape' is the shape of the
@@         input without batching dimension.
@@
      map<string, .inference.ModelOptimizationPolicy.Cuda.GraphSpec.Shape> input = 2;

    • hasGraphLowerBound

      boolean hasGraphLowerBound()
      @@      .. cpp:var:: LowerBound graph_lower_bound
@@
@@         Specify the lower bound of the CUDA graph. Optional.
@@         If specified, the graph can be used for input shapes and
@@         batch sizes that are in closed interval between the lower
@@         bound specification and graph specification. For dynamic
@@         shape model, this allows CUDA graphs to be launched
@@         frequently without capturing all possible shape combinations.
@@         However, using graph for shape combinations different from
@@         the one used for capturing introduces uninitialized data for
@@         execution and it may distort the inference result if
@@         the model is sensitive to uninitialized data.
@@
      .inference.ModelOptimizationPolicy.Cuda.GraphSpec.LowerBound graph_lower_bound = 3;
      Returns:
      Whether the graphLowerBound field is set.

    • getGraphLowerBound

      ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.LowerBound getGraphLowerBound()
      @@      .. cpp:var:: LowerBound graph_lower_bound
@@
@@         Specify the lower bound of the CUDA graph. Optional.
@@         If specified, the graph can be used for input shapes and
@@         batch sizes that are in closed interval between the lower
@@         bound specification and graph specification. For dynamic
@@         shape model, this allows CUDA graphs to be launched
@@         frequently without capturing all possible shape combinations.
@@         However, using graph for shape combinations different from
@@         the one used for capturing introduces uninitialized data for
@@         execution and it may distort the inference result if
@@         the model is sensitive to uninitialized data.
@@
      .inference.ModelOptimizationPolicy.Cuda.GraphSpec.LowerBound graph_lower_bound = 3;
      Returns:
      The graphLowerBound.

    • getGraphLowerBoundOrBuilder

      ModelConfigOuterClass.ModelOptimizationPolicy.Cuda.GraphSpec.LowerBoundOrBuilder getGraphLowerBoundOrBuilder()
      @@      .. cpp:var:: LowerBound graph_lower_bound
@@
@@         Specify the lower bound of the CUDA graph. Optional.
@@         If specified, the graph can be used for input shapes and
@@         batch sizes that are in closed interval between the lower
@@         bound specification and graph specification. For dynamic
@@         shape model, this allows CUDA graphs to be launched
@@         frequently without capturing all possible shape combinations.
@@         However, using graph for shape combinations different from
@@         the one used for capturing introduces uninitialized data for
@@         execution and it may distort the inference result if
@@         the model is sensitive to uninitialized data.
@@
      .inference.ModelOptimizationPolicy.Cuda.GraphSpec.LowerBound graph_lower_bound = 3;