Package inference

Interface ModelConfigOuterClass.ModelSequenceBatching.StateOrBuilder

All Superinterfaces:
com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder
All Known Implementing Classes:
ModelConfigOuterClass.ModelSequenceBatching.State, ModelConfigOuterClass.ModelSequenceBatching.State.Builder
Enclosing class:
ModelConfigOuterClass.ModelSequenceBatching
public static interface ModelConfigOuterClass.ModelSequenceBatching.StateOrBuilder extends com.google.protobuf.MessageOrBuilder

  • Method Details

    • getInputName

      String getInputName()
      @@    .. cpp:var:: string input_name
@@
@@       The name of the model state input.
@@
      string input_name = 1;
      Returns:
      The inputName.

    • getInputNameBytes

      com.google.protobuf.ByteString getInputNameBytes()
      @@    .. cpp:var:: string input_name
@@
@@       The name of the model state input.
@@
      string input_name = 1;
      Returns:
      The bytes for inputName.

    • getOutputName

      String getOutputName()
      @@    .. cpp:var:: string output_name
@@
@@       The name of the model state output.
@@
      string output_name = 2;
      Returns:
      The outputName.

    • getOutputNameBytes

      com.google.protobuf.ByteString getOutputNameBytes()
      @@    .. cpp:var:: string output_name
@@
@@       The name of the model state output.
@@
      string output_name = 2;
      Returns:
      The bytes for outputName.

    • getDataTypeValue

      int getDataTypeValue()
      @@    .. cpp:var:: DataType data_type
@@
@@       The data-type of the state.
@@
      .inference.DataType data_type = 3;
      Returns:
      The enum numeric value on the wire for dataType.

    • getDataType

      ModelConfigOuterClass.DataType getDataType()
      @@    .. cpp:var:: DataType data_type
@@
@@       The data-type of the state.
@@
      .inference.DataType data_type = 3;
      Returns:
      The dataType.

    • getDimsList

      List<Long> getDimsList()
      @@    .. cpp:var:: int64 dim (repeated)
@@
@@       The dimension.
@@
      repeated int64 dims = 4;
      Returns:
      A list containing the dims.

    • getDimsCount

      int getDimsCount()
      @@    .. cpp:var:: int64 dim (repeated)
@@
@@       The dimension.
@@
      repeated int64 dims = 4;
      Returns:
      The count of dims.

    • getDims

      long getDims(int index)
      @@    .. cpp:var:: int64 dim (repeated)
@@
@@       The dimension.
@@
      repeated int64 dims = 4;
      Parameters:
      index - The index of the element to return.
      Returns:
      The dims at the given index.

    • getInitialStateList

      List<ModelConfigOuterClass.ModelSequenceBatching.InitialState> getInitialStateList()
      @@  .. cpp:var:: InitialState initial_state (repeated)
@@
@@     The optional field to specify the initial state for the model.
@@
      repeated .inference.ModelSequenceBatching.InitialState initial_state = 5;

    • getInitialState

      ModelConfigOuterClass.ModelSequenceBatching.InitialState getInitialState(int index)
      @@  .. cpp:var:: InitialState initial_state (repeated)
@@
@@     The optional field to specify the initial state for the model.
@@
      repeated .inference.ModelSequenceBatching.InitialState initial_state = 5;

    • getInitialStateCount

      int getInitialStateCount()
      @@  .. cpp:var:: InitialState initial_state (repeated)
@@
@@     The optional field to specify the initial state for the model.
@@
      repeated .inference.ModelSequenceBatching.InitialState initial_state = 5;

    • getInitialStateOrBuilderList

      List<? extends ModelConfigOuterClass.ModelSequenceBatching.InitialStateOrBuilder> getInitialStateOrBuilderList()
      @@  .. cpp:var:: InitialState initial_state (repeated)
@@
@@     The optional field to specify the initial state for the model.
@@
      repeated .inference.ModelSequenceBatching.InitialState initial_state = 5;

    • getInitialStateOrBuilder

      ModelConfigOuterClass.ModelSequenceBatching.InitialStateOrBuilder getInitialStateOrBuilder(int index)
      @@  .. cpp:var:: InitialState initial_state (repeated)
@@
@@     The optional field to specify the initial state for the model.
@@
      repeated .inference.ModelSequenceBatching.InitialState initial_state = 5;

    • getUseSameBufferForInputOutput

      boolean getUseSameBufferForInputOutput()
      @@  .. cpp:var:: bool use_same_buffer_for_input_output
@@
@@     The optional field to use a single buffer for both input and output
@@     state. Without this option, Triton allocates separate buffers
@@     for input and output state
@@     which can be problematic if the state size is
@@     large. This option reduces the memory usage by allocating a single
@@     buffer. Enabling this option is recommended whenever
@@     the input state is processed before the output state is written.
@@     When enabled the state
@@     will always be updated independent of whether
@@     TRITONBACKEND_StateUpdate is called
@@     (however TRITONBACKEND_StateUpdate should still be called for
@@     completeness).
@@
@@     The default value is false.
@@
      bool use_same_buffer_for_input_output = 6;
      Returns:
      The useSameBufferForInputOutput.

    • getUseGrowableMemory

      boolean getUseGrowableMemory()
      @@  .. cpp:var:: bool use_growable_memory
@@
@@     The optional field to enable an implicit state buffer to grow
@@     without reallocating or copying existing memory.
@@     Additional memory will be appended to the end of the buffer and
@@     existing data will be preserved.
@@     This option is only available for CUDA memory and requires enabling
@@     use_same_buffer_for_input_output. When using this option,
@@     StateBuffer call will always return CUDA memory even if CPU memory
@@     is requested.
@@
@@     The default value is false.
@@
      bool use_growable_memory = 7;
      Returns:
      The useGrowableMemory.