Package com.gencior.triton.grpc

Class InferParametersBuilder

java.lang.Object
com.gencior.triton.grpc.InferParametersBuilder
public class InferParametersBuilder extends Object
Builder for constructing custom inference request parameters.

This class provides a fluent API for building a map of custom parameters to send with inference requests to Triton. Parameters are name-value pairs that can be used to control various aspects of inference execution, such as request priority, binary data output options, and model-specific behavior.

Reserved parameter names (sequence_id, sequence_start, sequence_end, priority, binary_data_output) are protected and cannot be set using this builder as they are managed separately.

Usage Example:


 Map<String, InferParameter> params = new InferParametersBuilder()
     .add("temperature", 0.7)
     .add("max_tokens", 100L)
     .add("use_cache", true)
     .build();

 InferResult result = client.infer(modelId, inputs, params);

Supported Parameter Types:

Since:
1.0.0
Author:
sachachoumiloff

  • Constructor Details

    • InferParametersBuilder

      public InferParametersBuilder()

  • Method Details

    • add

      public InferParametersBuilder add(String key, String value)
      Adds a string-valued parameter to the inference request.
      Parameters:
      key - the parameter name (must not be a reserved parameter)
      value - the string value for this parameter
      Returns:
      this builder instance for method chaining
      Throws:
      IllegalArgumentException - if the key is a reserved parameter name

    • add

      public InferParametersBuilder add(String key, long value)
      Adds a long integer-valued parameter to the inference request.
      Parameters:
      key - the parameter name (must not be a reserved parameter)
      value - the long integer value for this parameter
      Returns:
      this builder instance for method chaining
      Throws:
      IllegalArgumentException - if the key is a reserved parameter name

    • add

      public InferParametersBuilder add(String key, boolean value)
      Adds a boolean-valued parameter to the inference request.
      Parameters:
      key - the parameter name (must not be a reserved parameter)
      value - the boolean value for this parameter
      Returns:
      this builder instance for method chaining
      Throws:
      IllegalArgumentException - if the key is a reserved parameter name

    • add

      public InferParametersBuilder add(String key, double value)
      Adds a double-valued parameter to the inference request.
      Parameters:
      key - the parameter name (must not be a reserved parameter)
      value - the double (floating-point) value for this parameter
      Returns:
      this builder instance for method chaining
      Throws:
      IllegalArgumentException - if the key is a reserved parameter name

    • addInternal

      protected void addInternal(String key, GrpcService.InferParameter parameter)
      Adds a parameter to the builder without reserved key validation.

      This method is for internal use only and bypasses the reserved key check. Used internally to add reserved parameters that are managed by the SDK.

      Parameters:
      key - the parameter name
      parameter - the InferParameter protobuf message

    • build

      public Map<String,GrpcService.InferParameter> build()
      Builds and returns the accumulated parameters map.
      Returns:
      an unmodifiable map of parameter names to InferParameter values