Package com.gencior.triton.core.pojo

Class TritonMemoryUsage

java.lang.Object
com.gencior.triton.core.pojo.TritonMemoryUsage
public final class TritonMemoryUsage extends Object
Encapsulates memory usage information for a specific memory device on the Triton server.

This class represents memory usage statistics for a particular memory device, such as GPU memory (e.g., NVIDIA GPU VRAM) or CPU system memory. It tracks the total amount of memory currently allocated or reserved on a specific memory device identified by its ID.

This is an immutable object that wraps the gRPC message MemoryUsage.

Memory Types:

  • gpu: GPU device memory (e.g., CUDA device memory)
  • cpu: CPU system memory

Usage Example:


 List<TritonMemoryUsage> memoryUsages = modelStatistics.getMemoryUsage();
 for (TritonMemoryUsage usage : memoryUsages) {
     double memoryMb = usage.getByteSize() / (1024.0 * 1024.0);
     System.out.printf("%s device %d: %.2f MB%n", usage.getType(), usage.getId(), memoryMb);
 }
Since:
1.0.0
Author:
sachachoumiloff

  • Method Details

    • fromProto

      public static TritonMemoryUsage fromProto(GrpcService.MemoryUsage proto)
      Creates a TritonMemoryUsage from a gRPC MemoryUsage message.
      Parameters:
      proto - the gRPC MemoryUsage message from Triton server
      Returns:
      a new TritonMemoryUsage instance

    • getType

      public String getType()
      Returns the type of memory device.

      Typical values are "gpu" for GPU memory or "cpu" for CPU memory.

      Returns:
      the memory type (e.g., "gpu", "cpu")

    • getId

      public long getId()
      Returns the ID of the memory device.

      For GPUs, this is typically the CUDA device ID (0, 1, 2, etc.). For CPU memory, this is typically 0.

      Returns:
      the device ID

    • getByteSize

      public long getByteSize()
      Returns the amount of memory used by this device in bytes.
      Returns:
      the memory size in bytes