Struct GlobalDeviceBudget 

pub struct GlobalDeviceBudget { /* private fields */ }

Per-runtime byte-limit decorator.

Implementations

impl GlobalDeviceBudget

pub fn new( inner: Box<dyn DeviceMemoryResource + Send + Sync>, limit: usize, ) -> Self

Wrap inner with a hard limit in bytes. The initial reserved tally and release baseline are sampled atomically from the inner resource’s canonical accounting ledger, so callers may compose around an inner that already has live allocations — though in practice the decorator is installed before any allocation flows through it.

pub fn limit(&self) -> usize

Hard byte limit. Set at construction; not adjustable.

pub fn reserved_bytes(&self) -> usize

Bytes currently reserved against the budget (live + pending async free). Matches inner.bytes_outstanding() at every quiescent moment.

pub fn remaining(&self) -> usize

Headroom in bytes for the next allocation. Equal to limit - reserved_bytes, saturating at zero.

Trait Implementations

impl DeviceMemoryResource for GlobalDeviceBudget

fn materialize(&self, request: AllocationRequest) -> ResourceResult<DeviceBlock>

Materialize one request. Decorators must forward the same request.

fn allocation_accounting(&self) -> Arc<AllocationAccounting>

Exact terminal-backend accounting domain. Decorators return the identical Arc. Later allocations must pass through the domain’s installed budgets.

fn deallocate(&self, block: DeviceBlock) -> ResourceResult<()>

Return block to the resource. After this call the block’s state is BlockState::Retired (or BlockState::Quarantined for debug-guard resources). Reuse of the underlying memory is resource-specific but must respect the stream-ordered contract.

fn device_ordinal(&self) -> u32

CUDA device ordinal this resource serves. Resources are pinned to a single device.

fn bytes_outstanding(&self) -> usize

Bytes currently outstanding (live + retired-but-not-yet-freed). Used by tests and by the global budget adaptor.

fn budget_snapshot(&self) -> Option<ResourceBudgetSnapshot>

Return the outer resource stack’s reservable budget, if one exists. Base allocators have no declarative admission limit and return None; decorators must forward this query unless they enforce their own limit.

fn reap_pending(&self) -> ResourceResult<()>

Drain any retired-but-not-yet-freed bytes whose underlying CUDA work has completed. For synchronous backends this is a no-op. For stream-ordered async backends this synchronizes the streams that have queued cuMemFreeAsync calls and re-counts bytes_outstanding accordingly.

fn record_block_use( &self, block: &DeviceBlock, use_stream: StreamId, ) -> ResourceResult<()>

Record that work has been (or is being) submitted on use_stream that touches block’s bytes. Resources that participate in cross-stream lifetime tracking (notably the stream-ordered async backend) MUST attach a CUDA event from use_stream to the block; on deallocate(block), the block’s alloc_stream will wait on every recorded event before queueing the underlying free.

fn supports_block_use_tracking(&self) -> bool

Whether this resource (and any inner resources it composes) actually tracks cross-stream uses via record_block_use. Used by the launch recorder’s preflight to fail BEFORE queueing CUDA work, rather than after. The default returns false to match the trait’s default record_block_use behavior; resources that override record_block_use to track events MUST override this to return true. Decorators forward to inner.

fn prepare_block_use( &self, block: BlockId, use_stream: StreamId, access: Access, ) -> ResourceResult<()>

Pre-launch / pre-copy hook: queue any cross-stream waits required for use_stream to safely access block with access semantics. MUST be called BEFORE the GPU work is enqueued on use_stream.

fn finish_block_use( &self, block: BlockId, use_stream: StreamId, access: Access, ) -> ResourceResult<()>

Post-launch / post-copy hook: record an event on use_stream capturing the work just enqueued and update block’s dependency state.

fn allocate( &self, bytes: usize, stream: StreamId, tag: AllocTag, ) -> ResourceResult<DeviceBlock>

Allocate bytes bytes on the resource’s device, ordered on stream. The returned block is in BlockState::Live.

fn allocate_with_reservation_pressure( &self, bytes: usize, reservation_pressure_bytes: usize, stream: StreamId, tag: AllocTag, ) -> ResourceResult<DeviceBlock>

Allocate while accounting for bytes promised by the runtime but not yet materialized through this resource stack.