Trait DeviceMemoryResource 

pub trait DeviceMemoryResource: Send + Sync {
    // Required methods
    fn materialize(
        &self,
        request: AllocationRequest,
    ) -> ResourceResult<DeviceBlock>;
    fn allocation_accounting(&self) -> Arc<AllocationAccounting> ;
    fn deallocate(&self, block: DeviceBlock) -> ResourceResult<()>;
    fn device_ordinal(&self) -> u32;
    fn bytes_outstanding(&self) -> usize;

    // Provided methods
    fn allocate(
        &self,
        bytes: usize,
        stream: StreamId,
        tag: AllocTag,
    ) -> ResourceResult<DeviceBlock> { ... }
    fn allocate_with_reservation_pressure(
        &self,
        bytes: usize,
        reservation_pressure_bytes: usize,
        stream: StreamId,
        tag: AllocTag,
    ) -> ResourceResult<DeviceBlock> { ... }
    fn budget_snapshot(&self) -> Option<ResourceBudgetSnapshot> { ... }
    fn reap_pending(&self) -> ResourceResult<()> { ... }
    fn record_block_use(
        &self,
        block: &DeviceBlock,
        use_stream: StreamId,
    ) -> ResourceResult<()> { ... }
    fn supports_block_use_tracking(&self) -> bool { ... }
    fn prepare_block_use(
        &self,
        block: BlockId,
        use_stream: StreamId,
        access: Access,
    ) -> ResourceResult<()> { ... }
    fn finish_block_use(
        &self,
        block: BlockId,
        use_stream: StreamId,
        access: Access,
    ) -> ResourceResult<()> { ... }
}

Stream-ordered device memory resource. Implementations:

• crate::device_runtime::direct::DirectCudaResource — cudarc default (non-pooled) backend; candidate for the sanitizer/cert role, unproven until the manual Compute Sanitizer acceptance gate runs on a supported host.
• crate::device_runtime::async_resource::AsyncCudaResource — stream-ordered cuMemAllocAsync/cuMemFreeAsync backend; production default when the context supports async-alloc.
• crate::device_runtime::logging::LoggingResource — telemetry decorator over any inner resource.
• crate::device_runtime::budget::GlobalDeviceBudget — per-runtime byte-limit decorator over any inner resource.

Implementations must be thread-safe. The runtime composes resources via decoration (each resource wraps an inner Box<dyn DeviceMemoryResource + Send + Sync>).

Required Methods

fn materialize(&self, request: AllocationRequest) -> ResourceResult<DeviceBlock>

Materialize one request. Decorators must forward the same request.

The terminal backend binds its ledger with attach_resource before malloc, retains the ticket with its storage owner, and calls acquired immediately after arming the actual pointer. Failed initialization must still reach physical reclamation. Only proven physical release permits unsafe { ticket.confirm_physical_release() }; an error or unwind alone never proves storage was released.

Custom implementations must implement this lifecycle-bearing entry point; ordinary caller-facing allocation signatures remain conveniences below.

fn allocation_accounting(&self) -> Arc<AllocationAccounting>

Exact terminal-backend accounting domain. Decorators return the identical Arc. Later allocations must pass through the domain’s installed budgets.

fn deallocate(&self, block: DeviceBlock) -> ResourceResult<()>

Return block to the resource. After this call the block’s state is BlockState::Retired (or BlockState::Quarantined for debug-guard resources). Reuse of the underlying memory is resource-specific but must respect the stream-ordered contract.

block.alloc_stream is authoritative for ordering. If the caller has touched the memory on a different stream, they must have synchronized before calling deallocate.

fn device_ordinal(&self) -> u32

CUDA device ordinal this resource serves. Resources are pinned to a single device.

fn bytes_outstanding(&self) -> usize

Bytes currently outstanding (live + retired-but-not-yet-freed). Used by tests and by the global budget adaptor.

Provided Methods

fn allocate( &self, bytes: usize, stream: StreamId, tag: AllocTag, ) -> ResourceResult<DeviceBlock>

Allocate bytes bytes on the resource’s device, ordered on stream. The returned block is in BlockState::Live.

fn allocate_with_reservation_pressure( &self, bytes: usize, reservation_pressure_bytes: usize, stream: StreamId, tag: AllocTag, ) -> ResourceResult<DeviceBlock>

Allocate while accounting for bytes promised by the runtime but not yet materialized through this resource stack.

Budget decorators must include the reservation pressure in their admission decision. Telemetry decorators must forward this call so they observe both successful allocations and admission failures. All layers forward the same lifecycle-bearing request to materialize.

fn budget_snapshot(&self) -> Option<ResourceBudgetSnapshot>

Return the outer resource stack’s reservable budget, if one exists. Base allocators have no declarative admission limit and return None; decorators must forward this query unless they enforce their own limit.

fn reap_pending(&self) -> ResourceResult<()>

Drain any retired-but-not-yet-freed bytes whose underlying CUDA work has completed. For synchronous backends this is a no-op. For stream-ordered async backends this synchronizes the streams that have queued cuMemFreeAsync calls and re-counts bytes_outstanding accordingly.

Callers that need an accurate budget reading after a burst of asynchronous deallocations should call this before reading bytes_outstanding. Calling on a synchronous backend is harmless and free.

fn record_block_use( &self, block: &DeviceBlock, use_stream: StreamId, ) -> ResourceResult<()>

Record that work has been (or is being) submitted on use_stream that touches block’s bytes. Resources that participate in cross-stream lifetime tracking (notably the stream-ordered async backend) MUST attach a CUDA event from use_stream to the block; on deallocate(block), the block’s alloc_stream will wait on every recorded event before queueing the underlying free.

The default implementation returns ResourceError::StreamMisuse. This is intentional: a silent no-op default would let a launch builder call record_block_use against a resource that does not actually track cross-stream uses (e.g., crate::device_runtime::direct::DirectCudaResource), observe Ok(()), queue a kernel on a different stream, then drop the block — and quietly hit the cross-stream use-after-free that this API exists to prevent. False safety is worse than no safety. Resources that cannot track cross-stream uses MUST inherit this default; callers (notably the future xlog launch builder) MUST surface the error rather than masking it.

Override status today:

• crate::device_runtime::async_resource::AsyncCudaResource overrides with real event tracking.
• crate::device_runtime::logging::LoggingResource and crate::device_runtime::budget::GlobalDeviceBudget forward to their inner resource (so the underlying backend’s behavior surfaces unchanged).
• crate::device_runtime::direct::DirectCudaResource does NOT override — it correctly returns StreamMisuse and forces callers to either route allocations through AsyncCudaResource or take responsibility for cross-stream synchronization themselves.

Errors

• ResourceError::StreamMisuse from the default impl when the resource cannot track cross-stream uses.
• ResourceError::UseAfterFree if block is not the block currently live at block.ptr (caller likely handed back a stale DeviceBlock whose generation no longer matches the live entry).
• ResourceError::StreamMisuse if use_stream does not resolve in the resource’s stream pool.
• ResourceError::Driver for CUDA driver / event creation failures.

Callers that bypass this API and submit cross-stream work directly (raw cuMemcpyDtoHAsync, raw Vec<*mut c_void> kernel launches that the launch builder did not see, etc.) are responsible for their own cross-stream synchronization. The resource cannot infer arbitrary external CUDA work.

fn supports_block_use_tracking(&self) -> bool

Whether this resource (and any inner resources it composes) actually tracks cross-stream uses via record_block_use. Used by the launch recorder’s preflight to fail BEFORE queueing CUDA work, rather than after. The default returns false to match the trait’s default record_block_use behavior; resources that override record_block_use to track events MUST override this to return true. Decorators forward to inner.

fn prepare_block_use( &self, block: BlockId, use_stream: StreamId, access: Access, ) -> ResourceResult<()>

Pre-launch / pre-copy hook: queue any cross-stream waits required for use_stream to safely access block with access semantics. MUST be called BEFORE the GPU work is enqueued on use_stream.

On Access::Read the resource must queue waits on every outstanding write from a different stream. On Access::Write / Access::ReadWrite the resource must additionally queue waits on every outstanding read recorded on a different stream — the writer must observe completion of every prior reader. Same-stream events are skipped (CUDA stream order already covers them).

The default implementation returns ResourceError::StreamMisuse. Same rationale as record_block_use: a silent no-op default would let callers paired against a non-tracking backend believe the dependency edge was queued. Decorators forward; tracking backends override.

Errors

• ResourceError::StreamMisuse from the default impl when the resource cannot track cross-stream uses.
• ResourceError::UseAfterFree if block is not the id currently live at block.ptr.
• ResourceError::Driver for CUDA driver / event-wait failures.

fn finish_block_use( &self, block: BlockId, use_stream: StreamId, access: Access, ) -> ResourceResult<()>

Post-launch / post-copy hook: record an event on use_stream capturing the work just enqueued and update block’s dependency state.

Advance the recording stream’s read or write frontier. Publication must preserve other streams’ concurrent completions: disjoint ranges of one allocation can prepare before either operation publishes. Only events whose ordering is proved may be retired. Same-stream record and frontier replacement must be serialized so an older host publisher cannot erase a newer completion.

The default implementation returns ResourceError::StreamMisuse. Same rationale as record_block_use. Decorators forward; tracking backends override.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl DeviceMemoryResource for AsyncCudaResource

impl DeviceMemoryResource for DirectCudaResource

impl DeviceMemoryResource for GlobalDeviceBudget

impl DeviceMemoryResource for LoggingResource