Three follow-on defects from the 5-lane review of commit 16f74ab87c:
1. Zero-byte BO allocation (redox_drm_bo.c)
batch_bo_acquire() in redox_drm_cs.c passes a pipe_resource template
with format = 0 (PIPE_FORMAT_NONE). redox_resource_byte_count()
returned 0 for format = 0, producing a zero-byte GEM object. The
original commit message acknowledged this but didn't fix it.
Fix: in redox_resource_byte_count(), handle PIPE_BUFFER and format-NONE
specially. PIPE_BUFFER uses width0 as the raw byte count; size by
width0 * height0 * depth0 (which for buffers is just width0). This
is the root-cause fix — it also benefits every other PIPE_BUFFER
allocation, not just batch pools.
2. Unchecked mtx_init (redox_drm_winsys.c, .h)
(void) mtx_init(...) silently swallowed initialization errors,
leaving subsequent lock/unlock operations on an invalid mutex
(undefined behavior).
Fix: require thrd_success; on failure, return false from
redox_drm_initialize(). Add a batch_pool_mutex_initialized flag to
the winsys struct so redox_drm_destroy() only destroys an
initialized mutex. Also clean up the mutex on the cs_create failure
path. Initialize the flag and pool slots to safe defaults at the
top of redox_drm_initialize().
3. Oversized batch size truncation (redox_drm_cs.c)
pipe_resource::width0 is uint32_t. byte_count was stored as width0
without overflow checks; requests > UINT32_MAX would allocate a
truncated BO and then overflow on the memcpy.
Fix: in batch_bo_acquire(), return NULL when byte_count > UINT32_MAX
with a comment explaining the limit. No real batch exceeds 4 GiB.
Verified by static review; the C code cross-compiles only for the
Redox target (no host build).
vasilito
1ffc9299be