drm/dumb-buffers: Integer overflow in drm_mode_create_ioctl()
There is a comment here which says that DIV_ROUND_UP() and that's where
the problem comes from. Say you pick:
args->bpp = UINT_MAX - 7;
args->width = 4;
args->height = 1;
The integer overflow in DIV_ROUND_UP() means "cpp" is UINT_MAX / 8 and
because of how we picked args->width that means cpp < UINT_MAX / 4.
I've fixed it by preventing the integer overflow in DIV_ROUND_UP(). I
removed the check for !cpp because it's not possible after this change.
I also changed all the 0xffffffffU references to U32_MAX.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20180516140026.GA19340@mwanda