drm: fix drm_mode_addfb() on big endian machines.
authorGerd Hoffmann <kraxel@redhat.com>
Wed, 5 Sep 2018 06:04:43 +0000 (08:04 +0200)
committerGerd Hoffmann <kraxel@redhat.com>
Thu, 6 Sep 2018 06:40:19 +0000 (08:40 +0200)
commit6960e6da9cec3f6638121527c728305827ec12ab
treef962fb5c76fe4059b8129fbebdb347226b378093
parent00409fd6f14f1a3f06c6ea50a6f2fba17c5573d9
drm: fix drm_mode_addfb() on big endian machines.

Userspace on big endian machhines typically expects the ADDFB ioctl
returns a big endian framebuffer.  drm_mode_addfb() will call
drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
values though, which is wrong.  This patch fixes that.

Drivers (both kernel and xorg) have quirks in place to deal with the
broken drm_mode_addfb() behavior.  Because of this we can't just change
drm_mode_addfb() behavior for everybody without breaking things.  Add
the quirk_addfb_prefer_host_byte_order field to mode_config, so drivers
can opt-in.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20180905060445.15008-5-kraxel@redhat.com
drivers/gpu/drm/drm_framebuffer.c
include/drm/drm_mode_config.h