[media] V4L: mx2-camera: avoid overflowing 32-bits
authorGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Tue, 8 May 2012 14:46:44 +0000 (11:46 -0300)
committerMauro Carvalho Chehab <mchehab@redhat.com>
Tue, 15 May 2012 19:14:01 +0000 (16:14 -0300)
commit67e86524b907ed2189f4875cdbe7a5249c71ebb2
tree024321e97d066467df8b25fe92d59b6f2d244b9e
parentfc13baff743cf9fa49035974471c17378bfe6146
[media] V4L: mx2-camera: avoid overflowing 32-bits

In mx2_camera_try_fmt(), when applying i.MX25 restrictions to frame sizes,
the height is checked to be <= 0xffff. But later an integer multiplication
height * 4 * 0x3ffff is performed, which will overflow even for bounded
height values. This patch switches to using 64-bit multiplication and
division to avoid overflowing.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
drivers/media/video/mx2_camera.c