Trends in Mobile Imaging

Sam Beal Internet Marketing Manager

Introduction

There are four key elements to mobile imaging:

  • Media Processing
  • Display Technology
  • Content
  • Interface protocols between processor ICs and image controller IC’s

Today’s Application Processors for Smartphone and Tablet have substantial graphics processing capability. Most are fabricated in 28m high K metal gate CMOS nodes and the next generation of sub 20nm FINFETs will provide even more processing capability. Display Technology for LED and AMOLED ranges from 200 PPI for large form factors to 440PPI for small form factors. MIPI Interface standards dominate mobile electronics. Content is application and market segment driven. We’ll explore those in more detail below.

Smartphone Displays

HowToWatch4KSmartphone Steve Jobs introduced the 326 PPI “retina display” of the iPhone 4 with the claim that it exceeded the discrimination of the human retina. According to Dr. Raymond Soneira of DisplayMate Technologies this was marketing hyperbole. He calculated that the maximum discernible PPI at 12 inches from your eyes is 477 pixels per inch. At 18 inches it falls to 318 PPI. I think most people tend closer to 18 inches so Mr. Jobs was basically correct. Perhaps we will be able to identify 4K Smartphones by their owner’s viewing style. Advanced high resolution requires improved transistor density (and adequate dark space) and lower RC delays to reduce refresh times. Two transistor technology candidates, are amorphous oxide semiconductors and low temperature polysilicon. Faster refresh is required to improve motion blur. LCD TVs with 120Hz refresh are common and 240Hz is emerging. The same improvements will emerge in portable displays. Motion estimation and motion compensation (ME/MC) techniques use frame insertion to improve motion effects in action video and 3D gaming.[1]

Professional Tablets

Who will pay for more resolution than a digital movie requires? How about architects, general contractors, and a wide range of scientists and medical professionals.

High Resolution Content

ORPM Cloud based telemedicine, i.e., image processing in the cloud for display on mobile devices, will play a key role in improving and distributing diagnosis. John Gore, director of Vanderbilt University Institute of Imaging Science (VUIIS) is researching medical imaging that produces extreme close-up views of the details of tissues, revealing the dynamics of cellular and molecular biology.[2] Smartphones could be ideal for stereoscopic 3D technology allowing capture, playback and gaming. Codecs such as multiview H.264 (MVC), stereo JPEG (JPS) and JPEG Multi-Picture Object (MPO) are a critical element. OpenGL is used for gaming. 3D is achieved with dual images offset in color and phase, effectively doubling the resolution processing for a given display. Video frame rates are typically doubled to remove visual jitter. [3]

Image Interface Technology

The MIPI Alliance is a standards setting body with over 240 contributing member companies from the mobile platform, semiconductor and IP industries.Image display is supported by MIPI DSI with D-PHY connectivity. D-PHY allows power efficient, EMI reduced, serial communication between a CSI-2 or DSI host/device pair. 4 lanes of 1.5Gbps D-PHY supports displays of up to 2.5Kx2K resolution. A potential upgrade to M-PHY (DSI-2) could boost support to 4K.

Camera Interface Technology

Image capture is supported by the CSI-2 and D-PHY combination. Image data captured by the camera sensor should be presented to the CSI-2 Transmitter in RAW, RGB or YUV formats; the MIPI spec lists all the detailed formats that the CSI-2 connectivity infrastructure is required to support. The CSI-2 sensor interface provides you the option to compress the RAW data, and converts any kind of pixel data to bytes, which are then packetized and distributed over one or more lanes of the D-PHY. The higher the resolution of the captured image, the more speed and/or number of lanes needed. Given the max D-PHY throughput of 1.5Gbps per lane, with a maximum of 4 data lanes, the maximum camera resolution supported, assuming 24 bits/pixel in RGB format, with 30 frames/second is 8 megapixels. The proposed CSI-3 standard with MIPI M-PHY at Gear 3 (6.0Gbps) could support 60 fps.

What does the future hold

The trend to higher resolutions both in image capture and image display will continue. Will we ever see the quality of a RED EPIC Camera in a smartphone? Seems unlikely, but with stereoscopic multi-camera imaging and processing, who knows. One challenge that will not dissipate is the power required to transmit raw bandwidth between components in a system. From the tables above there is a level of resolution that will only be supported by compressed bit streams. Cameras will compress images on the fly and displays will decompress frames to buy 2 to 3 orders of headroom.

Technical References

  1. State of the art technologies and future prospective in display industry Moon, Joo-Tae, Samsung Display Labs, (IEDM), 2012
  2. Medical Imaging: Just What the Doctor (and the Researcher) Ordered: New Applications for Medical Imaging Technology, Mertz, L., Pulse 2013, IEEE
  3. Embedded stereoscopic 3D camera system for mobile platforms Aguirre, A.; Batur, A.U.; Hewes, G.; Pekkucuksen, I.; Venkatraman, N.; Ware, F.; Buyue Zhang, ICASSP 2012

Written by Related Link  Mobile Imaging Whitepaper 

Related Posts