Carnegie Mellon University scientists recently found that redesigning a cell phone display could make photographs better. It’s part of a growing revolution in phone camera technology. “Smartphones are already using computational photography, a digital image-capture method that employs AI-integrated software, digital computation, and powerful hardware—instead of relying solely on optical processes that are limited by the compact form factor of the phone,” Mario Endo, director of computer memory manufacturer Micron’s Mobile Business Unit, told Lifewire in an email interview. “This results in advanced-level photographs without pro gear or advanced editing tools.”
See Clearly
Most smartphones currently use cameras beneath the screen, but this design choice hurts picture clarity. The Carnegie Mellon researchers recently published a paper that shows how to redesign a cell phone display to make photographs clearer. “In an underpanel camera, the display blocks a large fraction of the light that a camera would normally receive, which leads to noisier images,” Aswin Sankaranarayanan, a professor of electrical and computer engineering at Carnegie Mellon and one of the paper’s authors, told Lifewire in an email interview. “This is additionally complicated by the camera having to image the scene through the mesh-like openings in the display, which causes a large blur due to an optical phenomenon called diffraction,” he added. “This blur makes the image less sharp and causes a flaring of bright sources, both of which lead to lower-quality images.” To solve this problem, the researchers propose putting the camera beneath the display. Openings in the display between the light-emitting diodes (LEDs) found at each display pixel allow the camera to image the world. Unlike the hole punch or the notch, the camera is completely hidden, so the display can be seamless. “Underpanel cameras help maximize display area by avoiding bezels or notches, and so they primarily help improve the aesthetics of the cell phone display,” Sankaranarayanan said. “However, once we move the camera below the display, we have the luxury of placing the camera in a more central location which would allow for natural eye contact in video calls and selfies.” He added that placing the camera beneath the display also leads to a large blur and loss of image quality. “The technology we have developed redesigns the display to reduce this blur and allow users to obtain images of higher quality,” he said.
Smarter Cameras
The use of computational software in addition to hardware to process pictures is transforming smartphone photography. Computational photography features can be found in phones such as the iPhone 12 Pro, Samsung Galaxy S21, Google Pixel 5, and Xiaomi Mi 11 Pro. As users tap on their camera button, adjustments to ISO, aperture, and shutter speed are made to capture the best image on the smartphone—powered by AI (artificial intelligence), which uses information in allocated memory to make fine adjustments, Endo said. “Most top-end phones have multiple lenses that switch mid-zoom without the user ever noticing, with the help of AI,” he added. “Some also have sensors large enough to take decent quality photos in low-light environments.” Endo pointed out that AI algorithms in recent smartphone cameras read the light, depth of field, and other factors to adjust settings automatically, instead of relying on an expensive digital camera and extensive training. “Composition, color saturation, and contrast are automatically tweaked using machine learning techniques to achieve the best results,” he said. “These cameras, guided by software, snap handfuls of images and select the best one, freeing users from having to hit the delete button over and over. Some of these advanced phones can also stack or combine multiple images to create the best photo.” Camera phone technology will soon get even better, Endo predicts. “We expect the technology to continue to progress, combined with hardware advancements such as wider aperture lenses and higher megapixel image sensors, to further improve on features such as higher quality low-light photos and zoom beyond 100x,” he said.
