Archives for March 2020

Image Sensors World        Go to the original article...

EI 2020 paper "CAOS smart camera-based robust low contrast image recovery over 90 dB scene linear dynamic range" by Nabeel. A. Riza and Mohsin. A. Mazhar from University College Cork, Ireland reports:

"Experimentally demonstrated for the first time is Coded Access Optical Sensor (CAOS) camera empowered robust and true white light High Dynamic Range (HDR) scene low contrast target image recovery over the full linear dynamic range. The 90 dB linear HDR scene uses a 16 element custom designed test target with low contrast 6 dB step scaled irradiances. Such camera performance is highly sought after in catastrophic failure avoidance mission critical HDR scenarios with embedded low contrast targets."

Go to the original article...

Image Sensors World        Go to the original article...

Techinsights analysis of components inside Samsung latest flagship smartphone Galaxy S20 Ultra 5G estimates its cameras cost at over 20% out of the total $528.50:


Update: Techinsights analyst Ray Fontaine publishes a detailed overview of Galaxy S20 Ultra cameras:

Go to the original article...

Image Sensors World        Go to the original article...

Only presented on SK Hynix Korean-language site so far, the company's 1.0um 'Black Pearl' pixel technology is said to reach market in Q1 this year. Four new 1.0um Black Pearl CIS products will be introduced in the Q1 2020, ranging from 8MP to 20MP. Hynix also says that 0.8um Black Pearl pixel is in works and expected to appear in 48MP product in the second half of this year.

They feature Quad Pixel to resize the pixel area and the Q2B Quad to Bayer Remosaic algorithm, said to be more efficient than competing products:


The 16MP Hi-1634 and 20MP Hi-2021 of the new line-up are optimized for ultra wide-angle smartphone cameras, while 8MP Hi-847 and 13MP Hi-1337 are optimized for telephoto smartphone rear cameras. The Hi-847 mass production is said to begun in February and the Hi-1337 production will begin in March:


SK Hynix's CIS Marketing Strategy Cho Ho-young said, “SK Hynix plans to officially brand all CIS products as 'Black Pearl' from this year and continuously introduce products that can deliver the best value to customers in the CIS field as well as memory semiconductors. Like Black Pearl (black pearl), one of the rarest and most beautiful pearls in the world, we will make more efforts to have a position that fits the brand meaning in the CIS field.”

The average number of camera per smartphone is expected to grow to 3.9 this year, according to Hynix research:

Go to the original article...

Image Sensors World        Go to the original article...

DARPA announces the Invisible Headlights program. The fundamental research effort seeks to discover and quantify information contained in ambient thermal emissions in a wide variety of environments and to create new passive 3D sensors and algorithms to exploit that information.

“We’re aiming to make completely passive navigation in pitch dark conditions possible,” said Joe Altepeter, program manager in DARPA’s Defense Sciences Office. “In the depths of a cave or in the dark of a moonless, starless night with dense fog, current autonomous systems can’t make sense of the environment without radiating some signal—whether it’s a laser pulse, radar or visible light beam—all of which we want to avoid. If it involves emitting a signal, it’s not invisible for the sake of this program.”

The program includes three phases:

1. Discovery – to determine if thermal emissions contain sufficient information to enable autonomous driving at night or underground;
2. Optimization – to refine models, experimental designs, and ensure system feasibility for achieving 3D vision at both low speeds (less than 25 mph) and high speeds (more than 25 mph);
3. Advanced Prototypes – to build and test passive demonstration systems that compete with active sensors.

“If we’re successful, the capability of Invisible Headlights could extend the environments and types of missions in which autonomous assets can operate – at night, underground, in the arctic, and in fog,” Altepeter said. “The fundamental understanding of what information is available in ambient thermal emissions could lead to advances in other areas, such as chemical sensing, multispectral vision systems, and other applications that exploit infrared light.”


Thanks to AF for the pointer!

Go to the original article...

Image Sensors World        Go to the original article...

Waymo announces its 5th Generation Driver platform featuring a lot more cameras and LiDARs at half the cost of the previous generation:

"With each generation of our custom hardware we’ve been able to bring down the cost of our sensors while delivering even more capabilities and compute power. With our fifth-generation hardware, we’ve simplified the design and manufacturing process so it can be production-ready, and our latest sensors deliver more performance than ever before, at half the cost of our previous generation."

Go to the original article...

Image Sensors World        Go to the original article...

BusinessWire: Starting in March 2020, Infineon's REAL3 ToF sensor will enable the video bokeh function for the first time in a 5G-capable smartphone in moving images. Using the precise 3D point cloud algorithm and software, the received 3D image data is processed for the application. The 3D image sensor captures 940 nm infrared light reflected from the user and the scanned objects. It also uses high-level data processing to achieve accurate depth measurements.

Infineon also develops a reference design for 3D authentication based on the Qualcomm Snapdragon 865 Mobile Platform. "Today, the smartphone is more than just an information medium; it is increasingly taking over security and entertainment functions," says Andreas Urschitz, Division President Power Management & Multimarket. "3D sensors enable new uses and additional applications such as secured authentication or payment by facial recognition. We continue to focus on this market and have clear growth targets. The collaboration with Qualcomm Technologies on reference designs using REAL3 image sensors underscores the potential and our ambitions in this area."

Infineon develops the 3D ToF sensor technology in cooperation with the software and 3D time-of-flight system by pmdtechnologies AG.

Go to the original article...

Image Sensors World        Go to the original article...

IEEE Spectrum publishes an article "Image Sensor Doubles as a Neural Net" about TU Wien, Austria, work on integration of AI engine onto an image sensor:

"Our image sensor does not consume any electrical power when it is operating," Lukas Mennel, an electrical engineer at TU Wien, says. "The sensed photons themselves provide the energy for the electric current."

Mennel notes the speed at which the system operates is only limited by the speed of the electrons in the circuits. In principle, this strategy could work on the order of picoseconds, or trillionths of a second, or three to four orders of magnitude faster than currently demonstrated, he says.


Nature too publishes comments to that same paper on neural network processor integrated onto an image sensor by Vienna University of Technology, Austria:

"Imaging under dim light would be difficult for the device described by the authors. A redesign would be needed to improve light absorption in the thin semiconductor and to increase the range of light intensities that can be detected. Furthermore, the reported design requires high voltages and consumes a lot of power; by comparison, the energy consumption per operation in a biological neural network is at the sub-femtojoule level (10e−15 to 10e−13 joules). It would also be useful to expand the response to ultraviolet and infrared light, to capture information unavailable in the visible spectrum."

Go to the original article...

Image Sensors World        Go to the original article...

EI 2020 paper "A 4-tap global shutter pixel with enhanced IR sensitivity for VGA time-of-flight CMOS image sensors" by Taesub Jung, Yonghun Kwon, Sungyoung Seo, Min-Sun Keel, Changkeun Lee, Sung-Ho Choi, Sae-Young Kim, Sunghyuck Cho, Youngchan Kim, Young-Gu Jin, Moosup Lim, Hyunsurk Ryu, Yitae Kim, Joonseok Kim, and Chang-Rok Moon from Samsung, Korea presents the company's approach to 3D ToF imaging:

"An indirect time-of-flight (ToF) CMOS image sensor has been designed with 4-tap 7 µm global shutter pixel in back-side illumination process. 15000 e- of high full-well capacity (FWC) per a tap of 3.5 µm pitch and 3.6 e- of read-noise has been realized by employing true correlated double sampling (CDS) structure with storage gates (SGs). Noble characteristics such as 86 % of demodulation contrast (DC) at 100MHz operation, 37 % of higher quantum efficiency (QE) and lower parasitic light sensitivity (PLS) at 940 nm have been achieved. As a result, the proposed ToF sensor shows depth noise less than 0.3 % with 940 nm illuminator in even long distance."


Update: Possibly, the paper talks about DepthVision camera used in Galaxy S20 Ultra smartphone shown in Samsung video:


Update #2: According to Techinsights teardown analysis, the depth sensor in Galaxy S10 Ultra 5G is made by Sony.

Go to the original article...

Image Sensors World        Go to the original article...

Electronic Imaging Symposium started publishing 2020 papers. "Studies on the effect of megapixel sensor resolution on displayed image quality and relevant metrics" by Sophie Triantaphillidou, Jan Smejkal, Edward W. S. Fry, and Chuang Hsin Hung from University of Westminster, UK and Huawei, China examines just one specific case of viewing pictures at 1.2MP desktop monitor:

"This paper investigates camera phone image quality, namely the effect of sensor megapixel (MP) resolution on the perceived quality of images displayed at full size on high-quality desktop displays. For the purpose, we use images from simulated cameras with different sensor MP resolutions. We employ methods recommended in the IEEE 1858 Camera Phone Image Quality (CPIQ) standard, as well as other established psychophysical paradigms, to obtain subjective image quality ratings for systems with varying MP resolution from large numbers of observers."

The paper basically says that there is a little value in increasing smartphone sensor resolution beyond 1.5MP in this specific viewing case:

Go to the original article...

Image Sensors World        Go to the original article...

MDPI paper "Automotive 3.0 µm Pixel High Dynamic Range Sensor with LED Flicker Mitigation" by Minseok Oh, Sergey Velichko, Scott Johnson, Michael Guidash, Hung-Chih Chang, Daniel Tekleab, Bob Gravelle, Steve Nicholes, Maheedhar Suryadevara, Dave Collins, Rick Mauritzson, Lin Lin, Shaheen Amanullah, and Manuel Innocent is a part of Special issue on the 2019 International Image Sensor Workshop (IISW2019).

"We present and discuss parameters of a high dynamic range (HDR) image sensor with LED flicker mitigation (LFM) operating in automotive temperature range. The total SNR (SNR including dark fixed pattern noise), of the sensor is degraded by floating diffusion (FD) dark current (DC) and dark signal non-uniformity (DSNU). We present results of FD DC and DSNU reduction, to provide required SNR versus signal level at temperatures up to 120 °C. Additionally we discuss temperature dependencies of quantum efficiency (QE), sensitivity, color effects, and other pixel parameters for backside illuminated image sensors. Comparing +120 °C junction vs. room temperature, in visual range we measured a few relative percent increase, while in 940 nm band range we measured 1.46x increase in sensitivity. Measured change of sensitivity for visual bands—such as blue, green, and red colors—reflected some impact to captured image color accuracy that created slight image color tint at high temperature. The tint is, however, hard to detect visually and may be removed by auto white balancing and temperature adjusted color correction matrixes."

Go to the original article...

Image Sensors World        Go to the original article...

Caltech researchers publish Arxiv.org paper "Multi-functional volumetric meta-optics for color and polarization image sensors" by Philip Camayd-Muñoz, Conner Ballew, Gregory Roberts, and Andrei Faraon.

"Three-dimensional elements, with refractive index distribution structured at sub-wavelength scale, provide an expansive optical design space that can be harnessed for demonstrating multi-functional free-space optical devices. Here we present 3D dielectric elements, designed to be placed on top of the pixels of image sensors, that sort and focus light based on its color and polarization with efficiency significantly surpassing 2D absorptive and diffractive filters. The devices are designed via iterative gradient-based optimization to account for multiple target functions while ensuring compatibility with existing nanofabrication processes, and experimentally validated using a scaled device that operates at microwave frequencies. This approach combines arbitrary functions into a single compact element even where there is no known equivalent in bulk optics, enabling novel integrated photonic applications."

Go to the original article...

Image Sensors World        Go to the original article...

Goodix under-display optical fingerprint sensor has been selected as a finalist for Edison Award 2020 in Optical & Imaging Technologies category.


Daum blog publishes some explanations of Goodix technology and a comparison with Egis:

Go to the original article...

Image Sensors World        Go to the original article...

EETimes article "Beam Profile Analysis Beats Facial Recognition Cheats" talks about TrinamiX facial recognition sensor:

In a recent briefing with EE Times in London, the managing director and founder of TrinamiX, Ingmar Bruder, told, “There are currently basically two ways of capturing depth: time-of-flight (ToF), and stereo, including structured light. We have figured out a third way, called beam profile analysis, for which we have several patents and own all the intellectual property. Our technology is based on back-scattering properties, creating a dense depth map using 40,000 dots for which we can measure the distance for each. Our technology is based on three components: a NIR camera module, flood illumination, and a light projector. This can sit on top of any facial recognition capability.”

“You are training the system based on the actual materials. In robot picking, you don’t need to pre-define the boundaries, as the system can look for the material and the boundaries. It can also sort materials, for example in recycling,” Bruder said. “One of the first products we are likely to see based on this technology is robot vacuum cleaners,” he said. “This is in evaluation at the moment, but products are likely on the market in 2021. Meanwhile, in consumer products, we will see both high-end and low-end smartphones using beam profile analysis in early 2021.”


Update: Sueddeutsche Zeitung report talks about the problem of silicone masks in face recognition.

Go to the original article...

Image Sensors World        Go to the original article...

GlobeNewswire: SRI International announces that they were recognized as an Edison Award finalist in the Aerospace Innovation category for their contributions to NASA’s Solar Probe Plus Mission.

The CMOS Imager used in the Parker Solar Probe was designed by Jim Janesick of SRI, a leading expert in CCD and CMOS development. The technology was conceptualized with extraterrestrial applications in mind and was supported by the Naval Research Laboratory (NRL). SRI and the NRL collaborated to create a 2k x 2k Imager that could withstand the radiation emitted by the sun, which was then selected by NASA for inclusion on the Parker Solar Probe mission, among others, where it will capture the closest images of the sun.

Go to the original article...

Image Sensors World        Go to the original article...

Photonis presents Mantis³ single photon counting camera with nanosecond time-stamping. The Mantis³ consists of a CERN Timepix3-based visible light camera coupled to one of Photonis' high-end image intensifier tubes. The TPX3Cam is a high-rate, event-driven, time-stamping camera. The coupled image intensifier enables single photon sensitivity and offers a choice of low noise photocathodes optimized for your application.


With image intensifier tube, the 62e- noise floor of Timepix3 chip is reduced to a level allowing single-photon detection:

Go to the original article...

Image Sensors World        Go to the original article...

The US Court of Appeals publishes its opinion on Collabo Innovation appeal on the verdict in its lawsuit against Sony. The appeal claims mostly talk about 1999 US7,023,034 patent originally filed by Panasonic and later acquired by Collabo. The appeal argues about the definition of trapezoid shape of the lightpipe, as presented in prior art.


The court concludes "We have considered Collabo’s remaining arguments and find them unpersuasive. For the foregoing reasons, we affirm the Board’s construction of “reflecting wall” and its conclusion that claims 3 and 12 are unpatentable as obvious."

Go to the original article...

Image Sensors World        Go to the original article...

Nature publishes KAIST paper "Biologically inspired ultrathin arrayed camera for high-contrast and high-resolution imaging" by Kisoo Kim, Kyung-Won Jang, Jae-Kwan Ryu, and Ki-Hun Jeong.

"Compound eyes found in insects provide intriguing sources of biological inspiration for miniaturised imaging systems. Here, we report an ultrathin arrayed camera inspired by insect eye structures for high-contrast and super-resolution imaging. The ultrathin camera features micro-optical elements (MOEs), i.e., inverted microlenses, multilayered pinhole arrays, and gap spacers on an image sensor. The MOE was fabricated by using repeated photolithography and thermal reflow. The fully packaged camera shows a total track length of 740 μm and a field-of-view (FOV) of 73°. The experimental results demonstrate that the multilayered pinhole of the MOE allows high-contrast imaging by eliminating the optical crosstalk between microlenses. The integral image reconstructed from array images clearly increases the modulation transfer function (MTF) by ~1.57 times compared to that of a single channel image in the ultrathin camera. This ultrathin arrayed camera provides a novel and practical direction for diverse mobile, surveillance or medical applications."

Go to the original article...