Image Sensors World        Go to the original article...

EETimes quotes Marc Pollefeys, Microsoft director of science who runs a computer vision lab at ETH Zurich:

“Most of the energy is spent moving bits around … so it would seem natural that … the first layers of processing should happen in the sensor,” Pollefeys told EE Times in a brief interview... “I’m following the neuromorphic work that promises very power-efficient systems with layers of processing in the sensor — that’s a direction where we need a lot of innovation — it’s the only way to get a device that’s not heavier than glasses and can do tracking all day.”

Researchers are still working on ways to map a user’s hands accurately into an environment so that they can be used to control virtual objects. Occlusions, segmentation failures, and noisy data have hindered such efforts for years, Pollefeys said."

MS Hololens Block Diagram

Go to the original article...

Image Sensors World        Go to the original article...

TechInsights publishes reverse engineering report of Sony IMX400 3-layer stacked image sensor with integrated DRAM, extracted from Xperia XZs phone. According to Sony ISSCC 2017 paper, the pixel array is in the top die, the DRAM array and row drivers are in the middle die, and the remaining blocks are in the bottom ISP die:


"Given that the DRAM die also has the CIS row drivers on it, then it must have been designed as a custom part, and is not one of the TSV-enabled (TSV = through-silicon via) commodity DRAMs that we have seen in recent years. Sony’s cross-section also shows that the center die has a thick back oxide and landing pad for TSVs coming down from the CIS above.

We can also see (if the scale bar is accurate), that the CIS and DRAM die substrates have been thinned down to ~2.6 µm, normal for a back-illuminated CIS (BI-CIS), but that’s the thinnest DRAM we have ever seen. Our own image (above) confirms that the CIS and DRAM silicon are of the same order in thickness, and the landing pads are also visible.

It seems likely that the CIS/ISP connection could use the DRAM landing pad layer as an interconnect, to avoid the challenge of drilling through two dies after the full stack was formed."


Thanks to RF for the info!

Go to the original article...

Image Sensors World        Go to the original article...

SPIETV publishes Quanergy CEO Louay Eldada presentation:

Go to the original article...

Image Sensors World        Go to the original article...

Autosens Detroit 2017 conference offers quite significant image sensor content:

• IEEE P2020 Working Group Meeting - Official meeting for the IEEE Standards Association Working Group on Automotive System Image Quality – P2020
• Challenges in automotive image quality testing, Norman Koren, Imatest
• Optimizing Imaging and Vision Systems for Autonomous Driving, Felix Heide, Algolux
• Image Processing, Gregory Rofett, ST Micro
• Image processing challenges for self-driving cars, Joyce Farrell, Stanford University
• Image quality and safety in automotive video applications, Marc Geese, Bosch
• Toward visible-light-based imaging for autonomous vehicles, Guy Satat, MIT
• Improving Image Quality through Camera Radiometric Calibration, Mary Pagnutti, Innovative Imaging and Research
• Novel, affordable automotive lidar solution, Filip Geuens, XenomatiX

Go to the original article...

Image Sensors World        Go to the original article...

Korea-based eWBM publishes more info on its DR1152 dual aperture depth processor announced last year:

Go to the original article...

Image Sensors World        Go to the original article...

Qualcomm publishes a page about its previously reported always-on vision camera module and its use-cases:


"Operating at less than 2mW of end-to-end power, and expected to be sold at low cost, the CVM provides smartphones and IoT devices with affordable, always-on computer vision awareness. By emitting CV data about what’s happening in a field of view rather than transmitting images, the CVM also delivers a much more privacy sensitive vision solution."

Face Detection for Smartphones
"For smartphones, it provides passive, always-on face detection that can auto wake the device when a face is detected and auto sleep (save power) when a face is not detected. The CVM can also auto stop the screen brightness for as long as a face is detected. Additionally, the CVM also supports intelligent screen orientation where, upon face detection, it adjusts and holds the screen orientation based on the position of the user's face. The always-on CVM can also trigger third party applications and hardware. For example, it could enable a biometric-type iris authentication process to be initiated when a face is detected or a QR scanner could be initiated when a QR code is detected.

Beyond face detection, simple gestures can wake and trigger the smartphone handset. Lastly, the CVM can provide ambient light sensing (ALS) and proximity functionality."

Interactivity Trigger for Toys and Smart Appliances
"For example, it could enable a smart refrigerator door to turn transparent and light up when it detects that a user is interested in seeing what's inside, or it can enable an appliance screen to present a menu of options when it sees a user is interested in engaging with it."

Occupancy Trigger (OT) for Smart Home Devices
"With a smaller form factor than passive infrared (PIR) sensors, and at a very low cost and power, the CVM will more accurately identify humans in a field of view, and track their specific motion, noting whether they are moving toward the device or perpendicular to the device. It also discriminates between them and other moving objects, such as vehicles and pets, which normally trigger PIR sensors. The OT also provides associated metadata about the humans, reporting on variables such as location and the number of people within the sensor's field of view. Lastly, the highly flexible OT is trainable for varied types of object detection such as vehicles or pets using machine learning approaches."

Standalone Data Tracker (SDT)
"Our CVM is being engineered with a coin cell battery and Bluetooth radio into a stamp-sized SDT that can be used in commercial, residential, and smart city applications."

Go to the original article...

Image Sensors World        Go to the original article...

PRNewswire: Cadence unveils the Tensilica Vision C5 DSP, its first neural network DSP IP core for vision, radar/lidar and fused-sensor applications. Camera-based vision systems in automobiles, drones and security systems require two types of vision-optimized computation. First, the input from the camera is enhanced using traditional computational photography/imaging algorithms. Second, neural-network-based recognition algorithms perform object detection and recognition. Existing neural network accelerator solutions are hardware accelerators attached to imaging DSPs, with the neural network code split between running some network layers on the DSP and offloading convolutional layers to the accelerator. This combination is inefficient and consumes unnecessary power.

Architected as a dedicated neural-network-optimized DSP, the Vision C5 DSP accelerates all neural network computational layers (convolution, fully connected, pooling and normalization), not just the convolution functions. This frees up the main vision/imaging DSP to run image enhancement applications independently while the Vision C5 DSP runs inference tasks.


EETimes writes "The [Cadence Tensilica C5] cores are among as many as 50 silicon products now available to run various forms of computer vision and machine-learning tasks, said Jeff Bier, founder of the Embedded Vision Alliance, chairman of its event this week, and president of consulting firm BDTI. “There are so many [chips], with new ones popping up weekly, that it’s difficult to get a reliable count.”

Go to the original article...

Image Sensors World        Go to the original article...

EETimes: Renesas Autonomy platform for ADAS application features ISP and vision processors:


Key Features of the Renesas' R-Car V3M SoC solution:

• Efficient image recognition engine and functional safety
  The R-Car V3M SoC implements a computer vision platform using different accelerators, including a versatile pipeline engine (IMP) and a computer vision engine (CVE), allowing the R-Car V3M to manage algorithms like optical flow, object detection and classification and convolutional neural networks.
• High level of integration for reduced cost
  The R-Car V3M includes an integrated ISP that makes the image ready for computer vision. The integration eliminates the need for an external ISP component in the front camera or in the sensor itself.
• Open solution for front camera

Samples of the R-Car V3M SoC will be available from December 2017. Mass production is scheduled to begin in June 2019.

Go to the original article...

Image Sensors World        Go to the original article...

Sony publishes its updated semiconductor business factsheet:

Go to the original article...

Image Sensors World        Go to the original article...

TED interview with Elon Musk has a couple of interesting notes about Tesla autonomous car plans:

"Tesla Model 3 is coming in July, Musk says, and it’ll have a special feature: autopilot. Using only passive optical cameras and GPS, no LIDAR, the Model 3 will be capable of autonomous driving. “Once you solve cameras for vision, autonomy is solved; if you don’t solve vision, it’s not solved … You can absolutely be superhuman with just cameras.”

Musk says that Tesla is on track for completing a fully autonomous, cross-country LA to New York trip by the end of 2017. “November or December of this year, we should be able to go from a parking lot in California to a parking lot in New York, no controls touched at any point during the entire journey,” Musk says."

Go to the original article...

Image Sensors World        Go to the original article...

Lund University, Sweden, announces that its researches were able to achieve 200fs temporally resolving camera. Their paper has been accepted for publication in "Light: Science & Applications" journal and available on-line: "FRAME: femtosecond videography for atomic and molecular dynamics" by Andreas Ehn, Joakim Bood, Zheming Li, Edouard Berrocal, Marcus Aldén, and Elias Kristensson:

"In this Letter, we present a laser probe-based method that unifies the aforementioned aspects and is capable of producing a video sequence of non-repetitive dynamic events at femtosecond timescales and beyond for spectroscopy. Our method, called Frequency Recognition Algorithm for Multiple Exposures (FRAME), allows acquisition of a series of laser-induced images at frame rates that are limited only by the laser pulse duration. Here, we used femtosecond laser pulses to demonstrate videography with record high sub-ps (THz) temporal resolution, although attosecond laser pulses could, in principle, increase the frame rate into the PHz regime. The novelty of the FRAME concept lies in superimposing a structural code onto the illumination to encrypt a single event that is then deciphered in the data post-processing. Because each image in the video sequence is extracted by using a unique spatial code, the method does not rely on a specific optical wavelength or laser bandwidth, and hence can be used for spectroscopic measurements. Here, we provide a proof-of-principle demonstration of our method by performing imaging light-in-flight – often regarded as the gold standard experiment for ultrafast imaging at 5 THz, a timescale on which even light appears stationary on a macroscopic scale."




Although the paper talks about 200fs temporal resolution, the University demo shows 50fs time step between the frames of their video:

Go to the original article...

Image Sensors World        Go to the original article...

IAM-Media: Ottawa, Canada-based Conversant NPE has acquired 29 image sensor patents from Dongbu HiTek, USPTO assignments records reveal.

Dongbu’s somewhat unusual willingness to sell its IP seems to be related to previous debt problems and the financial woes of its parent, the Dongbu Group chaebol (conglomerate). In the previous years, IAM-Media identified Dongbu HiTek as one of the top 15 sellers of patents worldwide:

Go to the original article...

Image Sensors World        Go to the original article...

PulseNews: Samsung is to convert Line 11 in Hwaseong fab in Gyeonggi Province active since mid-2000s to CIS production starting the first half next year to meet growing demand for mobile imaging, high-end photography, and security and surveillance applications. Sony is said to have a solid lead on the CIS narket, controlling 44.5% of global supply including iPhones. Samsung is at second place with a share of 15% as it supplies the imagers for Samsung premium phones.

SeekingAlpha quotes Samsung representative on that: "we’re planning to convert Line 11 to CIS given the fact that strong image sensor demand...

And to give thorough detail about the CMOS image sensor CIS that we’ll be using part of Line 11 to produce as you know with the increased adoption of dual camera, the application has itself been growing and also in addition to these mobile cameras we’re also trying to expand the applications to VR automotive as well as 360 degree cameras. So to respond to market demand as well as customer request, we will be converting part of Line 11 for our CIS production. Our target is to start mass production first half of 2018 but that is subject to market demand as well as customer request."

Samsung Hwaseong plant

Go to the original article...

Image Sensors World        Go to the original article...

Sony reports its quarterly and yearly results. Few slides showing image sensor business status and the new year forecas:


From the Sony PR:

Results for a fiscal year ended on March 31, 2017:

"Sales increased 4.6% year-on-year (a 15% increase on a constant currency basis) to 773.1 billion yen (6,903 million U.S. dollars). This increase in sales was primarily due to a significant increase in unit sales of image sensors mainly for mobile products, partially offset by the impact of foreign exchange rates, a significant decrease in sales of camera modules, a business which was downsized, and the decrease in production due to the 2016 Kumamoto Earthquakes. Sales to external customers increased 10.1% year-on-year.

Operating loss of 7.8 billion yen (70 million U.S. dollars) was recorded, compared to operating income of 14.5 billion yen in the previous fiscal year. This significant deterioration in operating results was primarily due to the negative impact of foreign exchange rates, the above-mentioned expenses resulting from the 2016 Kumamoto Earthquakes, and a 6.5 billion yen (58 million U.S. dollars) write-down of inventories of certain image sensors mainly for mobile products. This deterioration was partially offset by the above-mentioned year-on-year increase in sales and the decrease in impairment charges against long-lived assets related to the camera module business.
During the current fiscal year, there was a 43.7 billion yen negative impact from foreign exchange rate fluctuations."

Last quarter results:

"In the Semiconductors segment, sales increased significantly due to an increase in unit sales of image sensors. Sales to external customers also increased significantly year-on-year."

Reuters comments: "Image sensors have emerged as the biggest contributor to profit growth, as operations at a factory damaged by earthquakes a year ago return to normal just as smartphone makers are increasingly adopting dual-lens rear cameras in their handsets, requiring extra sensors.

"The market is growing for higher-end models with dual-lens rear cameras or with high-performance front cameras designed for self-shooting," Chief Financial Officer Kenichiro Yoshida said at an earnings briefing. "That serves as a tailwind for us."

Sony forecast its chip division, which includes image sensors, to return to profit this year, at 120 billion yen.

To meet increased demand, it will more than double capital expenditure on image sensors to 110 billion yen, raising production capacity by 13.6 percent by the end of March."

Go to the original article...

Image Sensors World        Go to the original article...

ETNews: SK Hynix is separating its 8-inch Foundry Business Department as an independent subsidiary tentatively named "SK Hynix System IC". The 8-inch M8 fab has a capacity of 100K wafers per year and image sensors is one of its main specialties.

According to IHS, SK Hynix’s Foundry Business made about $140M in sales last year, less that 1% of the whole company sales of $15.2b. The foundry business registered a loss last year.

SK Hynix will continue production of high-megapixel image sensor products of over 13MP at its 12-inch M10 fab.

Go to the original article...

Image Sensors World        Go to the original article...

PRNewswire: Switzerland-based ActLight presents its Heartrate Sensor for wearables, said to reduce power consumption by 80% compared to what is currently available on the market, while delivering the same performance. The Heartrate sensor is based on ActLight's proprietary Dynamic Photo Diode (DPD) technology and has been developed in co-operation with a global MedTech semiconductor vendor and EPFL.

In smartwatches and wearables, up to 80% of the power consumption is said to be related to heartrate monitoring when it is activated. Already tested and calibrated, ActLight's heartrate sensor is ready to be produced in large series to equip next generation wearables with reduced power consumption.

"Increased interest in fitness and wellness - has necessitated more affordable, precise, wearable sensing options. Our technology offers unique competitive advantages to our partners when compared to existing Photo Diodes used in wearable heartrate solutions," stated Serguei Okhonin, CEO of ActLight. "ActLight is now working with a number of partners to bring this technology into consumer electronics and MedTech grade solutions. We are open for IP licensing of our patented DPD technology that we own 100%," Okhonin continued.

EPFL publishes an article and an Youtube video on the new product:

Go to the original article...

Image Sensors World        Go to the original article...

Anandtech: ARM announces Mali-C71 ultra-wide dynamic range (UWDR) ISP with up to 24 stops DR for automotive camera applications. It removes noise and processes the multiple exposures from the camera, creates an ultra-WDR frame and sends it across to a display or a computer vision engine. The Mali-C71 is the first imaging product since ARM acquired Apical in 2016. OmniVision Technologies, a leading developer of advanced digital imaging solutions, has welcomed the development:

“OmniVision has been closely collaborating with the ARM ISP team (formerly Apical) for several years now, and see the Mali-C71 as an important advancement in image signal processing for emerging automotive applications. ARM’s expertise in ISP technology together with OmniVision’s best-in-class HDR image sensor technologies offers an industry leading solution for our automotive customers,” said Will Foote, senior partnership manager, OmniVision.

The Mali-C71 brings advanced error detection with more than 300 dedicated fault detection circuits to enable system-level certification to the modern automotive standards (ISO26262, ASIL D and IEC 61508, SIL3). It is able to process up to 4 real-time cameras and 16 camera streams with a single pipeline:

Mali-C71 block diagram

Go to the original article...

Image Sensors World        Go to the original article...

Cadence Tensilica Vision P-Series DSPs are said to be the first imaging/vision DSPs to pass Khronos Group’s conformance tests for the OpenVX 1.1 spec. This enables faster development of computer vision and imaging applications on Tensilica Vision P-Series DSPs being deployed in applications processors for mobile, automotive, drone, security, AR/VR and other markets. Applications developed using the standard OpenVX 1.1 API can be compiled and run on Tensilica Vision DSPs without any code changes.

Go to the original article...

Image Sensors World        Go to the original article...

TechInsights completes its teardown report of Samsung Galaxy S8, and publishes an estimation of the phone's BOM. The cameras are about 9.25% of the BOM, down from 10% in the last year's S7:

Go to the original article...

Image Sensors World        Go to the original article...

SPIETV publishes FLIR presentation on fast thermal imagers and their applications:

Go to the original article...

Image Sensors World        Go to the original article...

PCO DR whitepaper compiles a table comparing various image sensors DR. The rolling shutter BAE sCMOS sensor outperforms all global shutter alternatives on DR:

Go to the original article...

Image Sensors World        Go to the original article...

Albert Theuwissen has a unique experience with image sensor statistics and their parameters changing over time when his students run through the same devices characterization, course after course, year after year. So, he came up with a brilliant idea to present his knowledge in a report on CMOS sensors reproducibility, variability, and reliability. The first info about the report is expected to be available in a few weeks.

Go to the original article...

Image Sensors World        Go to the original article...

Strategy Analytics presentation dated by Sept. 2016 forecasts a large growth for ADAS camera market:

Go to the original article...

Image Sensors World        Go to the original article...

Prof. Shen Guozhen’s Group in Institute of Semiconductors, Chinese Academy of Sciences, with their collaborators in Hong Kong University of Science and Technology developed a flexible UV image sensor based on ZnO quantum dots (QDs) decorated ZTO nanowires (NWs). The device was fabricated on the flexible polyethylene terephthalate (PET) substrate, which displayed outstanding flexibility, electrical stability and folding endurance.

Duo to the rational band engineering at the QD/NW interface which results in effective separation of electron-hole pairs, the device showed ultrahigh specific detectivity (up to 9.0 × 1017 Jones), photoconductive gain (up to 1.1 × 107) and high response speed (47 ms).


EETimes: Non-uniform spatial image sampling has been a popular idea over many decades. University of Glasgow, Scotland, UK, gives it another try in arxiv.org open-acess paper "Adaptive foveated single-pixel imaging with dynamic super-sampling" by David B. Phillips, Ming-Jie Sun, Jonathan M. Taylor, Matthew P. Edgar, Stephen M. Barnett, Graham G. Gibson, and Miles J. Padgett.

"In this work we take a different approach and adopt a strategy inspired by the foveated vision systems found in the animal kingdom - a framework that exploits the spatio-temporal redundancy present in many dynamic scenes. In our single-pixel imaging system a high-resolution foveal region follows motion within the scene, but unlike a simple zoom, every frame delivers new spatial information from across the entire field-of-view. Using this approach we demonstrate a four-fold reduction in the time taken to record the detail of rapidly evolving features, whilst simultaneously accumulating detail of more slowly evolving regions over several consecutive frames."

Go to the original article...

Image Sensors World        Go to the original article...

Reuters: Sony revises upward its annual profit estimation, in part due to the lower costs in its image sensor business. The official annual results for the company's fiscal year ended on March 31, 2017 are to be announced on April 28.

Go to the original article...

Image Sensors World        Go to the original article...

TechInsights posted a teardown info on Galaxy S8 rear camera. The 12MP stacked dual pixel image sensor has been identified as one made by Sony, probably IMX333. Other Galaxy S8 phones might contain Samsung-made sensors.

The absence of dual camera in a flagship phone is said to save about $12.50 from the final BOM costs for Samsung.

Go to the original article...

Image Sensors World        Go to the original article...

Analog Devices one more time demos its IR spectrometer-based Material Sensing Platform for smartphones, developed together with Consumer Physics. The previous demo is here.

Go to the original article...

Image Sensors World        Go to the original article...

TechInsights publishes Samsung Galaxy S8 smartphone reverse engineering article listing Samsung S5C73C3 image processors among the other chips on the PCB. The front camera with iris scanner module is shown on the images below:


The front camera is based on Sony stacked IMX320, while the iris scanner BSI sensor is marked as S5K5E6YV (or S5K5E6YV). Both sensors have pixel pitch of 1.12um.

Go to the original article...

Image Sensors World        Go to the original article...

King Abdullah University of Science and Technology (KAUST), Saudi Arabia publishes an interview with Eric Fossum on CMOS sensor invention history and Quanta image sensor:

Go to the original article...

Image Sensors World        Go to the original article...

Mobileye CEO and Co-Founder Ziv Aviram demos his company self-driving car in the 360-deg surround video:



The full presentation at YPO Edge 2017 held in Vancouver, Canada can be seen here.

Go to the original article...