Image Sensors World        Go to the original article...

Edgehog is a Montreal-based startup that has developed a solution for the stray light problem in camera and LiDAR sensors' coverglass.

Edgehog glass, a next-generation anti-reflection technology, removes image artifacts through the innovative process of glass nanotexturing by creating a gradient of refractive index on filters and image sensor covers. This enables uncompromised visuals from cameras and flare-free imaging with CMOS image sensors even in challenging lighting conditions. The result is a cleaner raw signal from the hardware without expensive image processing, laying the foundation for superior computer vision applications. The advanced nanotextured Edgehog glass enables camera optics designers to achieve unparalleled image clarity for a wide viewing angle.

 

Email: info@edgehogtech.com
Phone: +1 (438) 230 0101
Web: http://www.edgehogtech.com

Go to the original article...

Image Sensors World        Go to the original article...

Huang et al. from Shanghai Advanced Research Institute recently published a paper titled "A 13-Bit, 12-ps Resolution Vernier Time-to-Digital Converter Based on Dual Delay-Rings for SPAD Image Sensor" in Sensors journal.

Go to the original article...

Image Sensors World        Go to the original article...

From IDQuantique: https://www.idquantique.com/sk-telecom-and-samsung-unveil-the-galaxy-quantum-4/

SK Telecom and Samsung unveil the Galaxy Quantum 4, providing more safety and performance with IDQ’s QRNG Chip

Geneva, June 12th 2023

ID Quantique (IDQ), the world leader in quantum-safe security solutions, SK Telecom and Samsung Electronics, have worked together to release the ‘Galaxy Quantum 4’, the fourth Samsung smartphone equipped with quantum technology, designed to protect customers’ information.

With features matching those of Samsung’s flagship smartphones of the S23 series – i.e. waterdrop camera with image stabilization (OIS) and nightography (night/low-light shooting), rear glass design, large capacity battery – along with strengthened quantum-safe technology, the Galaxy Quantum 4 will be a new choice for customers who value both high performance and security.

Like its predecessor, the Galaxy Quantum 4 is equipped with the world’s smallest (width 2.5mm x length 2.5 mm) Quantum Random Number Generator (QRNG) chipset, designed by ID Quantique; enabling trusted authentication and encryption of information. It allows smartphone holders to use an even wider number of applications and services in a safer and more secure manner by generating unpredictable true random numbers.

IDQ’s QRNG chip enhances the security of a very large number of services provided by the operator. QRNG protects the process from log-in/authentication/payment/unlock/OTP generation of service apps ranging from financial apps to social media apps and games offering a much higher level of trust to the users.

As an example, when an application provides authentication services, sensitive data such as fingerprints and facial images must be protected. Our QRNG, embedded in this new smartphone, can therefore be leveraged to generate encryption keys and, in conjunction with the keystore of the terminal, provide quantum enhanced security every time a user logs in to the app. The QRNG is also used to encrypt data stored in the external memory card.

As the previous version, the ‘Galaxy Quantum 4’ offers a differentiated security experience to customers by providing a ‘quantum indicator’ on the status bar so that customers can realize that they are using a quantum security service. Its price point is comparable to previous versions, but with increased performance and security.

“Protecting one’s private data is a priority for users. The Galaxy Quantum 4 is the latest in the Quantum series, which offers strong quantum security and premium performance. As a leading player in this area, we will continue to expand the use of quantum cryptography technology to provide users with greater security and safety,” said Moon Kab-in, Vice President and Head of Smart Device Center at SKT.

“Mobile phone users don’t want to get their data stolen. The Galaxy Quantum 4 includes top performances and more quantum-secured applications than ever before, bringing applications and services to a new level of security in the mobile phone industry” said Grégoire Ribordy, CEO and co-founder of ID Quantique.

Go to the original article...

Image Sensors World        Go to the original article...

From Semiconductor Engineering https://semiengineering.com/machine-vision-plus-ai-ml-opens-huge-opportunities/

Machine Vision Plus AI/ML Adds Vast New Opportunities

Traditional technology companies and startups are racing to combine machine vision with AI/ML, enabling it to “see” far more than just pixel data from sensors, and opening up new opportunities across a wide swath of applications.

In recent years, startups have been able to raise billions of dollars as new MV ideas come to light in markets ranging from transportation and manufacturing to health care and retail. But to fully realize its potential, the technology needs to address challenges on a number of fronts, including improved performance and security, and design flexibility.

Fundamentally, a machine vision system is a combination of software and hardware that can capture and process information in the form of digital pixels. These systems can analyze an image, and take certain actions based on how it is programmed and trained. A typical vision system consists of an image sensor (camera and lens), image and vision processing components (vision algorithm) and SoCs, and the network/communication components.

Both still and video digital cameras contain image sensors. So do automotive sensors such as lidar, radar, ultrasound, which deliver an image in digital pixel form, although not with the same resolution. While most people are familiar these types of images, a machine also can “see” can heat and audio signals data, and they can analyze that data to create a multi-dimensional image.
“CMOS image sensors have seen drastic improvement over the last few years,” said Ron Lowman, strategic marketing manager at Synopsys. “Sensor bandwidth is not being optimized for human sight anymore, but rather for the value AI it can provide. For instance, MIPI CSI, the dominant vision sensor interface, is not only increasing bandwidths, but also adding AI features such as Smart Region of Interest (SROI) and higher color depth. Although these color depth increases can’t be detected by the human eye, for machine vision it can improve the value of a service dramatically.”

Machine vision is a subset of the broader computer vision. “While both disciplines rely on looking at primarily image data to deduce information, machine vision implies ‘inspection type’ applications in an industry or factory setting,” said Amol Borkar, director of product management, marketing and business development, Tensilica Vision and AI DSPs at Cadence. “Machine vision relies heavily on using cameras for sensing. However, ‘cameras’ is a loaded term because we are typically familiar with an image sensor that produces RGB images and operates in the visible light spectrum. Depending on the application, this sensor could operate in infrared, which could be short wave, medium wave, long wave IR, or thermal imaging, to name a few variants. Event cameras, which are very hyper-sensitive to motion, were recently introduced. On an assembly line, line scan cameras are a slightly different variation from typical shutter-based cameras. Most current applications in automotive, surveillance, and medical rely on one or more of these sensors, which are often combined to do some form of sensor fusion to produce a result better than a single camera or sensor.”

Benefits
Generally speaking, MV can see better than people. The MV used in manufacturing can improve productivity and quality, lowering production costs. Paired with ADAS for autonomous driving, MV can take over some driving functions. Together with AI, MV can help analyze medical images.
The benefits of using machine vision include higher reliability and consistency, along with greater precision and accuracy (depending on camera resolution). And unlike humans, machines do not get tired, provided they receive routine maintenance. Vision system data can be stored locally or in the cloud, then analyzed in real-time when needed. Additionally, MV reduces production costs by detecting and screening out defective parts, and increases inventory control efficiency with OCR and bar-code reading, resulting in lower overall manufacturing costs.

Today, machine vision usually is deployed in combination with AI, which greatly enhances the power of data analysis. In modern factories, automation equipment, including robots, is combined with machine vision and AI to increase productivity.

How AI/ML and MV interact
With AI/ML, MV can self-learn and improve after capturing digital pixel data from sensors.
“Machine vision (MV) and artificial intelligence (AI) are closely related fields, and they often interact in various ways,” said Andy Nightingale, vice president of product marketing at Arteris IP. “Machine vision involves using cameras, sensors, and other devices to capture images or additional data, which is then processed and analyzed to extract useful information. Conversely, AI involves using algorithms and statistical models to recognize patterns and make predictions based on large amounts of data.”
This also can include deep learning techniques. “Deep learning is a subset of AI that involves training complex neural networks on large datasets to recognize patterns and make predictions,” Nightingale explained. ” Machine vision systems can use deep learning algorithms to improve their ability to detect and classify objects in images or videos. Another way that machine vision and AI interact is through the use of computer vision algorithms. Computer vision is a superset of machine vision that uses algorithms and techniques to extract information from images and videos. AI algorithms can analyze this information and predict what is happening in the scene. For example, a computer vision system might use AI algorithms to analyze traffic patterns and predict when a particular intersection will likely become congested. Machine vision and AI can also interact in the context of autonomous systems, such as self-driving cars or drones. In these applications, machine vision systems are used to capture and process data from sensors. In contrast, AI algorithms interpret this data and make decisions about navigating the environment.”

AI/ML, MV in autonomous driving
AI has an increasing number of roles in modern vehicles, but the two major roles are in perception and decision making.

“Perception is the process of understanding one’s surroundings through onboard and external sensor arrays,” said David Fritz, vice president of hybrid and virtual systems at Siemens Digital Industries Software. “Decision-making first takes the understanding of the surrounding state and a goal such as moving toward the destination. Next, the AI decides the safest, most effective way to get there by controlling the onboard actuators for steering, braking, accelerating, etc. These two critical roles address very different problems. From a camera or other sensor, the AI algorithms will use raw data from the sensors to perform object detection. Once an object is detected, the perception stack will classify the object, for example, whether the object is a car, a person, or an animal. The training process is lengthy and requires many training sets presenting objects from many different angles. After training, the AI network can be loaded into the digital twin or physical vehicle. Once objects are detected and classified decisions can be made by another trained AI network to control steering, braking, and acceleration. Using a high-fidelity digital twin to validate the process virtually has been shown to result in safer, more effective vehicles faster than simply using open road testing.”

How much AI/ML is needed is a question frequently asked by developers. In the case of modern factories, MV can be used to simply detect and pick out defective parts in an assembly line or employed to assemble automobiles. Doing the latter requires advanced intelligence and a more sophisticated design to ensure timing, precision, and calculation of motion and distance in the assembly process.
“Automation using robotics and machine vision has increased productivity in modern factories,” observed Geoff Tate, CEO of Flex Logix. “Many of these applications use AI. A simple application — for instance, detecting if a label is applied correctly — does not require a great deal of intelligence. On the other hand, a sophisticated, precision robot arm performing 3D motion requires much more GPU power. In the first application, one tile of AI IP will be sufficient, while the second application may need multiple tiles. Having flexible and scalable AI IPs would make designing robotics and machine vision much easier.”

Applications
Machine vision applications are limited only by one’s imagination. MV can be used in almost any industrial and commercial segment, so long as it requires vision and processing. Here is a partial list:
 Transportation (autonomous driving, in-cabin monitoring, traffic flow analysis, moving violation and accident detection);

•  Manufacturing and automation (productivity analysis, quality management);
•  Surveillance (detection of motion and intrusion monitor);
•  Health care (imaging, cancer and tumor detection, cell classification);
•  Agriculture (farm automation, plant disease and insect detection);
•  Retail (customer tracking, empty shelf detection, theft detection), and
•  Insurance (accident scene analysis from images).

There are many other applications. Consider drinking water or soft drink bottling. A machine vision system can be used to inspect fill levels, which typically is done by highly efficient robots. But robots occasionally make mistakes. MV can ensure the fill level is consistent and the labels are applied correctly.

Detecting any machine parts that deviate from measurement specification limits is another job for MV. Once the MV is trained on the specification, it can detect the parts that are outside the specification limits.

MV can detect uniform shapes such as squares or circles as well as odd-shaped parts, so it can be used to identify, detect, measure, count, and (with robots), pick and place.
Finally, combining AI, MV can perform tire assembly with precision and efficiency. Nowadays, OEMs automate vehicle assembly with robots. One of the processes is to install the four wheels to a new vehicle. Using MV, a robotic arm can detect the correct distance and apply just the right amount of pressure to prevent any damage.

Types of MV
MV technologies can be divided into one-dimensional (1D), two-dimensional (2D), and three-dimensional (3D).

1D systems analyze data one line at a time, comparing variations among groups. Usually it is used in production of items such as plastics and paper on a continual basis. 2D systems, in contrast, use a camera to scan line by line to form an area or a 2D image. In some cases, the whole area is scanned and the object image can then be unwrapped for detailed inspection. 

3D systems consist of multiple cameras or laser sensors to capture the 3D view of an object. During the training process, the object or the cameras need to be moved to capture the entire product. Recent technology can produce accuracy within micrometers. 3D systems produce higher resolution but are also more expensive.

Emerging MV startups and new innovations
Tech giants, including IBM, Intel, Qualcomm, and NVIDIA, have publicly discussed investments in MV. In addition, many startups are developing new MV solutions such as Airobotics , Arcturus Networks, Deep Vision AI , Hawk-Eye Innovations, Instrumental, lending AI, kinara, Mech-Mind, Megvii, NAUTO, SenseTime, Tractable, ViSenze, Viso, and others. Some of these companies have been able to raise funding in excess of $1 billion.

In transportation, insurance companies can use MV to scan photographs and videos of scenes of accidents and disasters for financial damage analysis. Additionally, AI-based MV can power safety platforms to analyze driver behavior.

In software, computer vision platforms can be created without the knowledge of coding. Other startups have developed the idea for MV authentication software. And in the field of sports, AI, vision, and data analysis could provide coaches the ability to understand how decisions are made by players during a game. Also, one startup devised a cost reduction idea for surveillance by combining AI and MV in unmanned, aerial drone design.

Both MV and AI are changing quickly, and will continue to increase in performance, including precision and accuracy, while high GPU and ML power will come down in cost, propelling new MV applications.

Arteris’ Nightingale noted there will be further improvements in accuracy and speed. “Machine vision systems will likely become more accurate and faster. This will be achieved through advancements in hardware, such as sensors, cameras, and processors, as well as improvements in algorithms and machine learning models,” he said, pointing to an increased use of deep learning, as well. “Deep learning has been a significant driver of progress in machine vision technology in recent years, and it is likely to play an even more substantial role in the future. Deep learning algorithms can automatically learn data features and patterns, leading to better accuracy and performance. There will be an enhanced ability to process and analyze large amounts of data, as machine vision technology can process and analyze large amounts of data quickly and accurately. We may also see advancements in machine vision systems that can process significantly larger datasets, leading to more sophisticated and intelligent applications.”

Further, MV and AI are expected to integrate with other technologies to provide additional high-performance, real-time applications.

“Machine vision technology is already integrated with other technologies, such as robotics and automation,” he said. “This trend will likely continue, and we may see more machine vision applications in health care, transportation, and security. As well, there will be more real-time applications. Machine vision technology is already used for real-time applications, such as facial recognition and object tracking. In the future, we may see more applications that require real-time processing, such as self-driving cars and drones.”

MV design challenges
Still, there are challenges in training an MV system. Its accuracy and performance depend on how well the MV is trained. Inspection can encompass parameters such as orientation, variation of the surfaces, contamination, and accuracy tolerances such as diameter, thickness, and gaps. 3D systems can perform better than 1D or 2D systems when detecting cosmetic and service variation effects. In other cases, when seeing an unusual situation, human beings can draw on knowledge from a different discipline, while MV and AI may not have that ability.

“Some of today’s key challenges include data flow management and control – especially with real-time latency requirements such as those in automotive applications — while keeping bandwidth to a minimum,” said Alexander Zyazin, senior product manager in Arm‘s Automotive Line of Business. “In camera-based systems, image quality (IQ) remains critical. It requires a hardware design to support ultra-wide dynamic range and local tone mapping. But it also requires IQ tuning, where traditionally subjective evaluation by human experts was necessary, making the development process lengthy and costly. The new challenge for MV is that this expertise might not result in the best system performance, as perception engines might prefer to see images differently to humans and to one another, depending on the task.”

In general, machines can do a better job when doing mundane tasks over and over again, or when recognizing an image with more patterns than humans can typically process. “As an example, a machine may do a better job recognizing an anomaly in a medical scan than a human, simply because the doctor may make a mistake, be distracted or tired,” said Thomas Andersen, vice president for AI and machine learning at Synopsys. “When inspecting high-precision circuits, a machine can do a much better job analyzing millions of patterns and recognizing errors, a task a human could not do, simply due to the size of the problem. On the other hand, machines have not yet reached the human skill of recognizing the complex scenes that can occur while driving a car. It may seem easy for a human to recognize and anticipate certain reactions, while the machine may be better in ‘simple’ situations that a human easily could deal with, but did not due to a distraction, inattention or incapacitation – for example auto stop safety systems to avoid an imminent collision. A machine can always react faster than a human, assuming it interprets the situation correctly.”

Another challenge is making sure MV is secure. With cyberattacks increasing constantly, it will be important to ensure no production disruption or interference from threat actors.

“Security is critical to ensuring the output of MV technology isn’t compromised,” said Arm’s Zyazin. “Automotive applications are a good example of the importance of security in both hardware and software. For instance, the information processed and extracted from the machine is what dictates decisions such as braking or lane-keep assist, which can pose a risk to those inside the vehicle if done incorrectly.”

Conclusion
MV designs include a mixture of chips (processors, memories, security), IPs, modules, firmware, hardware and software. The rollout of chiplets and multi-chip packaging will allow those systems to be combined in novel ways more easily and more quickly, adding new features and functions and improving the overall efficiency and capabilities of these systems.

“Known good die (KGD) solutions can provide cost and space efficient alternatives to packaged products with limited bonding pads and wires,” said Tetsu Ho, DRAM manager at Winbond. That helps improve design efficiency, provides enhanced hardware security performance, and especially time-to-market for product launch. These die go through 100% burn-in and are tested to the same extent as discrete parts. KGD 2.0 is needed to assure end-of-line yield in 2.5D/3D assembly and 2.5D/3D multichip devices to realize improvements in PPA, which means bandwidth performance, power efficiency, and area as miniaturization, driven by the explosion of technologies such as edge-computing AI.”

This will open new options for MV in new an existing markets. It will be used to support humans in autonomous driving, help robots perform with precision and efficiency in manufacturing, and perform surveillance with unmanned drones. In addition, MV will be able to explore places that are considered dangerous for humans, and provide data input and analysis for many fields, including insurance, sports, transportation, defense, medicine, and more.

Go to the original article...

Image Sensors World        Go to the original article...

From Semiconductor Engineering https://semiengineering.com/machine-vision-plus-ai-ml-opens-huge-opportunities/

Machine Vision Plus AI/ML Adds Vast New Opportunities

Traditional technology companies and startups are racing to combine machine vision with AI/ML, enabling it to “see” far more than just pixel data from sensors, and opening up new opportunities across a wide swath of applications.

In recent years, startups have been able to raise billions of dollars as new MV ideas come to light in markets ranging from transportation and manufacturing to health care and retail. But to fully realize its potential, the technology needs to address challenges on a number of fronts, including improved performance and security, and design flexibility.

Fundamentally, a machine vision system is a combination of software and hardware that can capture and process information in the form of digital pixels. These systems can analyze an image, and take certain actions based on how it is programmed and trained. A typical vision system consists of an image sensor (camera and lens), image and vision processing components (vision algorithm) and SoCs, and the network/communication components.

Both still and video digital cameras contain image sensors. So do automotive sensors such as lidar, radar, ultrasound, which deliver an image in digital pixel form, although not with the same resolution. While most people are familiar these types of images, a machine also can “see” can heat and audio signals data, and they can analyze that data to create a multi-dimensional image.
“CMOS image sensors have seen drastic improvement over the last few years,” said Ron Lowman, strategic marketing manager at Synopsys. “Sensor bandwidth is not being optimized for human sight anymore, but rather for the value AI it can provide. For instance, MIPI CSI, the dominant vision sensor interface, is not only increasing bandwidths, but also adding AI features such as Smart Region of Interest (SROI) and higher color depth. Although these color depth increases can’t be detected by the human eye, for machine vision it can improve the value of a service dramatically.”

Machine vision is a subset of the broader computer vision. “While both disciplines rely on looking at primarily image data to deduce information, machine vision implies ‘inspection type’ applications in an industry or factory setting,” said Amol Borkar, director of product management, marketing and business development, Tensilica Vision and AI DSPs at Cadence. “Machine vision relies heavily on using cameras for sensing. However, ‘cameras’ is a loaded term because we are typically familiar with an image sensor that produces RGB images and operates in the visible light spectrum. Depending on the application, this sensor could operate in infrared, which could be short wave, medium wave, long wave IR, or thermal imaging, to name a few variants. Event cameras, which are very hyper-sensitive to motion, were recently introduced. On an assembly line, line scan cameras are a slightly different variation from typical shutter-based cameras. Most current applications in automotive, surveillance, and medical rely on one or more of these sensors, which are often combined to do some form of sensor fusion to produce a result better than a single camera or sensor.”

Benefits
Generally speaking, MV can see better than people. The MV used in manufacturing can improve productivity and quality, lowering production costs. Paired with ADAS for autonomous driving, MV can take over some driving functions. Together with AI, MV can help analyze medical images.
The benefits of using machine vision include higher reliability and consistency, along with greater precision and accuracy (depending on camera resolution). And unlike humans, machines do not get tired, provided they receive routine maintenance. Vision system data can be stored locally or in the cloud, then analyzed in real-time when needed. Additionally, MV reduces production costs by detecting and screening out defective parts, and increases inventory control efficiency with OCR and bar-code reading, resulting in lower overall manufacturing costs.

Today, machine vision usually is deployed in combination with AI, which greatly enhances the power of data analysis. In modern factories, automation equipment, including robots, is combined with machine vision and AI to increase productivity.

How AI/ML and MV interact
With AI/ML, MV can self-learn and improve after capturing digital pixel data from sensors.
“Machine vision (MV) and artificial intelligence (AI) are closely related fields, and they often interact in various ways,” said Andy Nightingale, vice president of product marketing at Arteris IP. “Machine vision involves using cameras, sensors, and other devices to capture images or additional data, which is then processed and analyzed to extract useful information. Conversely, AI involves using algorithms and statistical models to recognize patterns and make predictions based on large amounts of data.”
This also can include deep learning techniques. “Deep learning is a subset of AI that involves training complex neural networks on large datasets to recognize patterns and make predictions,” Nightingale explained. ” Machine vision systems can use deep learning algorithms to improve their ability to detect and classify objects in images or videos. Another way that machine vision and AI interact is through the use of computer vision algorithms. Computer vision is a superset of machine vision that uses algorithms and techniques to extract information from images and videos. AI algorithms can analyze this information and predict what is happening in the scene. For example, a computer vision system might use AI algorithms to analyze traffic patterns and predict when a particular intersection will likely become congested. Machine vision and AI can also interact in the context of autonomous systems, such as self-driving cars or drones. In these applications, machine vision systems are used to capture and process data from sensors. In contrast, AI algorithms interpret this data and make decisions about navigating the environment.”

AI/ML, MV in autonomous driving
AI has an increasing number of roles in modern vehicles, but the two major roles are in perception and decision making.

“Perception is the process of understanding one’s surroundings through onboard and external sensor arrays,” said David Fritz, vice president of hybrid and virtual systems at Siemens Digital Industries Software. “Decision-making first takes the understanding of the surrounding state and a goal such as moving toward the destination. Next, the AI decides the safest, most effective way to get there by controlling the onboard actuators for steering, braking, accelerating, etc. These two critical roles address very different problems. From a camera or other sensor, the AI algorithms will use raw data from the sensors to perform object detection. Once an object is detected, the perception stack will classify the object, for example, whether the object is a car, a person, or an animal. The training process is lengthy and requires many training sets presenting objects from many different angles. After training, the AI network can be loaded into the digital twin or physical vehicle. Once objects are detected and classified decisions can be made by another trained AI network to control steering, braking, and acceleration. Using a high-fidelity digital twin to validate the process virtually has been shown to result in safer, more effective vehicles faster than simply using open road testing.”

How much AI/ML is needed is a question frequently asked by developers. In the case of modern factories, MV can be used to simply detect and pick out defective parts in an assembly line or employed to assemble automobiles. Doing the latter requires advanced intelligence and a more sophisticated design to ensure timing, precision, and calculation of motion and distance in the assembly process.
“Automation using robotics and machine vision has increased productivity in modern factories,” observed Geoff Tate, CEO of Flex Logix. “Many of these applications use AI. A simple application — for instance, detecting if a label is applied correctly — does not require a great deal of intelligence. On the other hand, a sophisticated, precision robot arm performing 3D motion requires much more GPU power. In the first application, one tile of AI IP will be sufficient, while the second application may need multiple tiles. Having flexible and scalable AI IPs would make designing robotics and machine vision much easier.”

Applications
Machine vision applications are limited only by one’s imagination. MV can be used in almost any industrial and commercial segment, so long as it requires vision and processing. Here is a partial list:
 Transportation (autonomous driving, in-cabin monitoring, traffic flow analysis, moving violation and accident detection);

•  Manufacturing and automation (productivity analysis, quality management);
•  Surveillance (detection of motion and intrusion monitor);
•  Health care (imaging, cancer and tumor detection, cell classification);
•  Agriculture (farm automation, plant disease and insect detection);
•  Retail (customer tracking, empty shelf detection, theft detection), and
•  Insurance (accident scene analysis from images).

There are many other applications. Consider drinking water or soft drink bottling. A machine vision system can be used to inspect fill levels, which typically is done by highly efficient robots. But robots occasionally make mistakes. MV can ensure the fill level is consistent and the labels are applied correctly.

Detecting any machine parts that deviate from measurement specification limits is another job for MV. Once the MV is trained on the specification, it can detect the parts that are outside the specification limits.

MV can detect uniform shapes such as squares or circles as well as odd-shaped parts, so it can be used to identify, detect, measure, count, and (with robots), pick and place.
Finally, combining AI, MV can perform tire assembly with precision and efficiency. Nowadays, OEMs automate vehicle assembly with robots. One of the processes is to install the four wheels to a new vehicle. Using MV, a robotic arm can detect the correct distance and apply just the right amount of pressure to prevent any damage.

Types of MV
MV technologies can be divided into one-dimensional (1D), two-dimensional (2D), and three-dimensional (3D).

1D systems analyze data one line at a time, comparing variations among groups. Usually it is used in production of items such as plastics and paper on a continual basis. 2D systems, in contrast, use a camera to scan line by line to form an area or a 2D image. In some cases, the whole area is scanned and the object image can then be unwrapped for detailed inspection. 

3D systems consist of multiple cameras or laser sensors to capture the 3D view of an object. During the training process, the object or the cameras need to be moved to capture the entire product. Recent technology can produce accuracy within micrometers. 3D systems produce higher resolution but are also more expensive.

Emerging MV startups and new innovations
Tech giants, including IBM, Intel, Qualcomm, and NVIDIA, have publicly discussed investments in MV. In addition, many startups are developing new MV solutions such as Airobotics , Arcturus Networks, Deep Vision AI , Hawk-Eye Innovations, Instrumental, lending AI, kinara, Mech-Mind, Megvii, NAUTO, SenseTime, Tractable, ViSenze, Viso, and others. Some of these companies have been able to raise funding in excess of $1 billion.

In transportation, insurance companies can use MV to scan photographs and videos of scenes of accidents and disasters for financial damage analysis. Additionally, AI-based MV can power safety platforms to analyze driver behavior.

In software, computer vision platforms can be created without the knowledge of coding. Other startups have developed the idea for MV authentication software. And in the field of sports, AI, vision, and data analysis could provide coaches the ability to understand how decisions are made by players during a game. Also, one startup devised a cost reduction idea for surveillance by combining AI and MV in unmanned, aerial drone design.

Both MV and AI are changing quickly, and will continue to increase in performance, including precision and accuracy, while high GPU and ML power will come down in cost, propelling new MV applications.

Arteris’ Nightingale noted there will be further improvements in accuracy and speed. “Machine vision systems will likely become more accurate and faster. This will be achieved through advancements in hardware, such as sensors, cameras, and processors, as well as improvements in algorithms and machine learning models,” he said, pointing to an increased use of deep learning, as well. “Deep learning has been a significant driver of progress in machine vision technology in recent years, and it is likely to play an even more substantial role in the future. Deep learning algorithms can automatically learn data features and patterns, leading to better accuracy and performance. There will be an enhanced ability to process and analyze large amounts of data, as machine vision technology can process and analyze large amounts of data quickly and accurately. We may also see advancements in machine vision systems that can process significantly larger datasets, leading to more sophisticated and intelligent applications.”

Further, MV and AI are expected to integrate with other technologies to provide additional high-performance, real-time applications.

“Machine vision technology is already integrated with other technologies, such as robotics and automation,” he said. “This trend will likely continue, and we may see more machine vision applications in health care, transportation, and security. As well, there will be more real-time applications. Machine vision technology is already used for real-time applications, such as facial recognition and object tracking. In the future, we may see more applications that require real-time processing, such as self-driving cars and drones.”

MV design challenges
Still, there are challenges in training an MV system. Its accuracy and performance depend on how well the MV is trained. Inspection can encompass parameters such as orientation, variation of the surfaces, contamination, and accuracy tolerances such as diameter, thickness, and gaps. 3D systems can perform better than 1D or 2D systems when detecting cosmetic and service variation effects. In other cases, when seeing an unusual situation, human beings can draw on knowledge from a different discipline, while MV and AI may not have that ability.

“Some of today’s key challenges include data flow management and control – especially with real-time latency requirements such as those in automotive applications — while keeping bandwidth to a minimum,” said Alexander Zyazin, senior product manager in Arm‘s Automotive Line of Business. “In camera-based systems, image quality (IQ) remains critical. It requires a hardware design to support ultra-wide dynamic range and local tone mapping. But it also requires IQ tuning, where traditionally subjective evaluation by human experts was necessary, making the development process lengthy and costly. The new challenge for MV is that this expertise might not result in the best system performance, as perception engines might prefer to see images differently to humans and to one another, depending on the task.”

In general, machines can do a better job when doing mundane tasks over and over again, or when recognizing an image with more patterns than humans can typically process. “As an example, a machine may do a better job recognizing an anomaly in a medical scan than a human, simply because the doctor may make a mistake, be distracted or tired,” said Thomas Andersen, vice president for AI and machine learning at Synopsys. “When inspecting high-precision circuits, a machine can do a much better job analyzing millions of patterns and recognizing errors, a task a human could not do, simply due to the size of the problem. On the other hand, machines have not yet reached the human skill of recognizing the complex scenes that can occur while driving a car. It may seem easy for a human to recognize and anticipate certain reactions, while the machine may be better in ‘simple’ situations that a human easily could deal with, but did not due to a distraction, inattention or incapacitation – for example auto stop safety systems to avoid an imminent collision. A machine can always react faster than a human, assuming it interprets the situation correctly.”

Another challenge is making sure MV is secure. With cyberattacks increasing constantly, it will be important to ensure no production disruption or interference from threat actors.

“Security is critical to ensuring the output of MV technology isn’t compromised,” said Arm’s Zyazin. “Automotive applications are a good example of the importance of security in both hardware and software. For instance, the information processed and extracted from the machine is what dictates decisions such as braking or lane-keep assist, which can pose a risk to those inside the vehicle if done incorrectly.”

Conclusion
MV designs include a mixture of chips (processors, memories, security), IPs, modules, firmware, hardware and software. The rollout of chiplets and multi-chip packaging will allow those systems to be combined in novel ways more easily and more quickly, adding new features and functions and improving the overall efficiency and capabilities of these systems.

“Known good die (KGD) solutions can provide cost and space efficient alternatives to packaged products with limited bonding pads and wires,” said Tetsu Ho, DRAM manager at Winbond. That helps improve design efficiency, provides enhanced hardware security performance, and especially time-to-market for product launch. These die go through 100% burn-in and are tested to the same extent as discrete parts. KGD 2.0 is needed to assure end-of-line yield in 2.5D/3D assembly and 2.5D/3D multichip devices to realize improvements in PPA, which means bandwidth performance, power efficiency, and area as miniaturization, driven by the explosion of technologies such as edge-computing AI.”

This will open new options for MV in new an existing markets. It will be used to support humans in autonomous driving, help robots perform with precision and efficiency in manufacturing, and perform surveillance with unmanned drones. In addition, MV will be able to explore places that are considered dangerous for humans, and provide data input and analysis for many fields, including insurance, sports, transportation, defense, medicine, and more.

Go to the original article...

Image Sensors World        Go to the original article...

[(July 5, 2023): There is a strong suspicion that this is a machine generated article and so its veracity is questionable.]

Experts Predict Stunning Growth for Global Image Sensor Market, Reaching USD 55.8 Billion by 2032

Go to the original article...

Image Sensors World        Go to the original article...

A recent preprint titled "Practical Entropy Accumulation for Random Number Generators with Image Sensor-Based Quantum Noise Sources" by Choi et al. is available here:  https://www.preprints.org/manuscript/202306.1169/v1

 Abstract: The efficient generation of high-quality random numbers is essential in the operation of cryptographic modules. The quality of a random number generator is evaluated by the min-entropy of its entropy source. Typical method used to achieve high min-entropy of the output sequence is an entropy accumulation based on a hash function. This is grounded in the famous Leftover Hash Lemma which guarantees a lower bound on the min-entropy of the output sequence. However, the hash function based entropy accumulation has slow speed in general. For a practical perspective we need a new efficient entropy accumulation with the theoretical background for the min-entropy of the output sequence. In this work, we obtain the theoretical bound for the min-entropy of the output random sequence through the very efficient entropy accumulation using only bitwise XOR operations, where the input sequences from the entropy source are independent. Moreover we examine our theoretical results by applying to the quantum random number generator that uses dark noise arising from image sensor pixels as its entropy source.

Go to the original article...

Image Sensors World        Go to the original article...

Go to the original article...

Image Sensors World        Go to the original article...

Go to the original article...

Image Sensors World        Go to the original article...

	RADOPT 2023: Workshop on Radiation Effects on      Optoelectronic Detectors and Photonics Technologies 28-30 Nov 2023 Toulouse (France)

First Call for Papers

You are cordially invited to participate to the second edition of the RADECS Workshop on Radiation Effects on Optoelectronics and Photonics Technologies (RADOPT 2023) to be held on 28^th-30^th November 2023 in Toulouse, France.

After the success of RADOPT 2021, this second edition of the workshop, will continue to combine and replace two well-known events from the Photonic Devices and IC’s community: the “Optical Fibers in Radiation Environments Days -FMR” and the Radiation effects on Optoelectronic Detectors Workshop, traditionally organized every-two years by the COMET OOE of CNES.

The objective of the workshop is to provide a forum for the presentation and discussion of recent developments regarding the use of optoelectronics and photonics technologies in radiation-rich environments The workshop also offers the opportunity to highlight future prospects in the fast-moving space, high energy physics, fusion and fission research fields and to enhance exchanges and collaborations between scientists. Participation of young researchers (PhD) is especially encouraged.
Oral and poster communications are solicited reporting on original research (both experimental and theoretical) in the following areas:

• Basic Mechanisms of radiation effects on optical properties of materials, devices and systems
• Silicon Photonics, Photonic Integrated Circuits
• Solar Cells
• Cameras, Image sensors and detectors
• Optically based dosimetry and beam monitoring techniques
• Fiber optics and fiber-based sensors
• Optoelectronics components and systems

Abstract Submission and Decision Notification:

Abstracts for both oral and poster presentations can be submitted. The final decision will be taken by the RADOPT Scientific Committee.

·       Abstract submission open: Monday April 3^rd, 2023

·       Abstract submission deadline: Friday July 9^th, 2023

à Send abstract to clementine.durnez@cnes.fr

Industrial Exhibition

An industrial exhibition will be organized during RADOPT 2023. The location allows the exhibits to be located adjacent to the auditorium where the oral sessions will be delivered. Please contact us for more details.

Go to the original article...

Image Sensors World        Go to the original article...

Canon presentation on "PPA Optimization Using Cadence Cerebrus for CMOS Image Sensor Designs" is available here: https://vimeo.com/822031091

Go to the original article...

Image Sensors World        Go to the original article...

The IEEE International Conference on Computational Photography (ICCP) program is now available online: https://iccp2023.iccp-conference.org/conference-program/

ICCP is an in-person conference to be held at the Monona Terrace Conventional Center in Madison, WI (USA) from July 28-30, 2023.

Early registration ends June 22: https://iccp2023.iccp-conference.org/registration/

Friday, July 28th

09:00 Opening Remarks

09:30 Session 1: Polarization and HDR Imaging
1) Learnable Polarization-multiplexed Modulation Imager for Depth from Defocus
2) Polarization Multi-Image Synthesis with Birefringent Metasurfaces
3) Glare Removal for Astronomical Images with High Local Dynamic Range
4) Polarimetric Imaging Spaceborne Calibration Using Zodiacal Light

10:30 Invited Talk: Melissa Skala (UW-Madison)
Unraveling Immune Cell Metabolism and Function at Single-cell Resolution

11:00 Coffee break

11:30 Keynote: Aki Roberge (NASA)
Towards Earth 2.0: Exoplanets and Future Space Telescopes

12:30 Lunch; Industry Consortium Mentorship Event

14:00 Invited Talk: Lei Li (Rice)
New Generation Photoacoustic Imaging: From Benchtop Wholebody Imagers to Wearable Sensors

14:30 Session 2: Emerging and Unconventional Computational Sensing
1) CoIR: Compressive Implicit Radar
2) Parallax-Driven Denoising of Passively-Scattered Thermal Imagery
3) Moiré vision: A signal processing technology beyond pixels using the Moiré coordinate

15:15 Poster and demo Spotlights

15:30 Coffee break

16:00 Poster and demo Session 1

17:30 Community Poster and Demo Session

Saturday, July 29th

09:00 Invited Talk: Ellen Zhong (Princeton)
Machine Learning for Determining Protein Structure and Dynamics from Cryo-EM Images

09:30 Session 3: Neural and Generative Methods in Imaging
1) Learn to Synthesize Photorealistic Dual-pixel Images from RGBD frames
2) Denoising Diffusion Probabilistic Model for Retinal Image Generation and Segmentation
3) NeReF: Neural Refractive Field for Fluid Surface Reconstruction and Rendering
4) Supervision by Denoising

10:30 Invited Talk: Karen Schloss (UW-Madison)

11:00 Coffee break

11:30 Keynote: Aaron Hertzmann (Adobe)
A Perceptual Theory of Perspective

12:30 Lunch; Affinity Group Meetings

14:00 Invited Talk: Na Ji (UC Berkeley)

14:30 Session 4: Measuring Spectrum and Reflectance
1) Spectral Sensitivity Estimation Without a Camera
2) A Compact BRDF Scanner with Multi-conjugate Optics
3) Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation
4) Compact Self-adaptive Coding for Spectral Compressive Sensing

15:30 Industry Consortium Talk: Tomoo Mitsunaga (Sony)
Computational Image Sensing at Sony

16:00 Poster and Demo Spotlights

16:15 Coffee Break

16:45 Poster and Demo Session 2

18:15 Reception

Sunday, July 30th

09:00 Session 5: Depth and 3D Imaging
1) Near-light Photometric Stereo with Symmetric Lights
2) Aberration-Aware Depth-from-Focus
3) Count-Free Single-Photon 3D Imaging with Race Logic

09:45 Invited Talk: Jules Jaffe (Scripps & UCSD)
Welcome to the Underwater Micro World: The Art and Science of Underwater Microscopy

10:15 Coffee Break

10:45 Invited Talk: Hooman Mohseini (Northwestern University)
New Material and Devices for Imaging

11:15 Keynote: Eric Fossum (Dartmouth)
Quanta Image Sensors and Remaining Challenges

12:15 Lunch; Industry Consortium Mentorship Event

12:45 Lunch (served)

14:00 Session 6: NLOS Imaging and Imaging Through Scattering Media
1) Isolating Signals in Passive Non-Line-of-Sight Imaging using Spectral Content
2) Fast Non-line-of-sight Imaging with Non-planar Relay Surfaces
3) Neural Reconstruction through Scattering Media with Forward and Backward Losses

14:45 Invited Talk: Jasper Tan (Glass Imaging)
Towards the Next Generation of Smartphone Cameras

15:15 Session 7: Holography and Phase-based Imaging
1) Programmable Spectral Filter Arrays using Phase Spatial Light Modulators
2) Scattering-aware Holographic PIV with Physics-based Motion Priors
3) Stochastic Light Field Holography

16:00 Closing Remarks

Go to the original article...

Image Sensors World        Go to the original article...

From JCN Newswire: https://www.jcnnewswire.com/english/pressrelease/82919/3/NEC-develops-the-world&aposs-first-highly-sensitive-uncooled-infrared-image-sensor-utilizing-carbon-

Go to the original article...

Image Sensors World        Go to the original article...

Webinar on latest trends in High-Speed Camera, Introducing the BSI Camera Sensor.

Join this free tech talk by our expert speakers of Phantom High-Speed Cameras - Vision Research in which we explore the latest trends in high-speed cameras focusing on Backside Illuminated (BSI) sensor cameras and the associated benefits of improved processing speed and fill factor along with the challenges in such high-speed designs.

Webinar registration [link]

Date: 22nd June 2023
Time: 2:30pm IST / 2:00am Pacific / 5:00am Eastern

Topics to be covered:
Introducing the BSI sensor camera
Introducing FORZA & Sensor Insights
Introducing the MIRO C camera
Demo & display of the High-Speed Camera & its accessories.

Go to the original article...

Image Sensors World        Go to the original article...

An interesting combination of tech+art: https://bjoernkarmann.dk/project/paragraphica 

Paragraphica is a context-to-image camera that uses location data and artificial intelligence to visualize a "photo" of a specific place and moment. The camera exists both as a physical prototype and a virtual camera that you can try.

Will this put the camera and image sensor industry out of business? :)

Go to the original article...

Image Sensors World        Go to the original article...

By TechInsights Image Sensor Experts: Eric Rutulis, John Scott-Thomas, PhD

We first heard about it at IEDM 2021, and Sony provided more details at the 2022 IEEE Symposium on VLSI Technology and Circuits conference. Now it’s on the market and TechInsights has had our first look at the “world’s first” two-layer image sensor and we present our preliminary results here. The device was found in a Sony Xperia 1V smartphone main camera having a 48 MP, 1.12 µm pixel pitch and we can confirm it has dual photodiodes (a Left and Right photodiode in each pixel for full array PDAF). The die size measures 11.37 x 7.69 mm edge-to-edge.

In fact, the sensor actually has three layers of active silicon, with an Image Signal Processor (ISP) stacked in a conventional arrangement using a Direct Bond Interface (DBI) to the “second layer” (we will use Sony’s nomenclature when possible) of the CMOS Image Sensor (CIS). Figure 1 shows a SEM cross-section through the array. Light enters from the bottom of the image, through the microlenses and color filters. Each pixel is separated by an aperture grid (with compound layers) to increase the quantum efficiency. Front Deep Trench Isolation is used between each photodiode and it appears that Sony is using silicon dioxide in the deep trench to improve Full Well Capacity and Quantum Efficiency (this will be confirmed with further analysis).This layer also has the planar Transfer Gate used to transfer photocharge from the diode to the floating diffusion. Above the first layer is the “second layer” of silicon that contains three transistors for each pixel; the Reset, Amp (Source-Follower) and Select transistors. These transistors sit above the second layer silicon and connection to the first layer is achieved using “Deep contacts” which pass through the second layer essentially forming Through Silicon Vias (TSVs). Finally, the ISP sits above the metallization of the second layer, connected using Hybrid (Direct) Bonding. The copper of the ISP used for connection to the CIS DBI Cu is not visible in this image.

Key to this structure is a process flow that can withstand the thermal cycling needed to create the thermal oxide and activate the implants on the second layer. Sony has described the process flow in some detail (IEDM 2021, “3D Sequential Process Integration for CMOS Image Sensor”).

Figure 2 is an image from this paper showing the process flow. The first layer photodiodes and Transfer Gate are formed, and the second layer is wafer bonded and thinned. Only then are the second layer gate oxides formed and the implants are activated. Finally, the deep contacts are formed, etching through the second layer, and contacting the first layer devices.

Figure 2: Process flow for two-layer CIS described in “3D Sequential Process Integration for CMOS Image Sensor”, IEDM 2021.

The interface between the first and second layer is shown in more detail in Figure 3. The Transfer Gate (TG in the image) is connected to the first metal layer of the second layer. Slightly longer deep contacts lie below the sample surface and are partially visible in the image. These connect the floating diffusion node between the first and second layer. A sublocal connection (below the sample surface) is used to interconnect four photodiodes just above the first layer to the source of the Reset FET and gate of the AMP (Source-Follower) FET.

                    Figure 3: SEM cross-section detail of the first and second layer interface.

The sublocal connection is explored more in Figure 4. This is a planar SEM image of the first layer at the substrate level. Yellow boxes outline the pixel, with PDL and PDR indicating the left and right photodiodes. One microlense covers each pixel. Sublocal connections are indicated and are used to interconnect the Floating Diffusion for two pixels and ground for four pixels. The sublocal connection appears to be polysilicon; this is currently being confirmed with further analysis.

Figure 4: SEM planar view of the pixel first layer at the substrate level.

The motivation for the two-layer structure is multiple. The photodiode full well capacity can be maintained even with the reduced pixel pitch. The use of sublocal contacts reduces the capacitance of the floating diffusion, increasing the conversion gain of the pixels. The increased area available on the second layer allows the AMP (Source-Follower) transistor area to be increased, reducing noise (flicker and telegraph) create in the channel of this device.

It's worth taking a moment to appreciate Sony’s achievement here. The new process flow and deep contact technology allow two layers of active devices to be interconnected with an impressive 0.46 µm (center-to-center) spacing of the deep contacts (or Through Silicon Vias). Even the hybrid bonding to the ISP is just 1.12 µm; the smallest pitch TechInsights has seen to date. At the recent International Image Sensors Workshop, Sony described an upcoming generation that will use “buried” sublocal connections embedded in the first layer and pixel FinFets in the second layer (to be published). Perhaps we are seeing the first stages of truly three-dimensional circuitry, with active devices on multiple layers of silicon, all interconnected. Congratulations, Sony!

TechInsights' first Device Essentials analysis on this device will be published shortly with more analyses underway.

Access the TechInsights Platform for more content and reports on image sensors.

Go to the original article...