Uncrewed Systems Technology 049 - April/May 2023

40 Focus | Image sensors A rolling shutter also allows multiple exposures to be combined into a single image. That improves the dynamic range and provides more metadata about the image that can be used by the ML algorithms. For example, the metadata can include the exposure data of specific areas of the image, with automatic white balance and a feedback loop for exposure control all implemented on the sensor chip. Taking one exposure from the sensor array currently provides 110 dB of dynamic range, but combining three or four exposures on the sensor can provide a range of 140 dB, rather than having to send multiple images from the sensor. That helps to cut the sensor’s bandwidth requirements, and performing the calculations on the sensor rather than in the AI chip helps to reduce the overall power consumption of the sensor system. The higher dynamic range is particularly important for situations such as detecting backlit traffic lights. If the pixel is approaching saturation it loses the ability to distinguish colours, particularly between yellow and red, so there is a benefit in increasing the dynamic range of a single frame by usingmultiple exposures. The next stage is to reduce the number of exposures to two with the same 140 dB dynamic range, by improving the pixel’s design so that it can be exposed for longer to absorb more light before it saturates. The current leading-edge pixel designs provide a dynamic range of 130 dB from a single frame. Improving the dynamic range will also help improve the efficiency and performance of future sensors when dealing with corner cases, those unusual situations that can confuse sensors when the vehicle emerges from a tunnel into bright light for example. The current state-of-the-art 8.3 MP 0.5 in CMOS digital image sensor has an array of 3848 x 2168 pixels with either the on-chip high dynamic range multiple exposure approach or a linear exposure output for off-sensor processing. The sensor also includes fault detection and embedded data that are designed to enable compliance with the ASIL B functional safety requirements. The device is programmable through a simple two−wire serial interface and supports the MIPI output interface. Agricultural and construction The cost reduction of sensor systems from consumer ADAS means the sensors are also being used in agricultural and construction equipment, taking advantage of the automotive temperature range of -40 to +85 oC. In this application, vehicle styling is less important than its ruggedness and ability to withstand harsh environments, which opens up the opportunity for larger lens designs. There is also a different trade- off for the resolution, as the speed of the vehicles is much slower but smaller objects need to be detected. Event-driven sensors Another type of image sensor is the event-driven sensor (EVS). Instead of outputting all the pixels in the sensor array, an EVS outputs only those that register a change. An EVS uses a different ML technique called spiking, or neuromorphic, neural networks that mimic the way the brain works. They have a lower power consumption and a faster response (lower latency) than a traditional CMOS sensor, as only the change beyond a set threshold, known as the temporal threshold, is registered. However, EVSs are not suitable for the main camera in a driverless vehicle on their own, as at high speed all the pixels are changing, but they are finding uses as April/May 2023 | Uncrewed Systems Technology This 8.3 MP sensor with rolling shutter has a dynamic range of 140 dB (Courtesy of onsemi) Another 8.3 MP sensor, this one also has a 140 dB dynamic range and features LED flicker mitigation and (Courtesy of Omnivision)