Uncrewed Systems Technology 049 - April/May 2023

39 designed. Instead of one large, expensive sensor to capture a wide field of view, research projects for L5 self-driving vehicles are showing that using an array of smaller, lower-cost sensors is practical, tapping into the existing ecosystem of AI algorithms. This array approach also gives a product designer more options. The driverless cars used for research have large sensor hubs protruding from the vehicle to house a wide range of sensors including the image sensor, Lidar and radar. As the technology moves tomass adoption, themultiple sensors will need to be integrated into the body of the vehicle, and styling becomes as important a design factor as the integration with the rest of the electronics in it. Lens This shift to the styling requirements has a major impact on lens technology. The pixel technology stays roughly the same as for the ADAS sensors, but there is demand for more pixels in the sensor array, going up from 8 MP now to 12 MP to give higher resolution. A higher resolution allows smaller objects to be detected at a greater distance, givingmore time for theML algorithms tomake an identification and allowing the self-driving system to operate safely at higher speeds. There is no set ‘perfect’ resolution, though – the required resolution is measured against specific use cases that address specific problems such as detecting a certain size of obstruction at a certain speed or distance. However, this higher resolution in turn requires a larger lens, which increases the size of the camera. At the same time, there is demand from vehicle designers for a smaller lens, of 0.25 in or so, to fit in with the styling, rather than the current 0.5 in. That creates a challenge for the light- gathering capabilities, as less light is captured through a smaller lens. That in turn drives demand for sensor arrays with larger, more sensitive pixels, leading to a more expensive sensor or reducing the resolution with fewer pixels. That trade-off between higher resolution and larger pixels is still being explored, using sensitivity as a metric, where the challenge is to shrink the pixel size but keep the low-light performance. The key here is ensuring a large enough dynamic range, as the pixel has to avoid becoming saturated in bright light yet still be sensitive to low light. Current high-volume image sensors for ADAS have a resolution of 8.3 MP using 2.1 µm 2 pixels. These are increasing in volume shipments as cameras on the windshield for ADAS Level 3 systems, and are now being used for L4 and L5 applications as an array. The arrays can be rectangular, for example 12 sensors in a 4 x 3 array, each with their own lens, or a ‘belt’ of sensors in a 12 x 1 array to give 360o visibility. Shutters The move to reduce costs also has an impact on the type of shutter used. Rolling shutters, where the image is acquired row by row, is more cost- effective than global shutter technology, which reads every row simultaneously into a memory buffer. The increase in performance of the pixel readout means rolling shutter sensors are now fast enough to be used with ML processing, where the ML algorithm is trained specifically using the output of the rolling shutter sensor. With a rolling shutter the sensor data is read out row by row, so additional memory is not required in the sensor, reducing the cost. However, that means the top rows of the sensor image are sensing light before the bottom ones, so a vertical post or stop sign can appear tilted. ML networks are a way to compensate for that though, by mapping the original image to the output of the sensor to compensate for the blur. Fast motion can result in blurring as a result of using a rolling shutter, but higher performance ML algorithms can increasingly compensate for this. Image sensors | Focus Higher resolution allows objects to be detected at a greater distance, allowing the self- driving system to operate safely at higher speeds Uncrewed Systems Technology | April/May 2023 Placing image sensors around autonomous vehicles is key to cutting the number of road accidents and deaths (Courtesy of onsemi)