Uncrewed Systems Technology 046

43 Video systems | Focus Integrated AI With the increasing popularity of AI accelerators, the technology is also being integrated into the chips alongside the encoders. One controller now includes a proprietary AI engine called a Dynamically Reconfigurable Processor (DRP-AI) alongside an ARM Cortex-A55 1.2 GHz processor core and a video encoder that can handle 1080 video at 30 frames per second with H.264 coding. The AI engine reduces processing time and power consumption by pre-processing images to reduce the CPU’s workload, and can accelerate multiple algorithms in a single application and offload the main processor for specialised tasks. New configurations can be dynamically loaded into the engine in 1 ms. The AI engine provides both real-time AI inference and image processing functions with the capabilities essential for camera support such as colour correction and noise reduction. This enables customers to implement AI- based vision applications without requiring an external image signal processor. Uncrewed Systems Technology | October/November 2022 2, was drafted in 2013. It provides better data compression of 25-50% over H.264 at the same level of video quality, or much improved video quality at the same bit rate. It supports resolutions up to 8192 x 4320, including 8K UHD. While AVC uses the integer DCT with blocks of 4 x 4 and 8 x 8 pixels, HEVC uses variable block sizes between 4 x 4 and 32 x 32 depending on the complexity of the image and the motion estimation. This provides more compression for simpler images that don’t change quickly, and so reduces the overall data rate. The primary changes for HEVC include the expansion of the pattern comparison and difference-coding areas from 16 x 16 pixel to sizes up to 64 x 64, with an additional filtering step called sample adaptive offset filtering. Effective use of these improvements requires much more signal processing capability for compressing the video, but has less impact on the amount of computation needed for decompression. H.266 Versatile Video Coding (VVC), also known as H.266, ISO/IEC 23090- 3 or MPEG-I Part 3, was finalised in July 2020. As with AVC and HEVC, it aims to improve the compression performance and support a broad range of applications. VVC has about 50% better compression for the same image quality than HEVC, which would take an SD feed down to around 500 kbit/s, although it also has support for lossless and subjectively lossless compression. It supports resolutions ranging from very low resolution up to 4K and 16K with frame refresh rates of up to 120 Hz. It also supports more colours (YCbCr 4:4:4, 4:2:2 and 4:2:0) with 8-10 bits per component, a wide colour gamut and high dynamic range with peak brightness of 1000, 4000 and 10000 nits. Work on high bit-depth support (12 and 16 bits per component) began in October 2020 and is ongoing. However, VVC has yet to be widely adopted in hardware for robotic systems. That is mainly because the complexity of the encoder is 10 times that of HEVC, so an encoder would have to be more than 10 times larger or 10 times faster, and would consume 10 times more power in the same technology as the HEVC encoder. Two generations of chip-making technology will increase the density of transistors by a factor of four, and increase the speed by a factor of two, enabling an H.266 encoder in a suitable power envelope. This could be possible in a leading-edge chip- making process, although the cost of the chips would be prohibitive. The RZ/V2L includes an AI engine for video processing (Courtesy of Renesas Electronics)