Unmanned Systems Technology 027 l Hummingbird XRP l Gimbals l UAVs insight l AUVSI report part 2 l O’Neill Power Systems NorEaster l Kratos Defense ATMA l Performance Monitoring l Kongsberg Maritime Sounder

83 Chip-level monitoring Tapping into chip-level diagnostics is a recent development for performance monitoring. Here, small monitors are added to key functional blocks and have their own comms network. This adds up to 1% of the die area of the SoC, and so far has been used for debugging the chip. Once the blocks are designed into the chip they gather lots of fine-grained data to understand what’s going on in the chip. As the blocks are still present in production it makes sense to use them to provide details of the chip’s performance. This data can be sent out via the comms links to the chip, whether it be Ethernet, USB3 or PCI Express. It can then be delivered to the vehicle operator or even the operator’s customer. That is opening up new ways of managing data. There is also the option of an on- chip analytics subsystem. This is a ring-fenced ML capability that could be pre-programmed to monitor specific parameters, such as throughput and latency, which will vary from application to application such as the length of the queue of data and response times. This means the embedded modules can watch for a specific event or spot any trends, and raise an alarm if necessary. Another advantage is that all the debug data is also available, by using a small memory buffer on the chip and a larger data store off it. The debugging and monitoring hardware for this on-chip monitoring could take up 1-1.5% of the chip die area. Having the monitoring on-chip improves the response time, allowing any issues to be detected more quickly. In tests on an SoC controlling a camera sensor, on-chip monitoring identified non- responsive pixels in a few microseconds. Taking the data off-chip for post- processing in software took up to 100 ms. The technology is being used in a driver assistance ADAS chip that will be available later this year. Other automotive controllers using the technology are due to be launched next year. On-chip neural networks Using on-chip ML starts with capturing enough representative data about the phenomenon being modelled. This usually involves placing sensors on or near the object being monitored in order to record its state and any changes over the time, for example the temperature or vibration. Chip makers now provide software and hardware development tools that help to capture and label the data. An example here would be a sensor subsystem that includes motion and environmental sensors, a microcontroller, an SD card connector and Bluetooth connectivity. This subsystem labels the data for ‘supervised learning’, where the data sets have to be characterised so that the different outputs can be classified correctly. These classified data sets are called the ‘ground truth’ and will be used to train and then validate the neural network. The system developer must decide on the type of topology the artificial neural network (ANN) should have in order to be able to best learn from the data and provide useful output for the target application. The developer will typically use popular off-the- shelf deep-learning frameworks to design and train ANN topologies. Training an ANN involves passing the data sets through the neural network in an iterative manner so that the network’s outputs can minimise any error on the output. This is usually done on a computer with virtually unlimited memory and computational power, to allow many iterations in a short period of time. The result is a pre-trained ANN. The next step is to embed the trained ANN into a microcontroller. The software tools allow fast, automatic conversion of trained ANNs into optimised code that can run on a microcontroller. This can be integrated with low-level drivers, middleware libraries and Performance monitoring | Focus Unmanned Systems Technology | August/September 2019 Monitoring hardware added to a system-on-chip can be used to assess the condition of the chip (Courtesy of UltraSoC)