Issue 40 Unmanned Systems Technology October/November 2021 ANYbotics ANYmal D l AI systems focus l Aquatic Drones Phoenix 5 l Space vehicles insight l Sky Eye Rapier X-25 l FlyingBasket FB3 l GCS focus l AUVSI Xponential 2021

38 Focus | AI systems neural networks would consume 1 W for the 4 TOPS, compared with 4 W for a more general-purpose DSP A driverless car operating at SAE Level 4 for example would require 1000 MAC units with 32 TOPS of performance. That is 20 times the performance of the latest GPU devices. The same architecture can scale up to 32,000 MAC units for higher performance by adding more cores. The advantage with this approach is that the neural networking accelerator can be added in a chip close to a camera, Lidar or radar sensor. Having the multicore approach also allows cores that are not being used to be shut down to reduce the overall power consumption. That is not possible with a large, general-purpose MAC array. AI in sensors Neural networks are not just used for image analysis though. The pattern recognition capability of a neural network can be used to improve the quality of the data from a sensor or radio transceiver instead of using a filter. This can significantly improve the accuracy of sensor systems and wireless links. Using AI for inertial navigation provides an improvement of five to 10 times over Kalman in sensor error tracking. This involves replacing the Kalman filter with a neural network inference engine that is trained using the motion data. Rather than having a linear estimate of motion from the filter, a neural network allows for a full non-linear estimate for higher accuracy. This can identify and correct gyroscope and accelerometer bias as well as scaling errors by accurately tracing the position, velocity and orientation. The trick for the sensors is to constrain the data for the inference engine by using a physics engine. That reduces the computing resources required and so provides a more deterministic capability. This doesn’t however allow the inference engine to resolve data that falls outside the physical limits. This approach also makes use of the real-time data in the sensor as well as the trained data. The framework can be updated with the data collected via a second, separate training engine to help improve the accuracy over time. By using the physics engine to constrain the inference engine, a three- layer neural network can run on a 200 MHz microcontroller with a floating- point unit (to provide the MAC blocks). This has been applied initially to an IMU for ground and air vehicles, but with a different set of training data and a different physics engine it can also be used for underwater odometry and in airborne radar systems. The determinism and reliability are the biggest challenges, to ensure that the inference engine can respond in a set amount of time, typically 0.6 ms. If the inference engine takes that long, the system defaults back to a linear physics model similar to a Kalman filter calculated in real time. Any slight inaccuracy is recovered in the next cycle. This can also enable the health of the sensors to be monitored by tracking the confidence of the data in a similar way to a Kalman filter. By looking at the co-variances of the data and how accurate they are versus how accurate the measurements are, it can identify any drift in the sensors. However, training the network takes a lot of data and time – one system October/November 2021 | Unmanned Systems Technology Scalable cores are being developed to handle machine learning networks (Courtesy of Cadence Design Systems/Tensilica) Reinforcement learning is a key element in an unmanned air traffic management system handling thousands of UAVs from multiple operators (Courtesy of Airwayz)