Why this diversity of onboard cameras? “Fisheye cameras are used to observe obstacles close to the vehicle,” Liu answers. “But then, the 30° FoV camera is a long-range device, primarily detecting traffic lights far away.” Lastly, the 100° FoV cameras give a good combination of vision width and range for complete perception coverage of the minibus’ surroundings. The radar at the front bumper is a 77 GHz unit, capable of distinguishing and classifying at least 120 different objects in typical operation, and of detecting objects up to 20 m away across a 60° horizontal FoV, or up to 70 m away with a 45° FoV, as well as up to 250 m when such objects are dead ahead. “Admittedly, we don’t use the radar that much – if the forward cameras and Lidars should somehow both fail, then we fall back on the radar as a last resort,” Liu comments. Perception model WeRide’s sensor fusion algorithms perform both low-level fusions (where raw data is fused) and mid-level fusions (where the detections are fused). The main body of the algorithmic model fuses the cameras’ vision map to the Lidar point cloud, generating a unified output, which reconciles all raw data gathered. “But L4 autonomy takes more than that, including the ability to define obscured objects that we can’t fully see, so our output models also feature image segmentation [AI recognition of multiple overlapping objects, such as cars in traffic], which heavily leverages the fusions of mid-level data,” Liu explains. The perception model has a 40 ms standard latency, but is also capable of latencies as low as 10 ms, thanks to an emergency ‘shortcut pipeline’ in which the sudden appearance of a sizeable obstacle at very close range – particularly at a blind junction, a blind turn or when driving among cyclists where one might fall off their bike in front of the bus – will trigger a control signal to the powertrain (including brakes and steering). That mode skips the detailed image segmentation or any other AI analytics, such as object classification, because in such instances alerting the Robobus that ‘something’ is ahead (and that safety actions are therefore imperative) takes priority over identifying and categorising what that something is. The perception model also boasts an object contour error of less than 5 cm, which Liu attributes to not only switching from the 64-channel model to the 128-channel model as the main Lidar, but also to WeRide’s exhaustive commitment to ever-better data labelling. WeRide’s engineers worked in Python to train the neural networks vital to achieving their models, and to construct their data processing and analytics flows. C++ and CUDA were used for the onboard deployment and offboard acceleration of the algorithms. “We continuously used our C++- based ‘data swarm’ platform to simulate perception results – and analyse how they changed – with different versions of the algorithm,” Liu recounts. “For the AI software, we used PyTorch, TensorFlow, TensorRT – all standard tools used across the industry, we’re fairly confident. And our cloud-based IDE [integrated development environment] and analytics system, hosted on our own internet data centre, was key to the engineers’ debugging work.” The WeRide analytics interface combines a left window composed of six camera FoVs (three forward-facing, three rearward) with a right window showing a top-down, 2D, map-like view of the Robobus (represented by a blue polygon that Liu and his colleagues call the “ego”, always at the centre) within its perceived surroundings, to play comprehensive recordings of sensor-fused data across the two, all synchronised for timing and position data. “The data collected by the sensors of the vehicles will be desensitised automatically by removing all personal information, such as the licence plate number or human face of traffic participants outside the vehicles accidently captured by the cameras, through an onboard masking programme,” Liu explains. “After desensitisation, the safety driver selects the best durations of the desensitised data and uploads them to the cloud, where the IDE pipeline processes it into our analytics database for playback and analysis to train our models.” Reliability for progress volz-servos.com Work with the leading manufacturer of electromechanical actuators for UAV. Volz Servos – provider of trusted, reliable and certifiable parts for your AAM. DA 15-N-HT
RkJQdWJsaXNoZXIy MjI2Mzk4