Perception software runs in sensors of autonomous vehicles
Believed to be the first commercially available, 2D/3D perception system designed to run in the sensors of autonomous vehicles, AEye will showcase its adaptive sensing platform at CES 2020 (7 to 10 January). It combines deterministic and artificial intelligence (AI) -driven perception to classify speed and range, motion forecasting and collision avoidance capabilities.
Its advent means that basic perception can be distributed to the edge of the sensor network. This allows autonomous designers to use sensors to not only search and detect objects, but also to acquire, and ultimately to classify and track these objects. The ability to collect this information in real-time both enables and enhances existing centralised perception software platforms, says AEye, by reducing latency, lowering costs and securing functional safety.
A perception system at the sensor level can potentially deliver more depth, nuance and critical information than with a 2D image-based system, says the company, for improved prediction for advanced driver assistance systems (ADAS) and autonomous vehicles.
This in-sensor perception system is based on AEye’s flexible iDAR platform that enables intelligent and adaptive sensing. The iDAR platform is based on biomimicry and replicates the perception of human vision through a combination of lidar, fused camera and AI. It is the first system to take a fused approach to perception – leveraging iDAR’s Dynamic Vixels, which combine 2D camera data (pixels) with 3D lidar data (voxels) inside the sensor, explains AEye. The software-definable perception platform allows for disparate sensor modalities to complement each other, enabling the camera and lidar to work together to make each sensor more powerful, while providing “informed redundancy” for functional safety.
Delivering perception at speed and at range has been a challenge for the autonomous vehicle industry. The reliability of detection and classification has to be improved, while extending the range at which objects can be detected, classified and tracked. The sooner an object can be classified and its trajectory accurately forecasted, the more time the vehicle has to brake, steer or accelerate in order to avoid collisions.
Rather than trying to capture as much data as possible, which requires time and power to process, second generation autonomous vehicle systems collect, manage and transform data into actionable information. The iDAR platform allows for applications ranging from ADAS safety augmentation, such as collision avoidance, to selective autonomy (highway lane change), to fully autonomous use cases in closed-loop geo-fenced or open-loop scenarios.
Engineers can now experiment using software-definable sensors without waiting for the next generation of hardware. They can adapt shot patterns in less than a second and simulate impact to find the optimal performance. They can also customise features or power usage through modular design, for instance using a smaller laser and no camera to create a specialised ADAS system, or mixing and matching short and long range lidar with camera and radar for more advanced, cost-sensitive 360 degree systems. Unlike with the industry’s previous generations of sensors, OEMs and Tier 1s can now also move algorithms into the sensors when it is appropriate, advises AEye.
AEye’s system more quickly and accurately searches, detects and segments objects and, as it acquires specific objects, validates that classification with velocity and orientation information. This enables the system to forecast the object’s behaviour, including inferring intent. By capturing better information faster, the system enables more accurate, timely, reliable perception, using far less power than traditional perception solutions, explains AEye.
The iDAR platform will be available via a software reference library which includes identification of objects (e.g. cars, pedestrians) in the 3D point cloud and camera. The system accurately estimates their centroids, width, height and depth to generate 3D bounding boxes for the objects.
It will also classify the type of objects detected to understand the motion characteristics of these objects. The segmentation function will further classify each point in the scene to identify specific objects those points belong to. This is especially important to accurately identify finer details, such as lane divider markings on the road.
Tracking objects through space and time helps keep track of objects that could intersect the vehicle’s path.
For range and orientation, identifying where the object is relative to the vehicle, and how it’s oriented relative to the vehicle helps the vehicle contextualise the scene around it.
Leveraging the benefits of agile lidar to capture the speed and direction of the object’s motion relative to the vehicle provides the foundation for motion forecasting, which is where the object will be at different times in the future. This helps the vehicle to assess the risk of collision and charter a safe course.
AEye’s iDAR software reference library will be available in Q1 2020.
