Topic
Automotive
EVS’ prowess in high performance embedding and image processing, specializing in FPGA and ASIC design technology, finds good examples in the deep strategic relationship with Seeing Machines, which extends back to 2015. EVS has significantly contributed to advanced key processing solutions for the Seeing Machines’ FOVIO Chip family, including the recently launched Occula Neural Processing Unit, that is highly optimized for enabling low-cost, high-performance OMS/DMS applications.
“Seeing Machines partners with the world’s best technology and IP companies to complement and enhance areas of expertise critical to our business. EVS is a technology company with which we have a long-standing technical relationship. Over the years that our companies have worked together, we have built trust and mutual respect between our technical teams and this is the foundation that has allowed Seeing Machines to share some of it’s most sensitive IP with EVS, enabling our teams to work closely together to build highly optimized silicon designs, such as the Occula NPU. EVS offers world-class competencies in state-of-the-art low-power vision processing and deep-edge embedding. These competencies, led by Roberto Marzotto and Marco Monguzzi, have aided Seeing Machines’ goal of providing the best value and highest price-vs-performance DMS solution IP to our customers. The Occula-R8 series of NPU designs were a major outcome of our partnership with EVS.“
― Timothy Edwards
Co-Founder, Seeing Machines
Problem
Revolutionizing Road Safety
In an era where technology and safety constantly intertwine, one innovation stands out—Driver Monitoring Systems (DMS) and Occupant Monitoring Systems (OMS). These systems are the new frontier in maintaining road safety and enhancing vehicle intelligence. With DMS observing driver behavior and OMS extending this scrutiny to all occupants and activities within the cabin, these features promise a leap in safety standards. But, the challenge lies in maintaining regulatory Key Performance Indicators (KPIs) at lowest cost to have a market advantage.
A Driver Monitoring System (DMS) uses a driver-facing camera to evaluate the presence and the state of the driver. It watches the face and eyes of the driver for signs that indicate risk due to distraction or fatigue and sends signals to the vehicle safety executive, which manages the risk typically through context-specific visual, haptic and auditory alerts.
For example, in conditionally automated vehicles, for the vehicle to control steering, the driver is required to pay sufficient attention to the road environment for safety reasons. In these systems a vision-based DMS notifies the driver if they begin to lose their focus on this task, and the driver is typically politely advised to pay more attention, perhaps with a vibration and amber flashing warning signals. However if the driver closes their eyes for more than a few seconds, representing an extreme risk, a harsh audio alert is typically issued to shock the driver and some vehicles may then lock out the automation function until the vehicle is pulled over.
DMS systems are now considered crucial for modern road safety and industrial safety groups like the Euro NCAP (New Car Assessment Program), as well as other regulatory groups worldwide are now requiring DMS technology incorporation into new vehicles as soon as 2023.
Today, Occupant Monitoring Systems (OMS) are seen as the next evolution of DMS. OMS systems extend beyond the detection and tracking the face of a single human (the driver) to whole of human body tracking of all individuals and objects in the cabin environment.
Some vehicles are now using video sensing over the whole cabin enabling them to provide new useful signals to the vehicle (e.g. detecting if anyone is smoking, or using a phone) and to improve the intelligence of other safety systems (e.g. seat belt detection, occupancy detection). The number of potential features provided by a high-end OMS system is expected to be more than thirty and with the need to monitor and track more than one individual, the processing task increases significantly over DMS solutions. This change also comes at a time when fitment rates, as a result of regulations will go to 100% of new vehicles.
Overall, the demand for ultra-efficient, very-low cost, specialized DMS and OMS processing solutions is sky-rocketing and the DMS or OMS solution that is able to meet regulatory KPIs at the lowest cost will have a market advantage.
In readiness for this time, in the early 2010s, our customer decided to capitalize on its DMS/OMS know-how to scale down a solution for the automotive industry to capture opportunities in a rapidly expanding market. This is where a challenging journey for EVS as an embedded technology developer began!
Solution
Building the DMS/OMS pipeline from the ground up
The technical strategy our customer has pursued, was to create its offer for the automotive industry based on a key decision: building the DMS/OMS pipeline from the ground up; accurately crafting the algorithms stack, neural network models and Neural Processing Unit (NPU) accelerators in lock-step, with research and engineering teams working closely together to build a single heavily optimized hardware and software “system”.
Added value
Implementation of a full-custom NPU
EVS’ expertise in high-performance embedding, specializing in FPGA and optimized neural-network design for embedded systems, has significantly contributed to implementing the envisioned OMS/DMS solution for the automotive industry and supported our customer to win additional market share.
Initially EVS was involved in the process of porting world-class DMS algorithms, running on ARM or sometimes prototypes running on x86 PC platforms, to a variety of embedded systems. In several iterations, we developed custom FPGA accelerators, as result of the software/hardware partitioning, to push algorithm performance on low-cost SoCs (System on Chip). After that, we broadened our responsibilities to taking the lead role on the detailed digital design and implementation of a full-custom NPU and its related toolchain to meet challenging efficiency requirements set by our customer.
The NPU is specialised to the task of human detection and tracking, enabling low-cost, high-performance applications for human-machine interfaces. More recently, we now also support our customer’s efforts to develop new embedded algorithms from scratch, to support the many new features required by their OMS/DMS platform.
Project partner
Our customer is Seeing Machines Ltd, a global company headquartered in Canberra, Australia, delivering safety technology to transport industries across the world. EVS and Seeing Machines have been working together since 2015.
Seeing Machines is a world leader in in-cabin safety monitoring, offering the world’s most efficient and optimized vision-based, AI driven, cabin-monitoring technologies, supplied to car makers worldwide. Its flagship embedded Driver Monitoring Engine (eDME) software platform is accelerated by the Occula-R8.0, R8.1 and 8.2® series of NPUs, which EVS co-designed working closely with Seeing Machines machine-learning experts. Occula® is licensed as design IP to semiconductor companies for integration into dedicated ASICs to yield highly competitive price-vs-performance DMS products that are able to meet regulatory requirements as well as pass the up-and-coming Euro NCAP tests for these systems.