Edge AI vision system helps pilots improve safety
With higher levels of automation and the presence of new types of interfaces in modern cockpits, pilots are required to assimilate vast amounts of information, especially when dealing with atypical events. Improving human–machine interaction in the cockpit is crucial, as it helps to enhance pilots' situational awareness and reduce their workload. The EU PEGGASUS consortium, led by CSEM, aimed to achieve just this by combining real-time gaze detection and hand–gesture recognition.
CSEM used the latest artificial intelligence and computer vision techniques to create a robust vision system adapted to the extreme conditions that can occur during a flight. Tested with pilots in a flight simulator, the PEGGASUS system detects both gaze and hand gestures, showing promise for enhancing pilot efficiency, training techniques, and flight safety. To design a powerful, custom-made solution, CSEM collaborated with aviation experts from SERMA Ingénierie, Swiss International Air Lines, and ETH Zurich, who provided essential complementary skills to ensure the project's success.