AI & 3D Sensing: Voice for Non-Verbal Performers
- Creating a successful opera typically involves talented performers, funding, advertising, and compelling music.
- over seven years, librettist Brenda Shaughnessy, composer Paola Prestini, and directors Jerron Herman and Jay Scheib collaborated to create "Sensorium Ex." The opera, which premiered in Omaha, Nebraska,...
- Luke Dubois, Associate Professor of Integrated Design & Media at New York University’s Tandon School of Engineering, plays a crucial role in this fusion.His work at the Ability...
“Sensorium Ex” in Omaha is revolutionizing opera by masterfully blending 3D sensing, AI, and disability solutions. This groundbreaking production spotlights speech-challenged artists, offering them a voice through assistive technology. The opera, a collaboration spanning seven years, highlights the power of combining technological innovation with artistic expression. The project features collaborations with experts like Luke dubois, who uses speech synthesizers too enhance interaction for non-verbal performers. These advancements, initially inspired by pioneers like Ray Kurzweil and Stephen Hawking, bring about profound impact. News Directory 3 shares how this innovative approach could change the landscape for theater and beyond. Consider its potential for physical therapy and mainstream performances.Discover what’s next for art and technology.
“Sensorium Ex” Opera Blends Tech, Talent, and Disability Solutions
Updated June 17, 2025
Creating a successful opera typically involves talented performers, funding, advertising, and compelling music. However,”Sensorium Ex” took a different path,focusing on a mother and her non-verbal,non-ambulatory son. The production featured a cast of predominantly disabled performers, many with speech impairments.
over seven years, librettist Brenda Shaughnessy, composer Paola Prestini, and directors Jerron Herman and Jay Scheib collaborated to create “Sensorium Ex.” The opera, which premiered in Omaha, Nebraska, in May 2025, merges 3D sensing, artificial intelligence, and disability solutions to deliver a groundbreaking performance. This innovative approach showcases the power of assistive technology and artistic creativity.
Luke Dubois, Associate Professor of Integrated Design & Media at New York University’s Tandon School of Engineering, plays a crucial role in this fusion.His work at the Ability Project supports assistive technology and adaptive design for people with disabilities. Dubois’ background in music, electronics, and speech synthesizers enables him to create solutions that blend art, engineering, and design.
Speech synthesizers, initially invented by Ray Kurzweil in 1975, used Optical Character Recognition to help visually impaired individuals “listen” to written text. Intel and stephen Hawking later advanced this technology, using keyboard text input and eye movements to communicate. While effective, the output often sounded robotic. “Sensorium Ex” addresses this issue by using state-of-the-art technology to bring the voices of Kitsune to life,making the performance more impactful.
“As an artist involved in the creation of Sensorium Ex, I can say with assurance that it is indeed nothing short of unusual,” said Hailey McAvoy, who plays Kitsune’s Mother. “The opera itself is unlike anything I have brought to life on stage… and they will carry an impact far beyond the production of this piece.”
Jakob, one of the leads playing Kitsune, said the opera is “life-changing… Everyone [is] so purposeful in their interactions… In the opera, everyone matter[s]. I am honored to perform alongside so many vibrant personalities in this beautiful composition.”
Jennifer Jordan, Jakob’s mother, added, “It’s been the most beautiful, life-affirming experience to watch him be seen for all that he truly is… Sensorium Ex is personal for me. I am that mother that believed for so long that there was so much more to my son than what the eye could see.”
What’s next
The success of “Sensorium Ex” raises questions about the future of art and technology. Could mainstream productions like “The phantom of the Opera” or “Hamilton” incorporate similar talent and assistive performance enhancements? The technology demonstrated in this opera could extend to other areas,such as physical therapy and rehabilitation,enhancing the quality of life for many.