— PROJECT NAME

Zyrobotics LLC -

Readable Storiez: Counting Zoo

— ROLE

Project Lead

Lead Developer

Lead Gameplay Design

— DATE

2020

Working in cooperation with Microsoft using their Azure Cognitive Services, developed a speech recognition system to help children with improving communication skills.

Built upon the existing Counting Zoo app, the system uses the player’s recorded words with Microsoft’s speech recognition machine learning software to analyze and compare their language to the default model.

For the default speech model we took samples from several hundred volunteers with various accents along with regular cadence in order to have a comprehensive baseline. Using this, any specific difficulties are recognized and highlighted so that teachers and speech therapists can work with their students to target specific trouble areas.

With this information, an additional intervention game was developed to emphasize the trouble words, repeating the words how they are supposed to be pronounced.

With these additions, we also worked with Zyrobotics’s existing STEMDash cloud system to save speech data to a user’s account online that can then be used to track their progress. This system uses websocket to transmit the data as a JSON stream.

The app is currently live on the App Store and Google Play.