Speech-to-text input stream

[ZhiyuAlexZhang]

What are you trying to do that you can't?

Wordplay requires learners to read and write code, which can pose challenges for people who speak a language fluently but cannot yet read or write it - common for oral-first languages, early language learners, or people with reading/writing disabilities. Wordplay needs basic speech input infrastructure that allows programs to receive and process spoken language as a data stream.

What is your idea?

The idea is to implement a Speech input stream using the Web Speech API that captures spoken words and makes them available as text values within Wordplay programs. By adding this piece of infrastructure, Wordplay programs can receive spoken input, which will enable many creative applications that respond to speech. It will also lay the groundwork for future NLP-based features and interactions.

[amyjko]

I'm excited to see this come together! A few core requirements to keep in mind:

• It should handle multiple languages
• There should be some degree of configurability (probably language, ability to reset the stream)
• There should be good error handling for the various things that can go wrong (e.g., no internet connection, rate limits, etc.)
• There will need to be documentation for this stream like there is for all of the other streams
• We should probably add an example to the built-in gallery showing a use case for this

As we talked about in the meetup, try thinking of a few different example projects for this to explore the possible configurability options that might be helpful.

[amyjko]

@ZhiyuAlexZhang How's this feature coming along?