Changes

The HTML5 specification introduces the <audio> and <video> media elements, and with them the opportunity to dramatically change the way we integrate media on the web. The current API provides ways to play and get limited information about audio and video, but gives no way to programatically access or create such media. We present a new enhanced API for these media elements, as well as a working Firefox prototype, which allows web developers to read and write raw audio data. We will demonstrate examples of how our audio data API can be leveraged to improve web accessibility, analyze audio streams, make in-browser synthesizers and instruments, process digital signals, create audio-based games, and drive animated visualizations; in tandem, we will also explore the code necessary for web developers to work with audio data in Javascript, and implement various audio algorithms, for example, Fast Fourier transform (FFT). Finally, we will entertain further possibilities that such an API would provide, such as text to audiospeech, audio speech to text analysis, "seeing" 3D using sound, etc.