LogoLogo
  • Home
  • Projects
  • About
  • Contact

Digging into the Microphone in Flash Player 10.1

Devon O. · January 30, 2010 · Actionscript, Flash · 39 comments
5

One of the big (well, maybe it wasn’t all that big, but it was interesting) announcements of the 10.1 Flash Player was its ability to access detailed sound data from the microphone in the same way you can access mp3 data now. Until now, microphone access has been limited to volume. With the 10.1 player though, you can extract a bytearray of 512 sound samples just as you can from an mp3 file.

Generally speaking, the Microphone class, like the Sound class, now listens for the SampleDataEvent.SAMPLE_DATA event. When this is triggered, the event’s data property will hold a bytearray of 8192 samples. Those samples can then be “read into” a sound instance (similar to the way dynamic sounds are created) and the sound instance played. At that point, you can use the good ol’ SoundMixer.computeSpectrum() method to read the current sound samples into another bytearray and do with them as thou wilt.

Below is a quick example using a 3d grid of Papervision3D spheres. Just click to get started then speak/sing/whistle/holler into your mic (of course, you’ll need the 10.1 player installed and you’ll have to allow access to the microphone):

Get Adobe Flash player 10.1 Beta

Seeing as how there doesn’t seem to be any documentation out there for doing this yet, for anyone interested, here’s a very basic example to play around with:

C#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
package {
 
    import flash.display.Shape;
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.events.SampleDataEvent;
    import flash.media.Microphone;
    import flash.media.Sound;
    import flash.media.SoundMixer;
    import flash.utils.ByteArray;
 
    /**
     * Simple Microphone test for Flash Player 10.1
     * @author Devon O.
     */
 
    [SWF(width='550', height='450', backgroundColor='#000000', frameRate='40')]
    public class Basic extends Sprite {
 
        // sound
        private var _soundBytes:ByteArray = new ByteArray();
        private var _micBytes:ByteArray;
        private var _micSound:Sound;
        private var _lines:Vector.<Shape> = new Vector.<Shape>(512, true);
 
        private var _ctr:int;
 
        public function Basic():void {
            if (stage) init();
            else addEventListener(Event.ADDED_TO_STAGE, init);
        }
 
        private function init(event:Event = null):void {
            removeEventListener(Event.ADDED_TO_STAGE, init);
 
            initEqualizer();
            initMic();
            initSound();
 
            addEventListener(Event.ENTER_FRAME, drawLines);
        }
 
        private function initEqualizer():void {
            var holder:Sprite = new Sprite();
            for (var i:int = 0; i < 512; i++) {
                var line:Shape = new Shape();
                with(line.graphics) {
                    beginFill(0xFFFFFF);
                    drawRect(0, -300, 1, 300);
                    endFill();
                }
                line.x = i;
                line.scaleY = 0;
                holder.addChild(line);
                _lines[i] = line;
            }
            holder.y = 400;
            holder.x = stage.stageWidth * .5 - holder.width * .5;
            addChild(holder);
        }        
 
        private function initMic():void {
            var mic:Microphone = Microphone.getMicrophone();
            if ( mic ) {
                mic.setLoopBack(false);
                mic.rate = 44;
        mic.gain = 60;
                mic.addEventListener(SampleDataEvent.SAMPLE_DATA, micSampleDataHandler);
            }
            else {
               // no mic
            }
        }
 
        private function micSampleDataHandler(event:SampleDataEvent) :void {
            _micBytes = event.data;
            _micSound.play();
        }
 
        private function initSound():void {
            _micSound = new Sound();
            _micSound.addEventListener(SampleDataEvent.SAMPLE_DATA, soundSampleDataHandler);
        }
 
        private function soundSampleDataHandler(event:SampleDataEvent):void {
            for (var i:int = 0; i < 8192 && _micBytes.bytesAvailable > 0; i++) {
                var sample:Number = _micBytes.readFloat();
                event.data.writeFloat(sample);
                event.data.writeFloat(sample);
            }
        }
 
        private function drawLines(event:Event):void {
            SoundMixer.computeSpectrum(_soundBytes, true);
            if (_soundBytes.bytesAvailable) {
                _ctr = 0;
                while (++_ctr < 512) {
                    _lines[_ctr].scaleY = _soundBytes.readFloat();
                }
            }
        }
    }
}

EDIT:
Here is the above code compiled to .swf form, for those who’d like to check it out:

Get Adobe Flash player

  Facebook   Pinterest   Twitter   Google+
  • Singularity?
    January 13, 2008 · 1 comments
    1840
    2
    Read more
  • It’s Beginning to Look a Lot Like Unity…
    December 24, 2010 · 0 comments
    1667
    8
    Read more
  • Starling, Nape Physics, and PhysicsEditor
    November 25, 2012 · 11 comments
    8185
    15
    Read more
39 Comments:
  1. Can you please post the example as a swf? I can’t compile right now, but would like to see how it works. Thx

    Sinbu · February 10, 2010
  2. Can you tell me where I can download the flash sdk to compile this, I dont have the events.SampleDataEvent class.
    I am using eclipse and flex builder 3.

    Thor · March 10, 2010
  3. Hey, Thor, I used the 3.5 Flex SDK and the new Global SWC, that you can download from the 10.1 player page: http://labs.adobe.com/technologies/flashplayer10/ (oh, and FlashDevelop :) )

    Devon O. · March 11, 2010
  4. Thanks for that, I forgot about the new Global SWC.
    I am having another rather annoying problem now, everything is working except the eventlistner is just not being called
    mic.addEventListener(SampleDataEvent.SAMPLE_DATA, micSampleDataHandler)

    I have checked that the mic is not muted, and have tried with loopback both on and off, but it never gets called.
    What am I doing wrong? can you post a single mxml file for me to try?
    I am using flex builder 3 and eclipse 3 with flex sdk3.5

    Thor · March 11, 2010
  5. OK, as soon as you post something it works …

    Sorry for the post, but I have just tried it on firefox, chrome and it works fine, but it doesnt work on IE8 on my development machine for some reason. It works on IE8 on a different machine when I post it on the web.

    Thor · March 11, 2010
  6. Glad you got it (somewhat) sorted. Very strange it doesn’t work in IE8 on one machine, but does on another. But I guess I’ve seen stranger things..

    Thanks for the coffee!

    d.

    Devon O. · March 12, 2010
  7. Does this work without playing back the sound that is coming in? I’d like to computeSpectrum of the mic input but I don’t want to have to playback and double up the sound.

    knuckfubuck · March 23, 2010
  8. Hey knuck, You can actually play around with the bytearray returned from the Microphone directly. In the example above you can just use the _micBytes bytearray instance in the soundSampleDataHandler() method and not write them into a new bytearray or play the sound. The drawback to this is that instead of 512 bytes (256 left/right) you get 8192 bytes and they are not nice clean normalized values.

    Devon O. · March 24, 2010
  9. I would like to capture a few seconds of voice and send it back to my web server in the WAV format. Is that possible? Do I need a WAV encoder? Any help is much appreciated!

    David Law · April 13, 2010
  10. Hey David, not really sure how you’d get the data back to your web server, but it shouldn’t be impossible.. There’s a nice article on the adobe developer site about recording sound snippets in AIR: http://www.adobe.com/devnet/air/flex/articles/using_mic_api.html And Didier Brun did an AIR to mp3 app using AIR’s new access to native processes and LAME: http://www.bytearray.org/?p=1142 Those would be the places I’d start anyway.

    Good luck

    Devon O. · April 13, 2010
  11. Thank you, Devon. The first link seems to provide what I need. I’ll report back if I manage to get it work.

    David Law · April 15, 2010
  12. Cool. I might check it out this weekend if I have some time and see what I can come up with.

    Devon O. · April 15, 2010
  13. Just to update. I have done it with the help of the article below:
    http://www.adobe.com/devnet/air/flex/articles/using_mic_api.html

    which was kindly pointed out by Devon. Thanks!

    David Law · April 22, 2010
  14. I have a question. Does it require CS5 if you compile in the Flash IDE?

    I installed the Flash Player 10.1 and I see it works in the browser. But when I compile it in Flash Pro CS4 it doesn’t work. Guess I need Flash Pro CS5?

    Jim Andrews · May 05, 2010
  15. Hey, Jim. The main thing is that you have to be able to compile to the 10.1 Flash Player. Obviously Flash Pro CS4 can’t do that. So you can use Flash Pro CS5 or you can use the Flex 3.5 or Flex 4.0 SDK (when I wrote this post, I was using the Flex 3.5 SDK and the FlashDevelop IDE).

    Devon O. · May 06, 2010
  16. Hi All,

    I have a question I would like to develop a WEB Flash application which records the voice from user and saves it to user local file. I did it with Adobe Air but how I can transform it to WEB.

    Vahe Tomas · June 03, 2010
  17. hey Vahe – see this post: https://blog.onebyonedesign.com/?p=531

    Devon O. · June 03, 2010
  18. Thanks a lot, now it works quite well :)

    Vahe Tomas · June 06, 2010
  19. Computing the spectrum of the mic input.

    Hi,
    I notice that someone else has talked about computing the spectrum without playing it back, but I don’t quite understand the response.
    How can you use the compute spec on the mic input without playing it back? Anything I try just gives zeros unless I actually play the sound back.

    Thanks
    Thor

    Thor · June 13, 2010
  20. Hey Thor, I don’t believe you can use the computeSpectrum method on the raw microphone bytes (though I may be wrong), but you can access them without playing the sound back by going straight into the bytearray holding the microphone data. In the above script, the bytearray _micBytes holds the raw microphone data. You can dig into that bytearray without first writing them into a _soundbytes array and playing that back. The _micBytes bytearray can get messy though, so be careful of that.

    Devon O. · June 13, 2010
  21. Hi,
    Thanks for the reply, I should have made it a bit clearer.
    I have done a lot of successful processing on the mic data, and have managed to access the byteArray etc.

    The reason I want to use computeSpectrum is that it has a built in fft (spectrum calculator) that doesn’t seem to be anywhere else in flex. Of course I could write my own fft/spectrum function, but that would take time, and run much slower than the built in one. I want to use the computeSpectrum function if at all possible, but just playing the sound back to get the spectrum isn’t acceptable as the user of my software would get horribly confused as to why the speech is being played back to them at that time.
    (It is speech recognition/language teaching software and sometimes you don’t want to play back what they said, just analyse it)

    If adobe is smart enough to make a spectrum function, but stupid enough so it can’t be used sensibly then I will be a bit annoyed …

    Is it not possible to just call computeSpectrum on given array, only the currently playing output sound?

    It is very difficult writing speech recognition code in flex, and it runs very slowly, so anything to speed it up is very helpful.

    Thanks
    Thor

    Thor · June 14, 2010
  22. Hi Thor,

    I’ve never seen anyone using computeSpectrum on a ‘non playing’ sound – and seeing as how it’s the SoundChannel rather than the Sound object itself that calls the method, I doubt that it is possible. Thinking off the top of my head, it may be somehow possible to load a bytearray into a sound instance, then use extract() to get cleaner data. Other than that, there may be something out there created in pixelbender or alchemy that could quickly and efficiently normalize a large bytearray. If I run across anything, I’ll post it, but don’t know of any way right this moment.

    d.

    Devon O. · June 14, 2010
  23. I’ve made a Visualizer without playing the samples back.
    Just operrated on the raw microphone output.

    http://www.swfcabin.com/open/1282119401

    Dennis S. · August 18, 2010
  24. Very cool!

    Devon O. · August 18, 2010
  25. nice post!
    your example code doesn’t seem to work. would you be so nice to upload a .fla file with it? i’ld love to check it out.

    tom · October 25, 2010
  26. If you don’t want the mic sound played back you can mute the sound playback with the SoundMixer. Just drop this in one of the init functions: SoundMixer.soundTransform = new SoundTransform(0);

    Daniel · May 18, 2011
  27. I love this one! just what i needed!

    Steven · June 01, 2011
  28. Are you guys not getting crazy echoes and schreeching noises when testing any of the SWFs? The sound jumps back and forth between speaker and microphone and goes crazy :D OSX + Macbook Pro here.

    I cant seem to get it to work when setting the SoundTransform to 0 like daniel suggested. No errors msgs, just no lines drawn

    David L. · November 04, 2011
  29. Same here. I can’t get how to access the raw data from the microphone neither (fp 11.2).

    Stef · March 25, 2012
  30. Actually managed to get the mic raw data:

    package {

    import flash.display.Shape;
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.events.SampleDataEvent;
    import flash.media.Microphone;
    import flash.media.Sound;
    import flash.media.SoundMixer;
    import flash.media.SoundTransform;
    import flash.utils.ByteArray;

    /**
    * Simple Microphone test for Flash Player 10.1
    * @author Devon O.
    */

    [SWF(width=’550′, height=’450′, backgroundColor=’#000000′, frameRate=’40’)]
    public class Basic extends Sprite {

    // sound
    protected static const LINES:int = 256;
    private var _soundBytes:ByteArray = new ByteArray();
    private var _micBytes:ByteArray;
    private var _micSound:Sound;
    private var _lines:Vector. = new Vector.(512, true);

    private var _ctr:int;

    public function Basic():void
    {
    if (stage) init();
    else addEventListener(Event.ADDED_TO_STAGE, init);
    }

    private function init(event:Event = null):void
    {
    initEqualizer();
    init();
    }

    protected function init():void
    {
    _soundBytes = new ByteArray();
    var mic:Microphone = Microphone.getMicrophone();
    mic.setSilenceLevel(0, 4000);
    mic.addEventListener(SampleDataEvent.SAMPLE_DATA, micSampleDataHandler);
    function micSampleDataHandler(event:SampleDataEvent):void
    {
    var i:int = 0;
    while(event.data.bytesAvailable)
    {
    if(i >= LINES-1)
    {
    i = 0;
    return;
    }
    //var sample:Number = event.data.readFloat();
    //_soundBytes.writeFloat(sample);
    _lines[i].scaleY = event.data.readFloat() * 3;
    i++;
    }
    }
    }

    private function initEqualizer():void
    {
    var holder:Sprite = new Sprite();
    for (var i:int = 0; i < LINES; i++)
    {
    var line:Shape = new Shape();
    with(line.graphics)
    {
    beginFill(0xFFFFFF);
    drawRect(0, -400, 400/LINES, 400);
    endFill();
    }
    line.x = i * 400/LINES;
    line.scaleY = 0;
    holder.addChild(line);
    _lines[i] = line;
    }
    holder.y = 200;
    holder.x = stage.stageWidth * .5 – holder.width * .5;
    addChild(holder);
    }

    }
    }

    Stef · March 25, 2012
  31. Is there any way to get the actual frequency in numbers?

    vamapaull · July 04, 2012
  32. Interesting.. I’m not really sure if it makes sense to talk about frequency when dealing with voice. But I’m definitely no sound expert. Best info (well at least as a starting point) I could find is this stack overflow question: http://stackoverflow.com/questions/2361895/how-to-determine-the-frequency-of-the-input-recorded-voice-in-iphone

    Devon O. · July 05, 2012
  33. I’m trying to make a guitar tuner type of application. For that I need to know the exact frequency number in Hz. I tried a lot to find a good answer but no luck. Basically, I’m trying to convert that ByteArray info into a frequency number.

    vamapaull · July 05, 2012

Leave a Comment! Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Devon O. Wolfgang

AIR | Unity3D | AR/VR

Unity Certified Developer

Technical Reviewer of “The Essential Guide to Flash CS4 AIR Development” and “Starling Game Development Essentials”

Reviewer of “The Starling Handbook”

Unity Engineer at Touch Press.

Categories
  • Actionscript (95)
  • AIR (16)
  • Flash (99)
  • Games (7)
  • Liberty (13)
  • Life (53)
  • Shaders (20)
  • Unity3D (21)
Recent Comments
  • MainDepth on Unity Ripple or Shock Wave Effect
  • Devon O. on Unity Ripple or Shock Wave Effect
  • Feral_Pug on Unity Ripple or Shock Wave Effect
  • bavvireal on Unity3D Endless Runner Part I – Curved Worlds
  • Danielius Vargonas on Custom Post Processing with the LWRP
Archives
  • December 2020 (1)
  • December 2019 (1)
  • September 2019 (1)
  • February 2019 (2)
  • December 2018 (1)
  • July 2018 (1)
  • June 2018 (1)
  • May 2018 (2)
  • January 2018 (1)
  • December 2017 (2)
  • October 2017 (1)
  • September 2017 (2)
  • January 2017 (1)
  • July 2016 (1)
  • December 2015 (2)
  • March 2015 (1)
  • September 2014 (1)
  • January 2014 (1)
  • August 2013 (1)
  • July 2013 (1)
  • May 2013 (1)
  • March 2013 (2)
  • December 2012 (1)
  • November 2012 (1)
  • September 2012 (3)
  • June 2012 (2)
  • May 2012 (1)
  • April 2012 (1)
  • December 2011 (2)
  • October 2011 (3)
  • September 2011 (1)
  • August 2011 (1)
  • July 2011 (1)
  • May 2011 (2)
  • April 2011 (2)
  • March 2011 (1)
  • February 2011 (1)
  • January 2011 (2)
  • December 2010 (3)
  • October 2010 (5)
  • September 2010 (1)
  • July 2010 (2)
  • May 2010 (5)
  • April 2010 (2)
  • March 2010 (7)
  • February 2010 (5)
  • January 2010 (5)
  • December 2009 (3)
  • November 2009 (1)
  • October 2009 (5)
  • September 2009 (5)
  • August 2009 (1)
  • July 2009 (1)
  • June 2009 (2)
  • May 2009 (6)
  • April 2009 (4)
  • March 2009 (2)
  • February 2009 (4)
  • January 2009 (1)
  • December 2008 (5)
  • November 2008 (2)
  • September 2008 (1)
  • August 2008 (6)
  • July 2008 (6)
  • June 2008 (9)
  • May 2008 (4)
  • April 2008 (3)
  • March 2008 (4)
  • February 2008 (9)
  • January 2008 (7)
  • December 2007 (6)
Copyright © 2021 Devon O. Wolfgang