Jul 152008
 

With the new beta 2 version of Flash Player 10 the Sound API has changed a bit.

Most of the changes is very straight forward and only renaming of some methods, events and properties:

SamplesCallbackEvent is now SampleDataEvent
samplesCallback is now sampleData or Event.SAMPLE_DATA
Instead of accessing Sound.samplesCallbackData to write to the output buffer one now use the SampleDataEvent.data.

Tinic Uro has updated his examples if you like to look at some example code with the new API.

So basically very straight forward changes but there was one thing that I managed to waste a bit of time on. This is what the documentation say about the buffer size for the SampleDataEvent:

Provide between 2048 and 8192 samples in a SampleDataEvent object. For best performance, provide as many samples as possible. The fewer samples you provide, the more likely it is that clicks and pops will occur during playback. This behaviour can differ on various platforms and can occur in various situations – for example, when resizing the browser. You might write code that works on one platform when you provide only 2048 samples, but that same code might not work as well when run on a different platform. If you require the lowest latency possible, consider making the amount of data user-selectable.

If you provide fewer than 2048 samples, Flash Player plays the remaining samples and then stops the sound as if the end of a sound file was reached, generating a SoundComplete event.

All my code was using a buffer size of less than 2048 and hence did not work any more.
I’m not sure why the available range has been made smaller since it was nice to be able to lower the latency by having a 512 samples buffer.
I guess it is a common problem that Flash developers are a bit irresponsible with the users CPU power and I can understand that they want to avoid the reputation of being a CPU hog, but IMHO it’s a bit like if they would have decided that the maximum frame rate should be limited to 20FPS.

Share/Bookmark
May 302008
 

Playing around with the new Flash player 10 audio processing functionality the need for optimization becomes very apparent when you want to apply effects to several tracks of audio.

With a sample rate of 44100 and a dozen stereo tracks we are talking over a million samples to be processed per second where each process you apply will have probably at least some 30 operations. All of a sudden the great performance of AVM2 becomes quite limiting.

So it’s important to squeeze out every drop of performance you can by optimizing the code.
First of all I have been benchmarking the performance of running code inside the processing loop, in a function, in an external class and inside a loaded swf (would have been neat for the possibility to plug in effects without recompiling the main swf).

The code I used for testing was to process a value and return it like this (obviously without function enclosure when doing the processing in the local scope):

public function calculate(num:Number):Number
{
	return num * 1.01;
}

The time needed in ms when calling the function 10 000 000 times:

  • Locally: 46
  • Calling a function in the same class: 213
  • Calling a function in a separate class: 213
  • Calling a function in an externally loaded swf: 2347

Not so surprising results.
Having processing code in an external swf is obviously not an option. I tried with both simply sticking a function in the swf or in a class which I retrieved by applicationDomain.getDefinition and both methods performed equally bad.
Doing processing locally instead of in a separate function or class is a lot faster, but obviously that could easily becomes very cumbersome and ugly.
At least there is nothing lost on having the function in a separate class compared to having a function in the same class.

Something that does surprise me a bit is that when just calling the function once and having the loop inside the function instead the resulting time was 75ms.
That’s about 30 ms added for just one function call so it seems like the first call is a lot more expensive.

One would think that the conclusion is that the best approach when processing audio if one like to avoid placing the code inside the samplesCallbackEvent loop seems to be calling the processing code once and then iterate over the size of the buffer in the class for the effect.
This is exactly what I was suggested by Spender when I posted a 3-band EQ example.

The problem there and why my attempt at implementing his suggestion failed at making an improvement is that reading and writing floats in a ByteArray is slower than the function calls.
Testing to writeFloat 10 000 000 times to then read loop through them to read them again takes 1727 ms. So compared with the 213 ms doing the same amount of function calls it’s clear that function calls actually is comparatively cheap. A Vector fares a bit better then the ByteArray with 1239 ms.

So the optimal approach seems to be to only do samples.readFloat once then use the returned value doing function calls for each process you like to apply before you do samplesCallbackData.writeFloat

May 212008
 

Yes, a Flash equalizer. We all heard it mentioned before…for some reason the word equalizer has been adopted to mean spectral analyser when mentioned along with Flash.

 

An equalizer will process the sound boosting or attenuating frequencies while a spectral analyser will only show you the spectral content of the audio.
Now with the new functionality of Flash Player 10 I hope this means that people will call things by their correct name since we now will have applications with both analysers and equalizers.

 

So with that off my chest, here is the code for an actual equalizer.

The algorithm is taken from this c++ snippet on musicdsp.org and might not be the highest fidelity but should be fairly efficient.

 

You can also view this online example. (requires Flash Player 10)

May 192008
 

I have had reason to play around with some of the new functionality of Flash Player 10 and the vectors is just awesome.
On top of the benefits of the strict typing they are about 50% faster than Arrays according to my tests.

Being completely new to the concept, how to create multidimensional vectors was not completely obvious since you need to type every dimension when declaring the bottom level dimension:

var v1:Vector.>>= new Vector.>>();
var v2:Vector.> ;
var v3:Vector.;
for (var i:int = 0; i < 10; i++) {
v2 = new Vector.>();
for (var ii:int = 0; ii < 10; ii++) {
v3 = new Vector.();
for (var iii:int = 0; iii < 10; iii++){
v3[iii] = iii;
}
v2[ii]=v3;
}
v1[i]=v2;
}

 

So far I have mostly been experimenting with the new samplesCallbackData to create a little mixer.
It seems like one needs to bypass the mixer in the Flash Player if one want to write to the output buffer because creating several sounds and then doing samplesCallbackData.writeFloat() on them will not work.
Of course each channel doesn’t have its own output buffer, so you can only write to the master output.
The problem I’m having with this is that if one would like to have a level meter for each individal track I cannot figure out a way to determine what sample is currently being output.
Here is a simplified version of how I implemented the mixing in the SamplesCallbackEvent handler:

var i:int = 0, l:int = _sounds_vect.lengt;
while (i < l) {
samples = new ByteArray();
samples.position = 0;
snd = _sounds_vect[i];
snd.extract(samples, _bufferSize);
_samples_vect[i] = samples;
_samples_vect[i].position = 0;
i++;
}
while (c < _bufferSize) {
left = 0;
right = 0;
i = 0;
while (i < l) {
valL = _samples_vect[i].readFloat();
valR = _samples_vect[i].readFloat();
left += valL;
right += valR;
i++;
}
valL = left;
valR = right;
_out_snd.samplesCallbackData.writeFloat(valL);
_out_snd.samplesCallbackData.writeFloat(valR);
c++;
}

So the audio is mixed in chunks the size of the buffer and then written to the buffer using samplesCallbackData.writeFloat().

 

For the main output I can create a level meter using:

_out_chn = new SoundChannel();
_out_chn = _out_snd.play();
function onEnterFrame(e:Event):void {
_masterSlider.drawLevel(_out_chn.leftPeak, _out_chn.rightPeak);
}

 

But for the individal channels I will never issue a play() and hence cannot find a way to get a really consistently behaving level meter or spectrum.
I’m sure there is some clever way to do it that is escaping me.

 

What I currently do is to get the value in the mixing loop like so:

while (i < l) {
valL = _samples_vect[i].readFloat();
valR = _samples_vect[i].readFloat();
left += valL;
right += valR;
if (c == 0){ // tried with (c == _bufferSize - 1) and (c == _bufferSize/2) as well
_levelR_vect[i] = valL;
_levelL_vect[i] = valR;
}
i++;
}

I then use the _levelL_vect and levelR_vect values in my onEnterFrame handler to draw the bars, but the result is a lot less accurate than what is possible using my_chn.leftPeak and far from satisfactory.
I guess what I would need is a way to be able to tell what sample from the output buffer that is playing a certain moment in time.

 

Apart from that small issue it’s great to have the functionality to generate and process audio and I think we will see some very cool applications appearing eventually.

Switch to our mobile site