Oct 232010
 

Pixel Bender can be a great tool for number crunching, but has a couple of limitations when trying to create an audio mixer.

  • The number of inputs is limited when using PixelBender toolkit to compile .pbj files.
  • Even if one overcomes the limitation by writing Pixel Bender assembly, the track count is fixed to a predetermined number determined by the number of inputs you have in the .pbj.

In some cases you might want your application to act as a normal audio app where you can add as many tracks as your CPU can handle, but not use more CPU than needed for the current track count.

What we need to accomlish that is a way to dynamically create a shader with the desired number of inputs. James Ward has created pbjAS, a library that enables you to create Pixel Bender shaders at runtime, based on Nicolas Cannasees haXe library.
Also, Tinic Uro has posted Pixel Bender assembly code to create a mixer.

So I set out to use pbjAS to recreate Tinics code with a dynamic number of channels.
The result is this class:

package com.blixtsystems.audio
{
	import flash.display.Shader;
	import flash.display.ShaderJob;
	import flash.utils.ByteArray;
	import pbjAS.ops.OpAdd;
	import pbjAS.ops.OpMul;
	import pbjAS.ops.OpSampleNearest;
	import pbjAS.params.Parameter;
	import pbjAS.params.Texture;
	import pbjAS.PBJ;
	import pbjAS.PBJAssembler;
	import pbjAS.PBJChannel;
	import pbjAS.PBJParam;
	import pbjAS.PBJType;
	import pbjAS.regs.RFloat;
	/**
	 * Shader to mix audio with a dynamic number of channels
	 * @author leo@blixtsystems.com
	 */
	public class MixerShader
	{
		private var _bufferSize:int;

		private var _pbj:PBJ = new PBJ();
		private var _shader:Shader;
		private var _buffer:Vector.<ByteArray> = new Vector.<ByteArray>();

		private var _numTracks:int;

		/**
		 * Constructor
		 * @param	numTracks	track count
		 */
		public function MixerShader(numTracks:int, bufferSize:int=2048)
		{
			_numTracks = numTracks;
			_bufferSize = bufferSize;
		}

		/*-----------------------------------------------------------
		Public methods
		-------------------------------------------------------------*/
		/**
		 * Mix audio
		 * @param	data	ByteArray in which to store the result, probably SampleDataEvent.data
		 */
		public function mix(data:ByteArray):void
		{
			var mixerJob:ShaderJob = new ShaderJob(_shader, data, 1024, _bufferSize/1024);
			mixerJob.start(true);
		}

		/*-----------------------------------------------------------
		Private methods
		-------------------------------------------------------------*/
		private function assembleShader():void
		{
			var channels:Array = [PBJChannel.R, PBJChannel.G];
			var chanStr:String = "rg";
			_pbj.version = 1;
			_pbj.name = "SoundMixer";
			_pbj.parameters =
			[
				new PBJParam
				(
					"_OutCoord",
					new Parameter
					(
						PBJType.TFloat2,
						false,
						new RFloat(	0, channels)
					)
				)

			];
			_pbj.code =
			[
				new OpSampleNearest
				(
					new RFloat(1, channels),
					new RFloat(0, channels),
					0
				),
				new OpMul
				(
					new RFloat(1, channels),
					new RFloat(3, channels)
				)
			];
			var i:int;
			for (i = 0; i < _numTracks; i++)
			{
				_pbj.parameters.push
				(
					new PBJParam
					(
						"track" + i,
						new Texture(2, i)
					)
				);
			}
			for (i = 0; i < _numTracks; i++)
			{
				_pbj.parameters.push
				(
					new PBJParam
					(
						"volume" + i,
						new Parameter
						(
							PBJType.TFloat2,
							false,
							new RFloat(i + 3, channels)
						)
					)
				);
			}
			for (i = 0; i < _numTracks-1; i++)
			{
				_pbj.code.push
				(
					new OpSampleNearest
					(
						new RFloat(2, channels),
						new RFloat(0, channels),
						i+1
					),
					new OpMul
					(
						new RFloat(2, channels),
						new RFloat(i+4, channels)
					),
					new OpAdd
					(
						new RFloat(1, channels),
						new RFloat(2, channels)
					)
				);
			}
			_pbj.parameters.push
			(
				new PBJParam
				(
					"output",
					new Parameter
					(
						PBJType.TFloat2,
						true,
						new RFloat(1, channels)
					)
				)
			);

			var pbjBytes:ByteArray = PBJAssembler.assemble(_pbj);
			_shader = new Shader(pbjBytes);
			_buffer = new Vector.<ByteArray>(_numTracks);

			// initialize the shader inputs
			for (i = 0; i < _numTracks; i++) {
				_buffer[i] = new ByteArray();
				_buffer[i].length = _bufferSize * 4 * 2;
				_shader.data["track" + i]["width"] = 1024;
				_shader.data["track" + i]["height"] = _bufferSize / 1024;
				_shader.data["track" + i]["input"] = _buffer[i];
				_shader.data["volume" + i]["value"] = [1, 1];
			}

		}

		/*-----------------------------------------------------------
		Getters/Setters
		-------------------------------------------------------------*/
		public function get numTracks():int { return _numTracks; }
		public function set numTracks(value:int):void
		{
			// needs to be at least one input, and no point reassembling pbj if track count has not changed
			if (_numTracks < 1 && _numTracks == value) return;

			_numTracks = value;
			assembleShader();
		}

		public function get buffer():Vector.<ByteArray> { return _buffer; }

	}

}

To use it you need to download pjbAS from James Ward and include the swc in your project.

Then in your audio engine do the following:

static public const BUFFER_SIZE:int = 2048;

// number of tracks (minimum 1)
private var _numTracks:int = 1;
// instantiate mixer shader
private var _mixerShader:MixerShader;
_mixerShader = new MixerShader(_numTracks, BUFFER_SIZE);

// here you have your sound objects stored
// adding or removing objects will change number of tracks in the shader
public var sounds:Vector.<Sound> = new Vector.<Sound>();

// SampleDataEvent callback
private function samplesCallback(e:SampleDataEvent):void
{
	_numTracks = sounds.length;

	// update number of track for the shader, causing pbj to recompile
	_mixerShader.numTracks = _numTracks;

	// extract audio data into the shader buffers
	for (var i:int = 0; i <  _numTracks; i++)
	{
		_mixerShader.buffer[i].position = 0;
		sound[i].extract(_mixerShader.buffer[i], BUFFER_SIZE);
		_mixerShader.buffer[i].position = 0;
	}
	// do shader job
	_mixerShader.mix(e.data);
}

According to Tinic Uro, the number of inputs in a shader is limited to 15, so adding more tracks than that will probably not work.

In my tests, playing back eight tracks using pure AS3 will often cause CPU peak usage over 16% of one 2.3 GHz core on my quad-core machine with the debug version of the Flash Player.
If I instead use Pixel Bender the peaks are rarely above 6%. Adding or removing tracks which causes the pbj to recompile, does not cause any noticeable spikes in CPU usage.
So using Pixel Bender will cut CPU usage to around 1/3 on my machine!

Big thanks to Tinic Uro, Nicolas Cannasse and James Ward who made it easy to accomplish!

Share/Bookmark
May 212008
 

Yes, a Flash equalizer. We all heard it mentioned before…for some reason the word equalizer has been adopted to mean spectral analyser when mentioned along with Flash.

 

An equalizer will process the sound boosting or attenuating frequencies while a spectral analyser will only show you the spectral content of the audio.
Now with the new functionality of Flash Player 10 I hope this means that people will call things by their correct name since we now will have applications with both analysers and equalizers.

 

So with that off my chest, here is the code for an actual equalizer.

The algorithm is taken from this c++ snippet on musicdsp.org and might not be the highest fidelity but should be fairly efficient.

 

You can also view this online example. (requires Flash Player 10)

May 092008
 

I added a little application to the download section.
It takes a Gregorian date and converts it to a dreamspell date and then calculates the kin, guide, analog and antipode to generate the affirmation for that day.
For those of you not familiar with the dreamspell calendar it’s a reinterpretation of the Mayan calendar by Dr. José Argüelles.
It’s based around 20 “glyphs” and 13 “tones” which make up the 260 day year called the Tzolkin.
The idea behind it is to create a new calendar system that actually is in tune with natural cycles, unlike the Gregorian system we use today, but mostly it’s used to cast horoscopes.
Here is one resource for more information about the calendar.

My wife wanted to have an neat application for displaying the affirmation of the day and to look up a persons birth “kin”, so I put this one together for her:

If this sort of stuff interest you are free to hotlink to the swf using the following URL:

http://www.resonantearth.com/ingrid/dreamspell.swf

You can download the swf here.

To get the source visit the download page.

Oct 242005
 

I got a request for it and decided to put up my scrollbar for anyone who might be interested.
It’s not using the V2 framework and is designed to be lightweight but flexible.

Features:
Only 4k uncompressed.
Can scroll one or several movie clips, text fields or buttons.
Responds to mousewheel, arrow keys and page up/down keys.
Configurable easing.
Fairly easy to alter style by simply editing the component.
Drag area that resizes with the amount of content to scroll.
Can update when changing the size of the content to be scrolled.

So basically just another scrollbar, but if you are looking for something less bloated then the MM V2 components and just want to do some scrolling it might be useful.

Get the download here.

UPDATE:
After doing some testing with several instances on multiple levels I noticed some issues.
I have now updated the download so you can use several instances as well as load them on top of each other and still have the mousescroll and keyboard control working fine.

To update the scrollbar reinstall the component using the extensions manager by double clicking the .mxp file.
Then drag an instance of the component onto the stage in the .fla with an old version. When prompted choose to replace existing items.
You can now delete the newly added instance, and your previous instances will be up to date.

UPDATE 2006-02-12:
An error in ScrollFocus.as made the key and mouse scroll stop working if you removed the first instantiated scrollBar.
The .zip file have been updated with the amended class file.

Oct 072005
 

I’ve done a little update to my system for enabling the back button and providing deep linking/bookmarking functionality for Flash sites and applications.
It’s now using Macromedias JS integration kit for Flash to JS communication.

Click here to download it.

Try the system here.

I was hoping I would solve issues with some browsers that it was not working with previously, but since there is more issues than if the browser supports the setVariable command to pass data from JS to Flash it seems like it will be very difficult to get a few browser/OS combinations to work.

Basically I found two different methods of approach.

One is using Rober Penners hidden frame method.
That’s still what the system does for IE, but using an iFrame instead instead of a regular frame set and removing the need to create actual HTML files to load into the frame by using document.write().
With FF I did not have much luck with that approach.
Just using JS to write into the hidden frame worked fine and made the back button work, but when trying to also update the hash to enable the bookmarking everything started going wrong.

The other approach is to at regular intervals check the location.hash and if it’s been updated cause a change of state in Flash.

Both approaches have various problems on different browsers.
For example Safari apparently will not record a history event when you use document.write() to update an iFrame.
Opera 8 will not record a history event if you update the hash, and after experimenting a bit with writing into the iFrame I gave up on getting it to work with that method.

It would be great with a system that works for every perceivable browser and OS, but as long as the system fails gracefully it’s at least not doing any harm to implement and as it is now it will cover a vast majority of users.

So far I tested and confirmed it working with FF 1.0.6 and 1.5 beta 1, IE6, NS8 and Opera 8.5
Opera 8.01 does not show the hash and back button fails, but at least deep links are working.

I would be very interested to hear from mac users in particular to find out if NS, Safari and IE still is not working.

Sep 092005
 

I was looking around for some example code for how to implement full state management in Flash with both functioning back/forward buttons and deeplinking/bookmarking, but couldn’t find any.

So I decided to put together my own and make it available in an attempt to silence the many Flash critics who, rightfully in my opinion, think that it is a major drawback with pure Flash sites.

It’s only tested and working with recent versions of firefox and IE on windows.
Backbutton and bookmarking fails with Opera 8, but deeplinking works.
I’m quite sure someone with decent JS skills should be able to fix that though.
The problem is that “document[movieid].SetVariable();” does not target the swf correctly, which is a mystery to me since I have seen examples using the same syntax that does work.
Another solution would be to rewrite the code to use localConnection instead in which case most browsers should have no problem.

But for now at least you can easily have state management functioning for the very large percentage of users that use FF or IE, and other browsers should hopefully all fail gracefully and be able to follow deeplinks in to your flash site.
So there is no excuse to now implement state management into your Flash site now.

You can try the system with this little ugly example
Please leave a comment if you are using a browser other than FF or IE on windows so I can make a list of what browsers it is working on.

Get the download here.

Update:
I made a new version using Macromedia JS integration kit for communication between Flash and JS.
Get the download here.
It did not solve the issues with Opera 8.01 apart from getting rid of the JS error, and I doubt that Safari will work still.
So until I get some feedback on how it works with the Browsers that been problematic I’m not sure how much of an improvement it is.
I updated the example above, so try it out and tell me if it’s not working with a particular browser.
So far confirmed to be working is FF 1.0.6 & 1.5 beta1, IE 6, Opera 8.5 and NS8. Not working so far is Opera 8.01

Aug 262005
 

BlixtBlog is a simple GNU GPL licensed blog system for flash that can also be used as a basic CMS system for a whole flash site.
It uses PHP/MySQL for the backend.

Main features:

  • Admin interface for easy adding of text, images and links and moderation of user comments.
  • Included state management enabling deeplinking, bookmarking and use of back/forward buttons in the browser.
  • Accordion style display of posts by subject and post by month as well as multiple categories of external links.
  • Generates RSS 2.0 feeds.
  • User comments with gravatar support.
  • Can be set to resize with the browser window or be used in fixed size.
  • User can resize text using ctrl mouse scroll or +/- buttons.
  • The package includes an example file of how to use the system for easy creation of a flash site manageable using your browser.
    Just configure what pages you like to have on your site and add your own graphics.

    Currently there is no HTML output generated, but that will be added in next release to enable search engines to find your content and deeplink directly into your flash page.
    Other planned additions is slideshow for photos and mp3 player.

    Get the download here.


    Click here to see an example
    of the system in use for generating a whole site.
    Also if you feel like testing making comments that is the prefered place to do it instead of posing a lot of asdsfdawasdf in here :)

    If you have any problems, discover any bugs or like to provide feedback please use the comment function on this page.

    EDIT:
    2005-09-01
    Now version 0.9.3 is up.
    It includes some very essential fixes, and you a strongly recommended to upgrade if you are running 0.9.2

    Apart from many major bugfixes a few new features was added:
    Possible to use embedded fonts.
    Display and loading of posts and comments split to pages.
    Ability to double up as a guestbook.

    Switch to our mobile site