HTML5 Musical Balls

musical-balls

Wow its been over a year since I last blogged. Anyways, here is something fun I put together today. Its a bouncing balls collision animation. When the balls collide they play a familiar tune. Make sure you try it on your iPad or iPhone as the sounds work via the web audio api (on iOS6+) and tilting the screen will change the direction the balls are traveling.

Demo here.

Source code here.


HTML5 Collage Creator

Collage

This week I started playing around with colour analysis in html5. I used it to create an image collage of your Facebook profile from your Facebook friends photos. I was quite surprised at how well it preformed particularly on mobile. As its nearly Christmas I managed to role this experiment into an interactive Christmas app. You can check it out here.

Collage

Creating this experiment was surprisingly easy. Here is an explanation of the technique.

First you need to load all your photos you want to use to create the collage into a Canvas element. One annoying gotcha of Html5 is that you cannot write images from a different domain to the Canvas due to security limitations. To get around this I used a simple php proxy to load the images via the local domain. Once loaded you can then get the average colour of the photo. To get the average colour I re-wrote some Flash code I had for doing this and modified it slightly to work with Canvas pixels. The photos are then stored in an array along with their corresponding average colour.

Next I load your Facebook profile pic into the canvas. Instead of getting the average colour of this image, I loop through every pixel of the image and get the pixel colour. This colour is then matched with the closest average colour from my previous list. Again is some Flash code which I re-wrote. Then I use easeljs to create a larger canvas image. The image is redrawn where each pixel is represented by the a friend image with the closest matched average colour to the pixel colour.
Finally the zooming effect at the end is achieved using a CSS3 transition. Interestedly this transition runs much faster when apply to a canvas element then when applied to a image element.

If you are interested in the code for this experiment you can view the javascript on the site as its un-compressed. The colour functions are also available below.


function getCanvasAverageColour(canvas, width, height) {
    var context, data, r,g,b, len, color = {r:0,g:0,b:0};
    r=g=b=0;
    context = canvas.getContext('2d')
    try {
        data = context.getImageData(0, 0, width, height);
    } catch(e) {
        alert('security error, img on diff domain? '+e);
        return color;
    }
    len = data.data.length;
    for (var i = 0; i < len; i += 4) {
        r += data.data[i];
        g += data.data[i + 1];
        b += data.data[i + 2];
    }
    len/=4;
    color.r = ~~(r / len);
    color.g = ~~(g / len);
    color.b = ~~(b / len);
    return color;
}
function getCanvasPixelColors(canvas, width, height){
    var bmp = canvas.getContext('2d').getImageData(0, 0, width, height);
    var pixels = bmp.data;
    var colors = [];
    var i = 0;
    for (i = 0; i < pixels.length; i += 4) {
        colors[i / 4] = {r:pixels[i],g:pixels[i + 1],b:pixels[i + 2],a:pixels[i + 3]};
    }
    return colors;
}
function similarColor( rgb1, rgb2, tolerance)
{
    tolerance==null? tolerance=0.01 : tolerance

	tolerance = tolerance * ( 255 * 255 * 3 ) << 0;

	var distance = 0;

	distance += Math.pow( rgb1.r - rgb2.r, 2 );
	distance += Math.pow( rgb1.g - rgb2.g, 2 );
	distance += Math.pow( rgb1.b - rgb2.b, 2 );

	return distance <= tolerance;
}

Away3D BezierPatch

image warp

I’ve been playing around the BezierPatch primitive in away3D recently. It works in a similar way to the bezier surface i created. Its quite easy to create some very interesting effects. The source for this is on wonder.fl so fork it and make it your own.

BezierPatch away3D – wonderfl build flash online


Warping Bitmaps with a Bezier Surface

image warp

If you have read my previous posts on drawTriangles, you will see that I love warping images. I decided to take this a bit further by using a Bezier surface. A bezier surface or bezier patch can warp a grid by using a set of control points. The control points in this example are the red circles, the grid can be seen by clicking the show grid checkbox. Try dragging the controls points to create your own warp or just sit back and watch the show.
There is also a method for doing this using away3d plus checkout this 3d surface built using nurbs.

This movie requires Flash Player 10

Update:
You can view the bezier surface in action on this client project I worked on recently. The surface was used to create chewing mouth animations from user’s facebook photos.


Flash 10 drawTriangles

image warp

Adobe introduced some new functions in the flash 10 drawing api for 3d support. One of them is drawTriangles. This enables bitmaps to be drawn to a 3d mesh. But it can also be used in 2d for skewing and warp effects. In a nutshell it allows you to define a shape and then map points of a bitmap to this shape. Click on the examples below to see the points of the rectangle warp to new positions. Here I’m defining the five points which make up the triangles and tweening the outside corners. The number of triangles and possibilities are infinite. This is great alternative to the infamous flash ide shape tween.
You can find some even better examples on wonder.fl. Just try searching for drawTriangles. Two of my favs are wonderwall and genie effect.

Random Warp

Bezier Minimise

This movie requires Flash Player 10

source for random warp

var _vertices:Vector.<Number >  = new Vector.<Number >   ;
var _indices:Vector.<int >  = new Vector.<int >   ;
var _uvtData:Vector.<Number >  = new Vector.<Number >   ;
var _image:BitmapData = new Foxy();
var border:Number = 0;
var _width:Number = stage.stageWidth;
var _height:Number = stage.stageHeight;
//define the triangles
_vertices.push(0, 0);
_vertices.push(_width, 0);
_vertices.push(_width/2, _height/2);
_vertices.push(0, _height);
_vertices.push(_width, _height);
_indices.push(1, 0, 2);
_indices.push(0, 2, 3);
_indices.push(1, 2, 4);
_indices.push(2, 3, 4);

//define the bitmap mapping to triangle points
_uvtData.push(0, 0);
_uvtData.push(1, 0);
_uvtData.push(0.5, 0.5);
_uvtData.push(0, 1);
_uvtData.push(1, 1);

stage.addEventListener(MouseEvent.CLICK, onClick);
function onClick(e=null):void
{

	var endVertices:Vector.<Number >  = new Vector.<Number >   ;
	endVertices.push(randNum, randNum);
	endVertices.push(_width-randNum, randNum);
	endVertices.push(stage.stageWidth/2, stage.stageHeight/2);
	endVertices.push(randNum, _height-randNum);
	endVertices.push(_width-randNum, _height-randNum);
	TweenMax.to(_vertices, 0.5, {endVector:endVertices, ease:Expo.easeOut, onUpdate:update});
}
function update():void
{
	graphics.clear();
	graphics.beginBitmapFill(_image);
	graphics.drawTriangles(_vertices, _indices, _uvtData);
	graphics.endFill();
}


source for bezier minimise

var _timeLine: TimelineMax;
var _vertices:Vector.<Number >  = new Vector.<Number >;
var _indices:Vector.<int >  = new Vector.<int >;
var _uvtData:Vector.<Number >  = new Vector.<Number >;
var _image:BitmapData = new Foxy();
var _width:Number = stage.stageWidth;
var _height:Number = stage.stageHeight;
//define the triangles
_vertices.push(startLT.x, startLT.y);
_vertices.push(startRT.x, startRT.y);
_vertices.push(startC.x, startC.y);
_vertices.push(startLB.x, startLB.y);
_vertices.push(startRB.x, startRB.y);
_indices.push(1, 0, 2);
_indices.push(0, 2, 3);
_indices.push(1, 2, 4);
_indices.push(2, 3, 4);

//define the bitmap mapping to triangle points
_uvtData.push(0, 0);
_uvtData.push(1, 0);
_uvtData.push(0.5, 0.5);
_uvtData.push(0, 1);
_uvtData.push(1, 1);

update();
function update():void
{
	_vertices[0]=startLT.x;_vertices[1]=startLT.y;
	_vertices[2]=startRT.x;_vertices[3]=startRT.y;
	_vertices[4]=startC.x;_vertices[5]=startC.y;
	_vertices[6]=startLB.x;_vertices[7]=startLB.y;
	_vertices[8]=startRB.x;_vertices[9]=startRB.y;

	graphics.clear();
	graphics.lineStyle(1, 0xFFFFFF);
	graphics.beginBitmapFill(_image);
	graphics.drawTriangles(_vertices, _indices, _uvtData);
	graphics.endFill();

}

stage.addEventListener(MouseEvent.CLICK, onClick);

function onClick(e=null):void
{

	if(_timeLine == null){
		var time:Number = 0.7;
		var ease:Function = Expo.easeInOut
		_timeLine = new TimelineMax({onUpdate:update});
		_timeLine.appendMultiple([
		TweenMax.to(startLT, time, {delay:0, bezierThrough:[{x:curveLT.x, y:curveLT.y}, { x:endLT.x, y:endLT.y}], ease:ease}),
		TweenMax.to(startRT, time, {delay:0, bezierThrough:[{x:curveRT.x, y:curveRT.y}, { x:endRT.x, y:endRT.y}], ease:ease}),
		TweenMax.to(startC, time, {delay:0, x:endC.x, y:endC.y, ease:ease}),
		TweenMax.to(startRB, time, {bezierThrough:[{x:curveRB.x, y:curveRB.y}, { x:endRB.x, y:endRB.y}], ease:ease}),
		TweenMax.to(startLB, time, {bezierThrough:[{x:curveLB.x, y:curveLB.y}, { x:endLB.x, y:endLB.y}], ease:ease})
		]);
	}else{
		if(_timeLine.reversed)
		_timeLine.play();
		else
		_timeLine.reverse();
	}

}

Box2D Ballooning

ballonscreen

Many of you may of seen Pixars recent movie UP. In the movie the main character manages to take flight in his house by attaching dozens of inflatable balloons to the roof. Seems a bit far fetched don’t you think? Obviously the guys at Pixar aren’t too concerned with the laws of physics.
Well here at fatlinesofcode we don’t mind if cartoons break the laws of physcis either. But we do love to test them.
I have built a fun prototype game in Box2D to test this house ballooning theory. The house is a box with multiple balls attached via distance joints. Each of the balls has an anti-gravitational force applied to it which provides lift to the house. To pilot this balloon the UP arrow key is used to simulate adding more hot air to the balloons. Each time you press it an upward force is applied to each balloon thus providing more lift to the house. Additionally the LEFT and RIGHT arrow keys can be used to apply left and right forces to the house. Combining these controls you can achieve steady flight. Check out the demo and try manoeuvring your balloon through the various obstacles.


Source here


Pixelated video effect [Part II]

space

I few weeks ago I posted a pixelation effect. I have since optimised the code and also added in support for webcam input.

Checkout the demo and source code below.

This movie requires Flash Player 10

Here is an explanation of the technique.

First get the bitmapData source from the video input. Then divide the data into a grid of new bitmaps using a double for-loop. Store each one these in a two dimensional array.

private function createBitmapGrid():void {

			var source:BitmapData = getVideoSource();

			var _w:Number=source.width;
			var _h:Number=source.height;
			var rows:Number=source.height/_divideGridBy; // cols
			var cols:Number=source.width/_divideGridBy; // rows
			var cellWidth:Number = _w/cols-_cellpadding;
			var cellHeight:Number = _h/rows-_cellpadding

			for (var i:uint=0; i<cols; i++) {
				for (var j:uint=0; j<rows; j++) {
					var tempBitmapData:BitmapData=new BitmapData(cellWidth,cellHeight, true, 0x000000);
					var tempBitmap:Bitmap =new Bitmap(tempBitmapData);
					tempBitmap.x=i*_w/cols;
					tempBitmap.y=j*_h/rows;
					if(! _particleArray[i])
					_particleArray[i] = {};
					_particleArray[i][j] = tempBitmap;
					_holder.addChild(tempBitmap);
				}
			} 			

		}

Then add an enterframe listener to update the all the bitmap cells in the grid. Add a double for-loop to go through each cell. Use the copyPixel function to copy a rectangular portion of the source image to each cell. After you copy the source data, find the average brightness of the cell and use it as a variable to adjust the z value of the bitmap cell.

this.addEventListener(Event.ENTER_FRAME, updateBitmapGrid);
private function updateBitmapGrid(e:Event):void {

			var source:BitmapData = getVideoSource();

			var _w:Number=source.width;
			var _h:Number=source.height;
			var rows:Number=source.height/_divideGridBy; // cols
			var cols:Number=source.width/_divideGridBy; // rows
			var cellWidth:Number = _w/cols-_cellpadding;
			var cellHeight:Number = _h/rows-_cellpadding

			for (var i:uint=0; i<cols; i++) {
				for (var j:uint=0; j<rows; j++) {
					var tempBitmap:Bitmap = null;
					if (_particleArray[i]) {
						if(_particleArray[i][j]){
							tempBitmap=_particleArray[i][j];
						}
					}
					if(tempBitmap != null){
						var p:Point = new Point(0, 0);
						var rect:Rectangle = new Rectangle(tempBitmap.x+_cellpadding, tempBitmap.y+_cellpadding, cellWidth, cellHeight);
						tempBitmap.bitmapData.copyPixels(source, rect, p);
						var color:Number=getAverageColour(tempBitmap.bitmapData);
						var brightness:Number=getBrightness(getAverageColour(tempBitmap.bitmapData));
						if(_moveZ)
						tempBitmap.z = 255 - (brightness * 1);

					}
				}
			}
		}
private function getAverageColour( source:BitmapData ):Number {
			var red:Number=0;
			var green:Number=0;
			var blue:Number=0;

			var count:Number=0;
			var pixel:Number;

			for (var x:Number = 0; x < source.width; x++) {
				for (var y:Number = 0; y < source.height; y++) {
					pixel=source.getPixel(x,y);

					red+=pixel>>16&0xFF;
					green+=pixel>>8&0xFF;
					blue+=pixel&0xFF;

					count++;
				}
			}

			red/=count;
			green/=count;
			blue/=count;

			return red << 16 | green << 8 | blue;
		}

		private function getBrightness(colour:Number):Number {
			var R:Number=0;
			var G:Number=0;
			var B:Number=0;

			R+=colour>>16&0xFF;
			G+=colour>>8&0xFF;
			B+=colour&0xFF;

			var br:Number=Math.sqrt(R*R*.241+G*G*.691+B*B*.068);
			return br;
		}

Download package


Pixelated video effect

I have been experimenting more and more with bitmapdata recently. Checkout this pixelation effect I have created for flash video.

Here is how it works. There is an onEnterFrame function which first creates a new Bitmap object from the video source data. A grid of smaller bitmaps is then created from this large bitmap. Each one of the smaller bitmaps is analysied for it’s average color and brightness. The z property (flash 10) is then set to the inverse of brightness. The resulting effect is that for each one of the grid blocks the block moves in and out as the brightness changes. Lighter blocks come forward and darker blocks move back.

Flash is magic. Now watch it and freak out.

This movie requires Flash Player 10

Apologies to anyone with a slower machine, this swf is fairly cpu intensive. I would also recommend that you install the very latest flash player(10,0,22,87 ) to ensure you get the best framerate possible.


Water droplet effect

This is an example of how to use the actionscript displacement filter and bitmapData to create a sliding water droplet effect.

example here

actionscript 3 source here


Flash 3D models

I’ve been playing around with latest version of Papervision. I found this great tutorial on how to load Collada mesh models. Here is a little swf I mashed up. It uses 2 collada models but groups them together as one object with actionscript. Tweener is used to control the animation and mouse events. You view the AS source and DAE model files here.

This movie requires Flash Player 10