JiblibFlash + Augmented Reality (Take Three)

Well here’s one more take. Now there’s two levels – you’ll bounce back and forth between them as you get through each maze. I made everything a bit smaller which improves cpu usage and also makes it easier to see. Also (hopefully) fixed it so that it will be viewable on Macs (Don’t know why Apple makes it so difficult to target the default webcam of a Mac, but man, what a pain). Once again, the marker can be found here.

I’m still not happy with how the movement is accomplished. It takes a whole lot of mucking around to get a good movement based on the AR transformation system, but it works. Mostly…

While not as clean as I normally like my code, it’s all posted below, if you’d like to try it out.. You’ll have to supply your own materials as well as flartoolkit, papervision3d, and jiglib libraries, but other than that, it’s all there. Enjoy

The two maze levels and their interface:

package  {
 
	import org.papervision3d.materials.utils.MaterialsList;
 
	/**
	 * Maze Level Interface
	 * @author Devon O Wolfgang
	 */
	public interface ILevel {
		function get wallList():MaterialsList;
		function get map():Array;
	}
 
}

package  {
 
	import org.papervision3d.materials.BitmapMaterial;
	import org.papervision3d.materials.utils.MaterialsList;
 
	/**
	 * First maze level with brick wall
	 * @author Devon O Wolfgang
	 */
	public class Level1Maze implements ILevel {
 
		[Embed(source = "../assets/wall.jpg")] private static const WallClass:Class;
 
		private var _wallMaterial:BitmapMaterial = new BitmapMaterial(new WallClass().bitmapData);
 
		private var _wallList:MaterialsList;
 
		private var _map:Array = [ 	[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
									[0, 0, 0, 0, 0, 0, 0, 1, 1, 1],
									[0, 0, 0, 0, 0, 0, 0, 1, 1, 1],
									[1, 1, 1, 1, 1, 0, 0, 0, 0, 1],
									[1, 1, 0, 0, 1, 0, 0, 0, 0, 1],
									[1, 1, 0, 0, 1, 0, 0, 1, 1, 1],
									[1, 1, 0, 0, 1, 0, 0, 1, 1, 1],
									[1, 1, 0, 0, 0, 0, 0, 0, 0, 1],
									[1, 1, 0, 0, 0, 0, 0, 0, 0, 1],
									[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
									];
 
		public function Level1Maze() {
			buildMaterialList();
		}
 
		private function buildMaterialList():void {
			_wallList = new MaterialsList( { all:_wallMaterial } );
		}
 
		public function get wallList():MaterialsList { return _wallList; }
 
		public function get map():Array { return _map; }
	}
}

package  {
 
	import org.papervision3d.materials.BitmapMaterial;
	import org.papervision3d.materials.utils.MaterialsList;
 
	/**
	 * Second level maze of hedges
	 * @author Devon O Wolfgang
	 */
	public class Level2Maze implements ILevel {
 
		[Embed(source = "../assets/hedge.jpg")] private static const WallClass:Class;
 
		private var _wallMaterial:BitmapMaterial = new BitmapMaterial(new WallClass().bitmapData);
 
		private var _wallList:MaterialsList;
 
		private var _map:Array = [ 	[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
									[1, 0, 0, 0, 0, 0, 0, 1, 1, 1],
									[1, 0, 0, 0, 0, 0, 0, 1, 1, 1],
									[1, 0, 0, 1, 1, 0, 0, 0, 0, 1],
									[1, 0, 0, 1, 1, 0, 0, 0, 0, 1],
									[1, 0, 0, 1, 1, 1, 1, 0, 0, 1],
									[1, 0, 0, 1, 1, 1, 1, 0, 0, 1],
									[1, 0, 0, 0, 0, 0, 1, 0, 0, 1],
									[1, 0, 0, 0, 0, 0, 1, 0, 0, 1],
									[1, 0, 0, 1, 1, 1, 1, 1, 1, 1]
									];
 
		public function Level2Maze() {
			buildMaterialList();
		}
 
		private function buildMaterialList():void {
			_wallList = new MaterialsList( { all:_wallMaterial } );
		}
 
		public function get wallList():MaterialsList { return _wallList; }
 
		public function get map():Array { return _map; }
	}
}

Not perfectly decoupled, but the FlarObject handles most of the AR stuff, while the MazeScene handles the PV3d and Jiglib stuff.

package  {
 
	import flash.display.BitmapData;
	import flash.display.DisplayObject;
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.media.Camera;
	import flash.media.Video;
	import flash.system.Capabilities;
	import flash.utils.ByteArray;
 
	import org.papervision3d.core.render.IRenderEngine;
	import org.papervision3d.scenes.Scene3D;
	import org.papervision3d.view.Viewport3D;
 
	import org.libspark.flartoolkit.core.FLARCode;
	import org.libspark.flartoolkit.core.param.FLARParam;
	import org.libspark.flartoolkit.core.raster.rgb.FLARRgbRaster_BitmapData;
	import org.libspark.flartoolkit.core.transmat.FLARTransMatResult;
	import org.libspark.flartoolkit.detector.FLARSingleMarkerDetector;
	import org.libspark.flartoolkit.pv3d.FLARBaseNode;
	import org.libspark.flartoolkit.pv3d.FLARCamera3D;
 
	/**
	 * Manages most FLAR goings on
	 * @author Devon O. Wolfgang
	 */
	public class FlarObject extends Sprite {
 
		[Embed (source="pat1.pat", mimeType="application/octet-stream")]
		private var Pattern:Class;
 
		[Embed (source="camera_para.dat", mimeType="application/octet-stream")]
		private var Params:Class;
 
		private var _fparams:FLARParam;
		private var _fpattern:FLARCode;
		private var _raster:FLARRgbRaster_BitmapData;
		private var _detector:FLARSingleMarkerDetector;
		private var _transformObject:FLARTransMatResult;
		private var _virtualcam:FLARCamera3D;
 
		private var _vid:Video;
		private var _webcam:Camera;
		private var _bmd:BitmapData;
 
		private var _renderEngine:IRenderEngine;
		private var _scene:Scene3D;
		private var _viewport:Viewport3D;
		private var _flarContainer:FLARBaseNode;
 
		private var _mazescene:MazeScene;
 
		private var _doRender:Boolean = false;
 
		public function FlarObject(videowidth:int = 640, videoheight:int = 480) {
			initFlar();
			initVideo(videowidth, videoheight);
			initBitmap();
			initCamera();
		}
 
		private function initFlar():void {
			_fparams = new FLARParam();
			_fparams.loadARParam(new Params() as ByteArray);
 
			_fpattern = new FLARCode(16, 16);
			_fpattern.loadARPatt(new Pattern());
 
			_transformObject = new FLARTransMatResult();
		}
 
		private function initVideo(vw:int, vh:int):void {
			_vid = new Video(vw, vh);
 
			// find default camera on Mac
			var camIndex:int = 0;
			for ( var i : int = 0 ; i < Camera.names.length ; i++ ) {
				if ( Camera.names[ i ] == "USB Video Class Video" ) {
					camIndex = i;
					break;
				}
			}
			_webcam = Camera.getCamera(String(camIndex));
 
			_webcam.setMode(vw, vh, 31);
			_vid.attachCamera(_webcam);
			addChild(_vid);
		}
 
		private function initBitmap():void {
			_bmd = new BitmapData(_vid.width, _vid.height);
			_bmd.draw(_vid);
			_raster = new FLARRgbRaster_BitmapData(_bmd);
			_detector = new FLARSingleMarkerDetector(_fparams, _fpattern, 80);
		}
 
		private function initCamera():void {
			_virtualcam = new FLARCamera3D(_fparams);
		}
 
		private function renderHandler(event:Event):void {
			_bmd.draw(_vid);
			try {
				if (_detector.detectMarkerLite(_raster, 80) && _detector.getConfidence() > .45) {
					_viewport.visible = true;
					_detector.getTransformMatrix(_transformObject);
					_flarContainer.setTransformMatrix(_transformObject);
 
					_mazescene.moveBall( _transformObject );
 
					_renderEngine.renderScene(_scene, _virtualcam, _viewport);
 
				} else {
					_viewport.visible = false;
				}
			} catch (err:Error) {
				// do nothing
			}
		}
 
		public function startRendering():void {
			if (_mazescene != null && !willTrigger(Event.ENTER_FRAME)) {
				_viewport.visible = true;
				addEventListener(Event.ENTER_FRAME, renderHandler);
				_doRender = true;
			} else {
				throw new Error("MazeScene must be set before rendering can begin!");
			}
		}
 
		public function stopRendering():void {
			_viewport.visible = false;
			removeEventListener(Event.ENTER_FRAME, renderHandler);
			_doRender = false;
		}
 
		public function set mazescene(value:MazeScene):void {
			_mazescene = value;
			_renderEngine = _mazescene.engine;
			_scene = _mazescene.scene;
			_viewport = _mazescene.viewport;
			_flarContainer = _mazescene.flarContainer;
			addChild(_viewport);
		}
	}
}

package  {
 
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.events.EventDispatcher;
	import jiglib.cof.JConfig;
	import jiglib.geometry.JSphere;
	import jiglib.math.JNumber3D;
	import jiglib.physics.PhysicsSystem;
	import jiglib.plugin.papervision3d.Papervision3DPhysics;
	import jiglib.plugin.papervision3d.Pv3dMesh;
	import org.libspark.flartoolkit.core.transmat.FLARTransMatResult;
	import org.papervision3d.core.math.Matrix3D;
	import org.papervision3d.core.math.Number3D;
	import org.papervision3d.materials.BitmapMaterial;
 
	import jiglib.geometry.JBox;
 
	import org.papervision3d.cameras.Camera3D;
	import org.papervision3d.lights.PointLight3D;
	import org.papervision3d.materials.shadematerials.FlatShadeMaterial;
	import org.papervision3d.materials.utils.MaterialsList;
	import org.papervision3d.objects.DisplayObject3D;
	import org.papervision3d.objects.primitives.Cube;
	import org.papervision3d.objects.primitives.Plane;
	import org.papervision3d.objects.primitives.Sphere;
	import org.papervision3d.render.BasicRenderEngine;
	import org.papervision3d.scenes.Scene3D;
	import org.papervision3d.view.layer.util.ViewportLayerSortMode;
	import org.papervision3d.view.layer.ViewportLayer;
	import org.papervision3d.view.Viewport3D;
 
 
	import org.libspark.flartoolkit.pv3d.FLARBaseNode;
 
	import org.papervision3d.Papervision3D;
	import org.papervision3d.core.render.IRenderEngine;
	import org.papervision3d.render.BasicRenderEngine;
	import org.papervision3d.scenes.Scene3D;
	import org.papervision3d.view.Viewport3D;
 
	/**
	 * Manages pv3d/jiglib shenanigans
	 * @author Devon O. Wolfgang
	 */
	public class MazeScene extends EventDispatcher {
 
		[Embed(source = "../assets/metal1.jpg")] private static const BallClass:Class;
 
		private var _scene:Scene3D;
		private var _viewport:Viewport3D;
		private var _engine:BasicRenderEngine;
 
		private var _flarContainer:FLARBaseNode;
 
		private var _physics:Papervision3DPhysics;
 
		private var _ground:JBox;
		private var _world:DisplayObject3D;
		private var _ball:JSphere;
		private var _visibleBall:Sphere;
 
		private var _cellsize:int = 12;
 
		private var _level:ILevel;
 
		private var _physicsWalls:Array = [];
		private var _visibleWalls:Array = [];
		private var _vplObjects:ViewportLayer;
 
		public function MazeScene(videowidth:int = 640, videoheight:int = 480) {
			// no tracing
			Papervision3D.PAPERLOGGER.unregisterLogger(Papervision3D.PAPERLOGGER.traceLogger);
			initScene(videowidth, videoheight);
		}
 
		private function initScene(vw:int, vh:int):void {			
			_viewport = new Viewport3D(vw, vh, false, false, true, true);
			_engine = new BasicRenderEngine();
			_flarContainer = new FLARBaseNode();
			_scene = new Scene3D();
			_scene.addChild(_flarContainer);
 
			_physics = new Papervision3DPhysics(_scene, 2);
			_physics.engine.setSolverType("ACCUMULATED");
			PhysicsSystem.getInstance().setGravity(new JNumber3D(0, 0, -5));
		}
 
		public function buildMaze():void {
 
			_vplObjects = new ViewportLayer(_viewport, null);
			_vplObjects.layerIndex = 1;
			_vplObjects.sortMode = ViewportLayerSortMode.Z_SORT;
			_viewport.containerSprite.addLayer(_vplObjects);
 
			var light:PointLight3D = new PointLight3D();
			light.x = 0; light.y = 300; light.z = 0;
 
			var groundMat:FlatShadeMaterial = new FlatShadeMaterial(light, 0x999999);
			var matList:MaterialsList = new MaterialsList( { all:groundMat } );
			var groundPlane:Cube = new Cube(matList, 125, 2, 125);
 
			_ground = new JBox(new Pv3dMesh(groundPlane), 125, 2, 125);
			_ground.material.restitution = 1.5;
			_ground.material.friction = 2;
			_ground.movable = false;
			_physics.addBody(_ground);
 
			// create walls
 
			var w:int = _level.map[0].length;
			var h:int = _level.map.length;
 
			for (var i:int = 0; i < h; i++) {
				for (var j:int = 0; j < w; j++) {
					if (_level.map[i][j] == 1) {
						var c:Cube = new Cube(_level.wallList, _cellsize, _cellsize, _cellsize, 1, 1, 1);
						_flarContainer.addChild(c);
						_vplObjects.addDisplayObject3D(c);
						_visibleWalls.push(c);
 
						var wall:JBox = new JBox(new Pv3dMesh(c), _cellsize, _cellsize, _cellsize);
						wall.moveTo(new JNumber3D(((j * _cellsize) - 65) + (_cellsize * .5),      ((i * _cellsize) - 65) + (_cellsize * .5),     _cellsize * .6));
						wall.movable = false;
						_physicsWalls.push(wall);
						_physics.addBody(wall);
					}
				}
			}
 
			var ballMat:BitmapMaterial = new BitmapMaterial(new BallClass().bitmapData);
			_visibleBall = new Sphere(ballMat, 6);
			_flarContainer.addChild(_visibleBall);
			_vplObjects.addDisplayObject3D(_visibleBall);
 
			_ball = new JSphere(new Pv3dMesh(_visibleBall), 6);
			_ball.mass = 10;
			resetBall();
			_physics.addBody(_ball);
		}
 
		private function destroy():void {
			var len:int = _visibleWalls.length;
			for (var i:int = 0; i < len; i++) {
				_flarContainer.removeChild(_visibleWalls[i]);
				_vplObjects.removeDisplayObject3D(_visibleWalls[i]);
				_physics.removeBody(_physicsWalls[i]);
			}
			_physicsWalls = [];
			_visibleWalls = [];
 
			_physics.removeBody(_ball);
			_physics.removeBody(_ground);
			if (_visibleBall != null) {
				_vplObjects.removeDisplayObject3D(_visibleBall);
				_flarContainer.removeChild(_visibleBall);
			}
		}
 
		private function resetBall():void {
			_ball.moveTo(new JNumber3D(40, 40, 25));
		}
 
		public function moveBall(trans:FLARTransMatResult):void {
			var mat:Matrix3D = new Matrix3D();
			mat.n11 =  trans.m01; mat.n12 =  trans.m00; mat.n13 =  trans.m02; mat.n14 =  trans.m03;
			mat.n21 = -trans.m11; mat.n22 = -trans.m10; mat.n23 = -trans.m12; mat.n24 = -trans.m13;
			mat.n31 =  trans.m21; mat.n32 =  trans.m20; mat.n33 =  trans.m22; mat.n34 =  trans.m23;
			var objRot:Number3D = Matrix3D.matrix2euler(mat);
			trace(objRot.x);
			if(_ball.z < 10 && _ball.z > 4)
				// too much kludge to get this working well. Experiment with these numbers and see what's good for you
				_ball.addWorldForce(new JNumber3D((-objRot.z + 100) * .15, ((objRot.x - 120) * -2 ) , 0), _ball.currentState.position);
 
			_physics.step();
			if (_ball.z < -30) dispatchEvent(new Event("nextLevel"));
		}
 
		public function get scene():Scene3D { return _scene; }
 
		public function get viewport():Viewport3D { return _viewport; }
 
		public function get engine():IRenderEngine { return _engine; }
 
		public function get flarContainer():FLARBaseNode { return _flarContainer; }
 
		public function get level():ILevel { return _level; }
 
		public function set level(value:ILevel):void {
			_level = value;
			destroy();
			buildMaze();
		}
	}
}

And finally the Main class to tie it all together:

package {
 
	import flash.display.Sprite;
	import flash.events.Event;
 
	/**
	 * Augmented Reality with Jiglib Maze game
	 * @author Devon O. Wolfgang
	 */
	public class Main extends Sprite {
 
		private var _flarObject:FlarObject;
		private var _mazeScene:MazeScene;
		private var _levels:Array;
		private var _currentLevel:int = 0;
 
		// game always starts with ball falling. Give it 2 tries
		private var _firstTry:Boolean = true;
 
		public function Main():void {
			if (stage) init();
			else addEventListener(Event.ADDED_TO_STAGE, init);
		}
 
		private function init(e:Event = null):void {
			removeEventListener(Event.ADDED_TO_STAGE, init);
			// entry point
 
			_levels = [ new Level1Maze(), new Level2Maze() ];
 
			initGame();
		}
 
		private function initGame():void {
			_flarObject = new FlarObject();
			_mazeScene = new MazeScene();
			_mazeScene.addEventListener("nextLevel", changeLevel);
			_mazeScene.level = _levels[_currentLevel];
			_flarObject.mazescene = _mazeScene;
			addChild(_flarObject);
			_flarObject.startRendering();
		}
 
		private function changeLevel(event:Event):void {
			if (_firstTry) {
				_firstTry = false;
			} else {
				_flarObject.stopRendering();
				_currentLevel++;
				if (_currentLevel > _levels.length-1) _currentLevel = 0;
				_mazeScene.level = _levels[_currentLevel];
				_flarObject.startRendering();
			}
		}
	}
 
}

Have fun…

EDIT:
I cleaned up the code a tad more uploaded the entire project. Anyone wanting to play around with this can download the whole kit and kaboodle here. Let me know if you discover anything interesting.

CW says:

May 27, 2009 at 2:31 am

Great work!
I am also trying the jiglib, however, I found the RigidBody cannot really rotate in all axis in one time like PV3D, have you encountered similar problems?
Devon O. says:

May 27, 2009 at 4:18 am

I have noticed that (if I understand what you’re saying). It would be nice to have a physics “container object”, sort of like a generic DisplayObject3d that you could attach multiple Rigid Bodies to then animate them all as one. Of course there may be and I just haven’t dug far enough into the system. And of course I may have no idea what you’re talking about…
Andras Csizmadia says:

May 27, 2009 at 2:42 pm

Hello,

I wanted to play with your example, but i cannot complie because of the missing Embed sources. Any chance that you post a zipped package?
Many thanks for the experiment!

Best Regards!
Devon O. says:

May 29, 2009 at 12:29 am

Hey Andras, all the files are now downloadable. Have at…
Collection of jiglibflash examples/tutorials 3d physic engine | JiglibFlash - 3D Physics Engine AS3 says:

July 23, 2009 at 7:00 am

[...] http://blog.onebyonedesign.com/?p=270 [...]
Sergio Maruqes Dias says:

September 17, 2009 at 4:52 am

hi,

i encountered the same problem as you with the rigidbodys. i hoped you could give me a solution to the container problem. i would like to rotate all my rigidbodys as one big piece like DisplayObject3d.
any idea please?

thanks in advance
Benjamin says:

January 11, 2010 at 4:41 am

Hi,

When the marker is tilted to a certain angle, the ball just roll over the wall entirely. Any idea why?
Ncuzz says:

January 27, 2010 at 5:20 pm

I have a error like this :
1067: Implicit coercion of a value of type jiglib.plugin.papervision3d:Pv3dMesh to an unrelated type jiglib.physics:PhysicsSystem.

1067: Implicit coercion of a value of type Number to an unrelated type jiglib.physics:PhysicsSystem.

1067: Implicit coercion of a value of type Number to an unrelated type jiglib.plugin:ISkin3D.

can you help me?
Ryo007 says:

March 8, 2010 at 1:21 pm

Hi Devon!

Great example. I’m trying to do something, but I have a problem, apparently you also have ..

The problem occurs when i turn the marker, what happens is that when you add force anyway this will target in the original direction of the marker, as if taken with respect to the marker force locally, and not from the new point of view.

I’ve been playing with mathematical equations to reverse the signs as I turn the marker, but anyway I have not had good results.

I leave a small picture (excuse the poor drawing in paint) http://yfrog.com/4jdibujocctp of what happens and how it should happen. Hopefully you can help, and thank you very much again for this excellent example.

ps: Sorry my bad english :P