Oooh… Shiny! Fun with Cube-Mapping

Cube mapping is a rendering technique to create shiny surfaces that appear to reflect the environment around them. Note that this differs from specular highlights – bright spots caused by reflected lights in the scene. Instead, cube maps simulate the reflection of other objects in the scene. Like the gentleman in this kettle, for example: (taken from the Snopes article on Reflecto-Porn”)

image

Cube maps are pretty cheap and easy to produce – making them possible even on modern mobile devices, and they can create a more convincing “base” lighting level than simple ambient lighting. So, how’d you do it?

 

Creating a Static Cube Map from Photographic Imagery

Cube maps are a set of six textures which, as the name suggests, can be assembled into the faces of a cube representing a full 360’ view of a location. For example, the image below shows a cubemap of the interior of the Santa Maria Dei Miracoli Via Del Corso church in Rome, Italy, taken from this site:

image

You can create a simple cubemap texture of any real-world location using a camera and tripod (and perhaps a bit of image post-processing to correct any lens distortion and stitch together the image edges). If you’re using Unity, you don’t need to bother stitching together the images as shown above. Instead, simply select Create –> Cubemap and drag the six separate images into the appropriate texture slots:

image 

To use the cubemap, create a new material that uses one of Unity’s “reflective” shaders, which have already implemented the necessary lighting calculations to apply the cubemap reflections. The result will look something like as follows:

image

The problem with the approach described so far might have already become apparent: the object is reflecting a static environment as was seen by the photographer in a particular location in a particular church in Italy. Unless your game happens to feature a shiny object placed at that same exact location, the cubemap is not going to add much realism to its appearance….

Looks like a shiny metal ball hovering in an Italian church :)

image
Doesn’t look like shiny metal ball hovering in the tropics :(

image

… that’s not to say that static cubemaps can’t ever be of use. Particularly in outdoor scenes, it can still be beneficial to use a cubemap that reflects the general environment (a snow scene, a desert etc.), and use that to provide low level lighting on the object. In these situations, you can often reuse the same texture used to create the skybox, for example. It won’t reflect specific features, but that often doesn’t matter.

 

Creating Dynamic Cubemaps

To create a more convincing reflection map, we need to sample the actual game scene in which the object is placed and then apply that as a cubemap texture onto the object. Here’s a script that does just that in Unity (works in free or Pro!):

  • Create a new C# script and copy ‘n’ paste the below
  • Attach the script to any game object in the scene at the location at which you want the cubemap to be sampled
  • Click play
  • The script will create a camera positioned at the centre of the object and point it sequentially up, right forward, down, left, and back, saving a PNG texture of each view in your projects “Assets” folder. Output files are named with the name of the object followed by _PositiveX, _PositiveY, _NegativeX etc. You can use these images to create a new cubemap texture as outlined above.
  • If the object on which the cube is placed has a reflective material (and the reflection map property is named “_Cube”, which it is in the inbuilt Unity reflective shaders), it will also automatically set the cubemap material on the object to allow you to preview it. Press R to update the preview if you move any of the objects in the scene. However note that this will be reset when you stop the player, so you’ll have to create a new cubemap material if you want the changes to be permanent.

using UnityEngine;
using System.Collections;
using System.IO; // Used for writing PNG textures to disk

public class CubemapProbe : MonoBehaviour {
	
	// Cubemap resolution - should be power of 2: 64, 128, 256 etc.
	public int resolution = 256;
	
	#region Cubemap functions
	
	// Return screencap as a Texture2D
	// http://wiki.unity3d.com/index.php?title=RenderTexture_Free 
	private Texture2D CaptureScreen() {
		Texture2D result;
		Rect captureZone = new Rect( 0f, 0f, Screen.width, Screen.height );
		result = new Texture2D( Mathf.RoundToInt(captureZone.width), Mathf.RoundToInt(captureZone.height), TextureFormat.RGB24, false);
		result.ReadPixels(captureZone, 0, 0, false);
		result.Apply();
		return result;
	}
	
	// Save a Texture2D as a PNG file
	// http://answers.unity3d.com/questions/245600/saving-a-png-image-to-hdd-in-standalone-build.html
	private void SaveTextureToFile(Texture2D texture, string fileName) {
		byte[] bytes = texture.EncodeToPNG();
		FileStream file = File.Open(Application.dataPath + "/" + fileName,FileMode.Create);
		BinaryWriter binary = new BinaryWriter(file);
		binary.Write(bytes);
		file.Close();
	}
	
	// Resize a Texture2D
	// http://docs-jp.unity3d.com/Documentation/ScriptReference/Texture2D.GetPixelBilinear.html
	Texture2D Resize(Texture2D sourceTex, int Width, int Height) {
		Texture2D destTex = new Texture2D(Width, Height, sourceTex.format, true);
		Color[] destPix = new Color[Width * Height];
		int y = 0;
        while (y < Height) {
            int x = 0;
            while (x < Width) {
                float xFrac = x * 1.0F / (Width );
                float yFrac = y * 1.0F / (Height);
				destPix[y * Width + x] = sourceTex.GetPixelBilinear(xFrac, yFrac);
                x++;
            }
            y++;
        }
        destTex.SetPixels(destPix);
        destTex.Apply();
		return destTex;
	}
	
	// Flip/Mirror the pixels in a Texture2D about the x axis
	Texture2D Flip(Texture2D sourceTex) {
		// Create a new Texture2D the same dimensions and format as the input
		Texture2D Output = new Texture2D(sourceTex.width, sourceTex.height, sourceTex.format, true);
		// Loop through pixels
		for (int y = 0; y < sourceTex.height; y++)
        {
            for (int x = 0; x < sourceTex.width; x++)
            {
				// Retrieve pixels in source from left-to-right, bottom-to-top
				Color pix = sourceTex.GetPixel(sourceTex.width + x, (sourceTex.height-1) - y);
				// Write to output from left-to-right, top-to-bottom
				Output.SetPixel(x, y, pix);
            }
        }
		return Output;
	}
	#endregion
	
	// Use this for initialization
	void Start () {
		
		// CreateCubeMap must be called in a co-routine, because we need it to wait for the end
		// of each frame render before grabbing the screen
		StartCoroutine(CreateCubeMap());
	}
	
	// This is the coroutine that creates the cubemap images
	IEnumerator CreateCubeMap()
	{
		// Initialise a new cubemap
		Cubemap cm = new Cubemap(resolution, TextureFormat.RGB24, true);
		
		// Disable any renderers attached to this object which may get in the way of our camera
		if(renderer) {
			renderer.enabled = false;
		}
		
		// Face render order: Top, Right, Front, Bottom, Left, Back
		Quaternion[] rotations = { Quaternion.Euler(-90,0,0), Quaternion.Euler(0,90,0), Quaternion.Euler(0,0,0), Quaternion.Euler(90,0,0), Quaternion.Euler(0,-90,0), Quaternion.Euler(0,180,0)};
		CubemapFace[] faces = { CubemapFace.PositiveY, CubemapFace.PositiveX, CubemapFace.PositiveZ, CubemapFace.NegativeY, CubemapFace.NegativeX, CubemapFace.NegativeZ };

		// Create a single face matching the settings of the cubemap itself
		Texture2D face = new Texture2D(resolution, resolution, TextureFormat.RGB24, true);
		
		// Use this to prevent white borders around edge of texture
		face.wrapMode = TextureWrapMode.Clamp;
		
		// Create a camera that will be used to render the faces
		GameObject go = new GameObject("CubemapCamera", typeof(Camera));
		
		// Place the camera on this object
		go.transform.position = transform.position;
		
		// Initialise the rotation - this will be changed for each texture grab
		go.transform.rotation = Quaternion.identity;
		
		// We need each face of the cube to cover exactly 90 degree FoV
		go.camera.fieldOfView = 90;
		
		// Ensure this camera renders above all others
		go.camera.depth = float.MaxValue;
		
		// Loop through and create each face
		for(int i = 0; i < 6; i++) {
			// Set the camera direction
			go.transform.rotation = rotations[i];
			// Important - wait the frame to be fully rendered before capturing
			// See http://answers.unity3d.com/questions/326384/texture2dreadpixels-unknown-error-not-inside-drawi.html
			yield return new WaitForEndOfFrame();
			// Capture the pixels to the texture
			face = CaptureScreen();
			// Resize to the chosen resolution - cubemap faces must be square!
			face = Resize(face, resolution, resolution);
			// Flip the image across the x axis
			face = Flip(face);
			// Retrieve the pixelarray of colours for the current face
			Color[] faceColours = face.GetPixels();
			// Set the current cubemap face
			cm.SetPixels(faceColours, faces[i], 0);
			// Save the texture
			SaveTextureToFile(face, gameObject.name + "_" + faces[i].ToString() + ".png");
		}
		
		// Apply the SetPixel changes to the cubemap faces to make them take effect		
		cm.Apply();
		
		// Assign the cubemap to the _Cube texture of this object's material
		if(renderer.material.HasProperty("_Cube")) {
			renderer.material.SetTexture("_Cube", cm);
		}
		
		// Cleanup
		DestroyImmediate(face);
		DestroyImmediate(go);
		
		// Re-enable the renderer
		if(renderer) {
			renderer.enabled = true;
		}
	}
	
	
	// Update is called once per frame
	void Update () {
		
		// Recalculate the cubemap if "R" is pressed)
		if(Input.GetKeyDown(KeyCode.R)){
			StartCoroutine(CreateCubeMap());
		}
	}	
}

 

Note that there are still a few issues with this approach – you can currently only attach this script to one cubemap in the scene at the time (because it needs the camera of the face currently being rendered to be on top of all others). If you have Unity Pro you could get around this by rendering directly from the camera to a render texture. Also, the reflection is created from a single point at the “centre” of the object, which may be some distance from it’s exterior reflective surface. There’s also some distortion issues at the edges of the field of view, but it’s good enough for me. If you make any improvements to it please let me know!

And here’s that shiny sphere again, this time using a cubemap created from the game scene in which it is placed, using the script above:

image

 

Rendering the faces of the cubemap textures takes a second or so, you wouldn’t want to do it in realtime in the game. If you have an object that moves around and you want it to reflect its current environment, one approach is to pre-render cubemaps for various “zones” in your game (e.g. in each room in an indoor scene, or where scenery changes significantly in an outside environment), then probe and swap in the closest appropriate cubemap texture to the object’s current location. This approach is described in more detail at http://www.blog.radiator.debacle.us/2013/05/cubemapped-environment-probes-source.html

About these ads
This entry was posted in Game Dev and tagged , , . Bookmark the permalink.

One Response to Oooh… Shiny! Fun with Cube-Mapping

  1. Christina Norwood says:

    Great script, very useful, but what’s the purpose of the Flip function in the script? it produces images that are upside down and that doesn’t seem to be useful, or have I missed something?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s