Home > Games Programming, Graphics, Programming, XNA > XNA Picking Tutorial Part I

XNA Picking Tutorial Part I

Introduction:

In the following three or four articles I’m gonna discuss XNA picking related issues including:
1- Ray picking.
2- RTS style selection box (two approaches).
3- Projection and why you need it.

I’m not an XNA professional so I might make some mistakes, please if you think any thing is wrong tell me and I’ll be happy to respond ๐

What is picking?

Well.. in short, picking is something you’ll need if you want to know what 3D object the mouse cursor is over! in other words picking lets you “pick” a 3D object with your mouse, if you have ever played a 3D RTS (real time startegy) game like C&C Generals, selecting a unit with you mouse is called picking!

In a 2D game knowing what’s the mouse is over – in most cases – is simple, just check if the mouse is inside a rectangle or a circle for example, in a 3D world it’s a little trickier!.

In a previous post I visualized how picking 3D objects works in XNA. (no need to check the old post, every thing you want is here…)
Today I’m gonna explain how the operation works, the official picking sample is great, and much of my explanation is taken from there…
Picking will be explained step by step in addition to some other picking issues…

1-Ray Picking:

A ray goes from a specified point towards a specified direction ( it doesn’t have an ending point).
XNA provides a Ray struct with some good intersection detection methods, you can check ray intersection against bounding volumes (BoundingBox, BoundingSphere, BoundingFrustum) and Planes.
The main idea of picking is to create a ray that goes from a point in the near plane of the camera frustum towards another point in the far plane of the camera frustum, these two points are specified using the mouse screen coordinates (mouse x and y), once we have that ray we can check for intersections with 3D objects using there bounding spheres for example..

Click on image to see full size

Whenever you draw a 3D object to your 2D screen it has to be projected using the matricesย  world, view and projection, multiplying the 3D position of a vertex with these three matrices produces the 2D position of the vertex in the viewport. this operation is called projection, transforming from world coordinates to screen coordinates.

When picking we want to do the opposite : transform from screen coordinates (mouse position on the screen) to world coordinates, This operation is called Unprojection.

XNA struct Viewport provides methods for both projecting & unprojecting points, for now we want the unprojection method which is:

```public Vector3 Unproject(Vector3 source, Matrix projection, Matrix view, Matrix world);
```

the first parameter is a Vector3, but the mouse position (2D vector) only has x,y what’s that? remember that we need a Ray that goes from a point in the near plane towards another point in the far plane? so we need to unproject two points.
The two points will be

```Vector3 nearPoint = new Vector3( mousePosition.x, mousePosition.y , 0);
Vector3 farPoint = new Vector3( mousePosition.x, mousePosition.y , 1);
```

for the near point we specified zero as the z component of the point which means this point is as close as possible to the camera, while one for the second point means it’s as far as possible from the camera… These two points still need to be unprojected so we use the Unproject method mentioned above.
The other three parameters of the method Unproject are the same matrices used for projection when drawing the scene, which we can get from the camera we used for drawing, world can be just Matrix.Identity, so to unproject the 2D points:

```Viewport viewport = GraphicsDevice.Viewport;
nearPoint = viewport.Unproject(nearPoint, camera.Projection, camera.View, Matrix.Identity);
farPoint = viewport.Unproject(farPoint, camera.Projection, camera.View, Matrix.Identity);
```

The two points nearPoint & farPoint are now in world space ( They are 3D points now :D).
If you know simple vector math you should know that to get a vector from point A to point B , you subtract point B from point A, so we create a direction vector and normalize it:

```Vector3 direction = farPoint - nearPoint;
direction.Normalize();
```

and the ray we need is

```//Ray(Vector3 position, Vector3 direction);
Ray ray = new Ray( nearPoint, direction);
```

You can now use this ray for picking! just make sure you transform your mesh’s bounding sphere before checking for intersection ( The function RayIntersectsModel in the official sample does that for you ๐ ) , note that the method Ray.Intesects returns a float? which means a nullable float… if there’s no intersection the function will return null, otherwise it returns the distance from the ray position to the intersection point.

So to wrap our ray function:

```public static Ray GetMouseRay(Vector2 mousePosition, Viewport viewport, Camera camera)
{
Vector3 nearPoint = new Vector3(mousePosition, 0);
Vector3 farPoint = new Vector3(mousePosition, 1);

nearPoint = viewport.Unproject(nearPoint, camera.Projection, camera.View, Matrix.Identity);
farPoint = viewport.Unproject(farPoint, camera.Projection, camera.View, Matrix.Identity);

Vector3 direction = farPoint - nearPoint;
direction.Normalize();

return new Ray(nearPoint, direction);
}
```

Notes on the code above:
1- viewport is the viewport the objects you’re picking are rendered in, by defualt you’ll be rendering only in one viewport which is GraphicsDevice.Viewport
2- camera class contains the view,projection matrices.
3- In my humble opinion ๐ , it’s good to embed the GetMouseRay method in the Camera class since unprojection is camera related, this way you only have to call

```Ray ray = camera.GetMouseRay(mousePosition,viewport);
```

Okay! just to make sure I got it right I wrote a small XNA4 app that visualizes the camera frustum and the mouse ray ๐ ( I updated it from XNA3.1 from a previous article ).

Click on an image to view full size

In the right view port you’ll see the main scene, nothing special, some models rendered using BasicEffect.
The left view port is the interesting one! ๐ the whole scene is rendered again, viewed from a different angle (another camera) and the main camera (used in the right view port) is rendered here ( just like in 3DS Max) , you can see the mouse ray in red and the projected scene ( the right viewport ) in the near plane.
– Instructions are written inside the app –

You can download the source code (VS2010 Solution) and\or the exe here:

Ray Picking Sample XNA4.0 Bin (37kB)

Ray Picking Sample XNA4.0 Source (54kB)

As I mentioned before, this project was updated from an older one, but currently I’m writing an XNA camera library that contains several camera types, and I intent to add the VisualCamera as a utility class called CameraVisualizer, I’ll release the camera package as soon as it’s done..

Well… this is the end of part one, part two will demonstrate picking multiple objects using a selection rectangle, just like the ones used in 3D RTS games, Thanks for reading ๐

1. 29/03/2011 at 2:33 am

Hi,

I would like to do the inverse, transform a world point to screen point, how can i do that?

Thaks.

2. 25/05/2011 at 8:47 am

I wanted to get the point where mouse intersects with the mesh in order to know where to place the bounding spheres, so that I click where I will place that bounding sphere, it tells me the point, and then I stop the debug and create a new BoundingSphere in that point, just the first time, for the volumes setup as its hard to aproximate the position by guessing random numbers… ๐
Feel free to remove the comments at about page (or move them here if possible), and once again thank you for your help! ๐

3. 25/05/2011 at 3:43 pm

You don’t have to guess!, use some sort of a loop for each keys row in the keyboard to place the bounding spheres.

If you know a little about modeling, you could create real keys for the keyboard.
I don’t think it will has that effect on performance, besides you’ll get the BoundingSpheres placed correctly.

4. 07/11/2011 at 11:06 am

Hi,

What happen when the objets that we want to select are in movments !!

I’ve tride this method but the selection doesn’t work when my objects are moving.

it’s work when they don’t move !

Thanks

• 07/11/2011 at 2:29 pm

Movement shouldn’t affect the process because we’re doing the picking in a single frame, Maybe your objects are moving too fast??

5. 07/11/2011 at 3:31 pm

Hi,

thanks for your informations. My Objects move slowly but when they move, the selection doesn’t work and when they don’t move it’s perfect !!!!

Incredible for me :((

I made a solar system and planet are in rotation but slowly. The user can move the solar system by freedrag, zoom. I’ve tried two thing for the move:

1. Move the camera
2. Move the objects

May be is the problem for selection !

I don’t understand this problem

• 10/11/2011 at 1:50 am

Hmmm…that’s weird!

What you can try:
a) Try using the same picking algorithm while you have only one planet.

b) Since your objects are planets you can do the following to pick a planet, for each planet:
1-Create a BoundingSphere that has the same radius and position of the planet

2-Unproject the mouse coordinates to get a Ray.
3-Check for collision between the mouse Ray & the planet BoundingSphere

Plan b) should work fine, if it doesn’t you should render both mouse ray & bounding sphere & see what is wrong

c)Make picking work only when you click the mouse button, add a break point at the beginning of the picking function, run & click on an object to debug & run the code line by line to see what’s wrong.

I really hope it works for you :), please let me know.

6. 18/05/2012 at 9:21 am

In ur sample ..how did u draw a ray as Redline.?Please mentioned the part of code

7. 18/05/2012 at 9:32 am

Kartheeswaran Jeyakumar :
In ur sample ..how did u draw a ray as Redline.?Please mentioned the part of code

• 18/05/2012 at 3:50 pm

In the VisualCamera class, we update the position of two vertices to draw the ray, line#174:
mouseRayVerts[0].Position = mouseRay.Position;
mouseRayVerts[1].Position = mouseRay.Position + farPlane * mouseRay.Direction;

Then in the Draw method line#218
device.DrawUserPrimitives(PrimitiveType.LineList, mouseRayVerts, 0, 1);

8. 15/05/2013 at 1:22 pm

I have a problem in picking, and I don’t manage to find where it is.
In fact I want to pick some points that I have drawing thanks to their coordinates…
Everything works but for the points which are far away from the center of the screen, there is a gap between the display of the point on the screen and the place I have to put the mouse to find it.

Have you ever heard of such a problem ?
Thank you.

• 16/05/2013 at 10:17 am

I can’t be sure but I guess you can render the Ray you’re getting and visually check if it really intersects the points or not.

You can also try projecting your points and make the picking process happen in a 2D space, just like explained in the second part of this tutorial (Second approach).

1. 25/11/2010 at 7:10 pm
2. 09/12/2010 at 6:37 am
3. 16/12/2010 at 1:11 pm