Help stamp out GL_PROJECTION abuse.

By Steve Baker One of the most common OpenGL programming errors is misuse of the GL_PROJECTION matrix stack. This matrix stack is often perceived as being the correct place to put the position and orientation of the 'camera' - but this is not in fact the case.

Unfortunately, when someone does go and put their camera transform into the GL_PROJECTION matrix (instead of into GL_MODELVIEW where it belongs) - the consequences are rather subtle:

Lighting: OpenGL has to transform vertex normals into world coordinate space - that is to say WITHOUT the effects of perspective - but WITH the effects of the camera position. Hence, only the GL_MODELVIEW matrix is applied to the normals for lighting. If you put the camera transform into the GL_PROJECTION matrix then your lighting will be wrong. However, some people do their own lighting - and in any case, it can be a subtle error that you might not have noticed previously.
Fog: OpenGL has to figure out how far each vertex is from the camera. Once again, perspective effects are not relevent to this calculation - so the GL_PROJECTION matrix is not used. If you put the camera transform into the GL_PROJECTION matrix then your fogging will be wrong. Since few people use fog, many people have the error and don't know it.
TexGen: Since OpenGL uses the eyepoint to figure out some of the TexGen'ed texture coordinates, if the camera position is all mixed up with the projection information then you'll end up with some pretty strange texture coordinates. (Thanks to Brian Sharp at 3Dfx for pointing this one out)
Z-buffer: I believe (but have no proof) that a screwed up GL_PROJECTION matrix could cause your Z values to be computed strangely. This would likely only manifest itself as a lack of Z precision under some circumstances - so you might not notice - and it might not matter.

So, it's easy to see how someone could have a program that pretty much works correctly - that would appear to fail when the programmer first tries to play with Fog - or first notices a peculiar lighting anomaly. Perhaps one might not notice until switching to a new OpenGL implementation causes the behaviour to break.

Hence, most people are very disbelieving when their problems are diagnosed as projection abuse.

The Golden Rule.

The rule is:

The only functions that should be called when the glMatrixMode is set to GL_PROJECTION are:

glLoadIdentity - to initialise the stack.
gluPerspective/glFrustum/glOrtho/gluOrtho2 - to set the appropriate projection onto the stack.
You *could* use glLoadMatrix to set up your own projection matrix (if you understand the restrictions and consequences) - but I'm told that this can cause problems for some OpenGL implementations which rely on data passed to glFrustum, etc to determine the near and far clip planes.

Mitigating Circumstances.

It is actually possible to store the camera position in the GL_PROJECTION matrix under certain VERY specific circumstances - and if you are VERY familiar with OpenGL and attempting to use it only for rasterization. However, it is pretty much always the case that if you are asking the question then you don't understand enough to deal with the answer. If you don't already know what I mean then don't do it.

OK, So how DO I set up the Camera?

Well, you need to push the inverse of the camera position/orientation transform onto the GL_MODELVIEW stack before you start drawing polygons. If you use gluLookAt to do this, it's fairly painless - but using glRotate/glTranslate is likely to cause problems for all but the simplest camera motion since you want the INVERSE of the camera location.

What I do is to compose the camera matrix by hand, and then invert it before doing a glLoadMatrix. However, that does require a certain confidence with matrix operations that many people seem to lack.