Face culling and dot product

Hello, I seem to be getting something wrong, or I’m missing some detail on calculating the angle to the camera to skip drawing faces. But only when the camera DOES NOT look at the models directly. And I’m not being able to determine if it’s something wrong in the way I use nalgebra, or the way I understood the algorithm

I have this algorithm to perform culling before projecting the points:

for each mesh…

let world_isometry : Isometry3<f32> =
    Translation3::new(mesh.position.x,mesh.position.y,mesh.position.z) *
    UnitQuaternion::from_euler_angles(mesh.rotation.x, mesh.rotation.y, mesh.rotation.z);

for each face on the mesh

if !self.is_backface_b(&mesh.vertices[face.a], &camera, &face, &world_isometry) {
    // draw face

This is the culling algorithm:

    fn is_backface_b(&self,face_vertex : &Vertex, camera : &Camera, face:&Face, world_mat : &Isometry3<f32>) -> bool {
        let cam_pos : Point3<f32> = &camera.position;
        let view_vector : Point3<f32> = cam_pos - face_vertex.coordinates;
        let normal : Point3<f32> = world_mat.transform_point(&face.normal);
        let result : f32 = view_vector.dot(&normal.coords);
        result < 0.0

This seems to work OK when the camera is looking directly at the meshes, but draws extra triangles (and hides others) if the camera actually looks at the center of the screen. I’m pretty sure I’m missing something.

Any help is greatly appreciated.


I edited your message to make the code blocks more readable. It is best to use code blocks (with three backticks) instead of inline code (with one backtick) for large chunks of code.

It appears that you are confusing points and vectors here. A point identifies a location in space. A normal does not have a location in space as it is just a direction; so it should be a vector, not a point.

For example the view_vector is a vector so its type should be Vector3<f32> instead of Point3<f32>. Similarly, a normal should be a vector too, so face.normal should have the type Vector3<f32>, and it should be transformed with: let normal = world_mat.transform_vector(&face.normal);. So your code should look like this:

    fn is_backface_b(&self,face_vertex : &Vertex, camera : &Camera, face:&Face, world_mat : &Isometry3<f32>) -> bool {
        let cam_pos : Point3<f32> = &camera.position;
        let view_vector = cam_pos - face_vertex; // Will have the type Vector3<f32>
        let normal  = world_mat.transform_vector(&face.normal); // Will have the type Vector3<f32>.
        let result : f32 = view_vector.dot(&normal);
        result < 0.0

The type of face.normal is very important. By doing world_mat.transform_point(&face.normal), you are applying both the rotation and the translation transformation to the normal, which is wrong because a normal (i.e. a direction) does not have a location in space, it cannot be translated. By applying world_mat.transform_vector(&face.normal) with face.normal a Vector3<f32>, only the rotation part of the isometry will be applied.

1 Like

Thank you very much. Effectively, that was the issue. I was mixing Point3 and Vector3. And actually, I thought Vector3 was Point3 and vice versa, so I failed to detect the problem when i tried playing around that subject.