Skip to content

inverseTranspose(mat2) returns the inverse without transposing #1448

Description

@paulofduarte

glm::inverseTranspose for mat2 returns the plain inverse, the transpose step is missing. The mat3 and mat4 overloads are correct.

#include <glm/glm.hpp>
#include <glm/gtc/matrix_inverse.hpp>

glm::mat2 const M(1.f, 3.f, 2.f, 4.f);
glm::mat2 const A = glm::inverseTranspose(M);
glm::mat2 const R = glm::transpose(glm::inverse(M));
// A != R, but A == glm::inverse(M)

Cause: in glm/gtc/matrix_inverse.inl, the mat<2, 2, T, Q> overload constructs the same matrix as inverse(mat2) (func_matrix.inl), the two middle cofactors are never swapped.

Not a recent regression: the same implementation appears in the original glm/gtx/inverse_transpose.inl (created 2006) and was carried unchanged through the GTC promotion (cd183ed, 2010). Present on master and in 1.0.3. Configuration-independent (pure arithmetic); observed with clang on macOS/aarch64.

PR to follow with the one-line fix plus a regression test in test/gtc/gtc_matrix_inverse.cpp.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions