We study the mathematical structure of the solution set (and its tangent space) to the matrix equation $G^{\ast} J G = J$ for a given square matrix $J$. In the language of pure mathematics, this is a Lie group which is the isometry group for a bilinear (or a sesquilinear) form. The tangent space to ${G : G^{\ast} JG = J}$ consists of solutions to the linear matrix equation $X^{\ast} J + JX = 0$. We explicitly demonstrate the computation of the solutions to the equation $X^{\ast} J \pm JX = 0$ for real and complex matrices. We provide numerical examples and visualizations.