Alex Loftus
← Back to the lesson

What A transpose A measures

A Gram matrix records pairwise column dot products. Its diagonal records squared lengths.
A Gram matrix records pairwise column dot products. Its diagonal records squared lengths.

A table of pairwise comparisons

Put two column vectors uu and vv into a matrix A=[u v]A=[u\ v]. Transposing AA turns those columns into rows. Multiplying ATAA^\mathsf{T}A then pairs every original column with every other column:

ATA=[u⋅uu⋅vv⋅uv⋅v].A^\mathsf{T}A= \begin{bmatrix} u\cdot u&u\cdot v\\ v\cdot u&v\cdot v \end{bmatrix}.

This is a Gram matrix. Its diagonal describes each vector on its own; its off-diagonal entries compare different vectors.

Read an example

Choose u=(1,2)u=(1,2) and v=(2,0)v=(2,0). Then

ATA=[5224].A^\mathsf{T}A=\begin{bmatrix}5&2\\2&4\end{bmatrix}.

The diagonal entries 5 and 4 are squared lengths. The two matching 2s are the same dot product in opposite orders. This is why a real Gram matrix is symmetric.

If we normalized each nonzero column to length one first, the diagonal would be all ones and the off-diagonal entries would be cosine similarities. Without that normalization, vector length also influences the comparisons.

Which objects are being compared?

ATAA^\mathsf{T}A compares the columns of AA. The reversed product AATAA^\mathsf{T} compares its rows. When rows represent observations and columns represent features, this distinction tells you whether the comparison table is about features or observations.

Read a zero

What would an off-diagonal zero mean?

Answer: The corresponding columns are orthogonal under the ordinary real dot product. It does not by itself establish statistical independence.