In the singular value decomposition of the data matrix $X$, we have the two unitary matrices $U$ and $V$. We have introduced that the unitary matrices satisfy $U^TU=UU^T=I$ and $VV^T=V^TV=I$. Here these identity matrices have different size.
Actually, unitary matrices preserve the angle between any two vectors in the vector space. Since the inverse of the unitary matrix is its own transpose, it can not only preserve the angles between vectors, but also preserve the length of the vectors. Essentially, unitary transformation is a coordinate transformation into a new representation. It just takes all of those vectors and rotates them into a new representation. It will not change the length of any vector and angles between any two vectors.
Here you will find that we just consider the real-valued case. However, when we consider the data matrix $X$ is a complex-valued matrix, the singular vectors $U$ and $V$ are complex-valued matrices. Compared with the real-valued matrix, the most important difference is that the transpose of a complex-valued matrix is actually a conjugate transpose.
To sum up, we give the mathematical form of unitary transformation here. For any two vectors $x$ and $y$ in vector space, then we have
where $(\cdot,\cdot)$ is the inner product of two vectors and the operator $U$ means the unitary transformation, which in a unitary matrix in discrete case. Based on this mathematical form, we realize that the inner product of $x$ and $y$ is unchanged when we transform $x$ and $y$ by unitary transformation.
Since we have introduced the unitary transform in the vector space, we interpret singular vector decomposition geometrically. Here we consider the size of data matrix is $n\times m$. Now we consider any vector $\vec{v}$ in space $\mathbb{R}^m$ and vector $\vec{v}$ is palced on the unit sphere. As we all know, the matrix is a linear transformation about the vector. Therefore, we consider the transformed vector $X\vec{v}$, which is placed on the ellipsoid in the vector space $\mathbb{R}^n$. Here the gemoetric interpretation of the singular value decomposition can be given. The length of these principal axes are specially given by the singular values of the data matrix $X$. The orientation of this ellipsoid is somehow given by these left singular vectors in $U$. Like the interpretation of linear transformation, $X$ multiplies the vector on the left side or on the right side, which corresponds to a change to a rotation of space. Finally, $X$ is not a square matrix, so the rotation of the corresponding space is actually a change in dimensionality.