change of basis

A vector in a vector space \(v\in V\) can be described in terms of coordinates in \(B_{old}\), \(v_{old}\) or coordinates in \(B_{new}\), \(v_{new}\). A change of basis matrix changes a description in one set of coordinates to the other.

What are these coordinates? Given a basis \(B_{old}\), any vector \(w\in V\), can be specified in terms of the basis vectors in \(B_{old} = (v_1, ..., v_n)\): \[ w = \sum_{i=1}^{n} a_i v_i \] The coordinates of \(w\) in \(B_{old}\) are given by: \(w_{old} = (a_1, ..., a_n)\)

The change of basis matrix is then \(A\) such that: \[ v_{old} = Av_{new} \]

Below, I have a copy of the wikipedia entry with my own commentary for understanding.

Let \(F\) be a field and \(F^n\) be the set of \(n\) - tuples where scalar multiplication and addition is done component-wise.

An ordered basis \(B=(v_1, v_2, ..., v_n)\) of a \(F\) vector space \(V\) defines a linear isomorphism \(\phi: F^n \rightarrow V\) by \[ \phi(x_1, ..., x_n) = \sum_{i=1}^{n} x_iv_i \]

Commentary:

• \(F^n\) is given in units of basis \(B\).
• \(V\) corresponds to 'objective reality'. In \(V\), there is only one description of a vector – the description according to the standard basis.
• The 'type' of \(\phi\) is:
  • input: coordinates, which are given relative to the basis \(B\)
  • output: the vector description in objective reality
• Question: what are the coordinate of \(v_1\) relative to? A: The standard basis.

Let \(B_{old} = (v_1,...,v_n)\) be the 'old' basis of a change of basis, and \(\phi_{old}\) be the associated isomorphism. Given a change of basis matrix \(A\), consider it as the matrix of an endomorphism \(\psi_A\) of \(F^n\). Then, define: \[ \phi_{new} = \phi_{old} \circ \psi_{A} \] and \[ B_{new} = \phi_{new}(\phi_{old}^{-1}(B_{old})) \]

Commentary:

• The 'type' of \(\phi_{new}\) is:
  • Input: Coordinates in terms of \(B_{new}\)
  • Output: Coordinates in objective reality, i.e., w.r.t. the standard basis
• Similarly, the 'type' of \(\phi_{old}\) is:
  • Input: Coordinates in terms of \(B_{old}\)
  • Output: Coordinates in objective reality
• The type of \(\psi_{A}\):
  • Input: coordinates w.r.t \(B_{new}\)
  • Output: the equivalent coordinates w.r.t \(B_{old}\), i.e. the coordinates in \(B_{old}\) that describe the same underlying vector
• Do a quick check to make sure that the types check out
• Question: why isn't it \(B_{new} = \phi_{new}(\psi_{A}(\phi_{old}^{-1}(B_{old})))\)? A: I'm not sure, but I think it's because \(\phi_{old}^{-1}(B_{old})\) are all unit vectors, i.e. each column vector is 1 at one entry and 0 everywhere else. In which case, if the ordering of the basis is fixed, applying \(\phi_{new}\) to the unit vectors will get us the description of the basis vectors of \(B_{new}\), in terms of 'objective reality', which is what we want.

Compose with \(\phi^{-1}_{old}\) on the left and \(\phi^{-1}_{new}\) on the right: \[ \phi^{-1}_{old} = \psi_{A} \circ \phi^{-1}_{new} \]

For a vector \(v\in V\): \[ \phi^{-1}_{old}(v) = \psi_{A} \left( \phi^{-1}_{new}(v) \right) \]

• change of basis: The vector in question is fixed. Take the coordinates in one basis for that vector and find what the coordinates in the other basis would be. This can be summed up in the formula: \[ \mathbf{x}^U = (U\rightarrow V)\mathbf{x}^V \] where \((U\rightarrow V)\) is a change of basis matrix.
• linear transformation: The basis is fixed. Take the coordinates for a vector. The linear transformation moves the vector to a new place. Find the coordinates of the new vector. This can be summed up in the formula: \[ \mathbf{y} = (U\rightarrow V)\mathbf{x} \] Notice that the change of basis matrix shows up again. See Boris Belousov's blog for details.

We talk about endomorphisms, but the story is kind of similar for any matrix \(A\) that represents a linear map from \(V\) to \(W\).

Given a matrix \(M\) that represents a linear map from \(V\) to \(V\) over a basis \(B_{old}\), then what will the linear map's matrix be for a new coordinate system with respect to \(B_{new}\)? (Note: whenever we say that a matrix represents a linear map, that map is always implicitly given with respect to a particular set of basis vectors and induced coordinate system. However this isn't always mentioned if there's only one set of bases per vector space).

Then, the endomorphsim, with respect to the new bases is: \[ PMP^{-1} \] where \(v_{old} = Pv_{new}\)

You can quickly check to see what is going on. Multiplying by a \(v_{new}\) on the left gets us the coordinates \(v_{old}\), we carry out the linear map in terms of the old coordinates, and then we convert back to the new coordinates.

• eli bendersky
• change of basis vs linear transformation
• wikipedia page
• A change in basis matrix defines a homomorphism between \(V\) and \(W\) – see 2.2.1 in Math for Machine Learning