Linear Algebra: Geometric Intuition

Definitions (summarized from 3blue1brown)
  • span - the set of all possible vectors you can reach with a given set of vectors
  • linearly dependent vector - a vector that can be expressed as a linear combination of the others
  • basis vectors - a set of linearly independent vectors that span the full vector space
  • determinant - Think of the columns of the matrix as vectors at the origin forming the edges of a skewed box. The determinant gives the volume of that box. reference
  • inverse of a matrix - if you multiply a matrix by this, you will undo its transformation and get the identity matrix (if the determinant is zero, then there is no inverse, because you can't unsquish a 2D line and get a 3D space)
  • rank - number of dimensions in the output of a transformation (if output of transformation is a line it's rank 1, plane -> rank 2, etc)
  • full rank - when rank == number of columns (rank is as high as it can be)
  • column space - the set of all possible outputs of the matrix
  • null space - the space of all vectors that become the zero vector after you apply the matrix
  • eigenvector of a matrix - this is a vector that stays within its own span when we apply the matrix
  • eigenvalue of a matrix - the factor by which the eigenvector gets stretched after the transformation
  • diagonal matrix - all elements of matrix are zero except diagonal. all the columns are eigenvectors for such a matrix
Generally useful stuff
  • size of a matrix: [# rows x # columns]
  • matrix = transformation = function
  • x ⋅ y = y ⋅ x
Matrix factorization

Express matrix as product of two things

Ex. matrix of people and movies

Idea: each movie has features (action, comedy, etc)

[people, features] * [features, movies] = [people, movies] (The two things we are multiplying are the factors)

Take the dot product of (the person row from [people, features]) and (the movie column from [features, movies])

To learn the matrix factorization, initialize randomly, then gradient descend to reasonable values