Linear Algebra: Geometric Intuition

• span - the set of all possible vectors you can reach with a given set of vectors
• linearly dependent vector - a vector that can be expressed as a linear combination of the others
• basis vectors - a set of linearly independent vectors that span the full vector space
• determinant - Think of the columns of the matrix as vectors at the origin forming the edges of a skewed box. The determinant gives the volume of that box. reference
• inverse of a matrix - if you multiply a matrix by this, you will undo its transformation and get the identity matrix (if the determinant is zero, then there is no inverse, because you can't unsquish a 2D line and get a 3D space)
• rank - number of dimensions in the output of a transformation (if output of transformation is a line it's rank 1, plane -> rank 2, etc)
• full rank - when rank == number of columns (rank is as high as it can be)
• column space - the set of all possible outputs of the matrix
• null space - the space of all vectors that become the zero vector after you apply the matrix
• eigenvector of a matrix - this is a vector that stays within its own span when we apply the matrix
• eigenvalue of a matrix - the factor by which the eigenvector gets stretched after the transformation
• diagonal matrix - all elements of matrix are zero except diagonal. all the columns are eigenvectors for such a matrix

• size of a matrix: [# rows x # columns]
• matrix = transformation = function
• x ⋅ y = y ⋅ x

Express matrix as product of two things

Ex. matrix of people and movies

Idea: each movie has features (action, comedy, etc)

[people, features] * [features, movies] = [people, movies] (The two things we are multiplying are the factors)

Take the dot product of (the person row from [people, features]) and (the movie column from [features, movies])

To learn the matrix factorization, initialize randomly, then gradient descend to reasonable values