Linear Algebra for GATE Machine Learning
Linear algebra is the geometry underneath ML: data as vectors, models as matrix multiplications, similarity as dot products, compression as projections. GATE tests the geometric meaning, not rote computation.
Important concepts
- Vectors, norms, dot products, cosine similarity.
- Matrix rank, inverse, singularity; solving Xw=y.
- Eigenvalues/eigenvectors; symmetric matrices and SVD.
- Projections, least squares as orthogonal projection.
- PCA: variance-maximising directions, explained variance.
- Positive definiteness and convexity of quadratics.
GATE relevance
Rank/nullity reasoning, when XᵀX is invertible, eigendecomposition reading, PCA variance questions, and least-squares geometry.
Important formulas
w=(XᵀX)⁻¹Xᵀy · Av=λv · cos θ=a·b/‖a‖‖b‖ · ‖Ax‖²=xᵀAᵀAx
Worked example
Collinear features make XᵀX singular, so the normal equation fails — infinitely many w fit. Ridge adds λI, making (XᵀX+λI) invertible for any λ>0. That one line explains regularisation geometrically.
Practice questions
Practice questions (Regression/Unsupervised) + mock test. Theory: Guide Unit 3, Unit 6, maths refresher.
Related topics
Probability · Statistics · Deep learning
FAQ
Why do eigenvalues appear in ML?
They give the stretching directions of a matrix — variance axes in PCA, stability in optimisation, and rank information.
What does PCA actually keep?
The orthogonal directions of maximum variance; explained-variance ratios say how much information each keeps.