Linear Algebra for GATE Machine Learning

Linear algebra is the geometry underneath ML: data as vectors, models as matrix multiplications, similarity as dot products, compression as projections. GATE tests the geometric meaning, not rote computation.

Important concepts

GATE relevance

Rank/nullity reasoning, when XᵀX is invertible, eigendecomposition reading, PCA variance questions, and least-squares geometry.

Important formulas

w=(XᵀX)⁻¹Xᵀy · Av=λv · cos θ=a·b/‖a‖‖b‖ · ‖Ax‖²=xᵀAᵀAx

Worked example

Collinear features make XᵀX singular, so the normal equation fails — infinitely many w fit. Ridge adds λI, making (XᵀX+λI) invertible for any λ>0. That one line explains regularisation geometrically.

Practice questions

Practice questions (Regression/Unsupervised) + mock test. Theory: Guide Unit 3, Unit 6, maths refresher.

Related topics

Probability · Statistics · Deep learning

FAQ

Why do eigenvalues appear in ML?

They give the stretching directions of a matrix — variance axes in PCA, stability in optimisation, and rank information.

What does PCA actually keep?

The orthogonal directions of maximum variance; explained-variance ratios say how much information each keeps.