GATE DA — Section 3
Calculus & Optimization
Comprehensive study notes on single-variable functions, limits, continuity, differentiability, Taylor series, and optimization for GATE Data Science & AI.
1. Functions of a Single Variable: Domains & Properties
A real-valued function of a single variable is a mapping f: D → ℝ where D ⊆ ℝ is the domain. In machine learning, loss functions, activation curves (e.g., Sigmoid, ReLU, GELU), and likelihood functions are single-variable slices along optimization coordinates.
- Monotonicity: If
f'(x) ≥ 0on an intervalI,f(x)is monotonically non-decreasing. Iff'(x) > 0, it is strictly increasing and invertible. - Convexity: A function
f(x)is strictly convex on intervalIif for allx_1, x_2 ∈ Iandλ ∈ (0, 1):f(λ x_1 + (1-λ)x_2) ≤ λ f(x_1) + (1-λ)f(x_2). Equivalently, iff''(x) ≥ 0everywhere, every local minimum is guaranteed to be a global minimum.
2. Limits, Continuity & Differentiability
Understanding limits is crucial for evaluating gradient behavior and asymptotic stability:
- Limit Existence:
lim_{x→a} f(x) = Lif and only if both left-hand and right-hand limits exist and coincide:lim_{x→a^-} f(x) = lim_{x→a^+} f(x) = L. - Continuity: A function
fis continuous atx = aif:f(a)is defined.lim_{x→a} f(x)exists.lim_{x→a} f(x) = f(a).
- Differentiability:
fis differentiable ataif the derivative limit exists:f'(a) = lim_{h → 0} [f(a + h) - f(a)] / h - The Fundamental Theorem: Differentiability implies continuity. The converse is false:
f(x) = |x|is continuous atx = 0, but has left derivative-1and right derivative+1, so it is not differentiable atx = 0.
3. L'Hôpital's Rule & Indeterminate Forms
When evaluating limits producing indeterminate forms 0/0 or ∞/∞, where f and g are differentiable in an open interval around a (except possibly at a) and g'(x) ≠ 0:
For exponential indeterminate forms like 1^∞, 0^0, or ∞^0, convert using natural logarithms: y = f(x)^{g(x)} ⇒ ln y = g(x) ln f(x).
4. Taylor Series & Polynomial Approximation
Taylor series expansion allows us to approximate complex, non-linear objective functions with polynomials around an operating point x = a:
When expanded around a = 0, it is called a Maclaurin series. Key expansions to memorize for GATE:
e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \dotsln(1 + x) = x - \frac{x^2}{2} + \frac{x^3}{3} - \dots \quad \text{for } -1 < x ≤ 1sin(x) = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \dotscos(x) = 1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \dots\frac{1}{1 - x} = 1 + x + x^2 + x^3 + \dots \quad \text{for } |x| < 1
5. Maxima, Minima & Single-Variable Optimization
Single-variable optimization is the foundation for gradient descent step-size tuning and regularizer parameter search:
- Fermat's Theorem (First Derivative Test): If
f(x)has a local extremum at an interior pointcand is differentiable, thenf'(c) = 0. Points wheref'(c) = 0or wheref'(c)is undefined are called critical points. - Second Derivative Test: Let
cbe a critical point wheref'(c) = 0:- If
f''(c) > 0,f(x)is strictly concave up → Local Minimum. - If
f''(c) < 0,f(x)is strictly concave down → Local Maximum. - If
f''(c) = 0, the test is inconclusive (requires higher-order derivative inspection or sign change check; e.g.,x^3has an inflection point atx = 0, whereasx^4has a local minimum).
- If
6. Worked Step-by-Step Examples
Worked Example 1: Limit with Indeterminate Power (1^∞)
Problem: Evaluate L = lim_{x → 0} (1 + 3x)^{2/x}.
Solution: This has the indeterminate form 1^∞.
Take the natural logarithm: ln L = lim_{x → 0} \frac{2}{x} ln(1 + 3x) = 2 \cdot lim_{x → 0} \frac{ln(1 + 3x)}{x}.
Applying L'Hôpital's rule (form 0/0): ln L = 2 \cdot lim_{x → 0} \frac{\frac{3}{1 + 3x}}{1} = 2 \cdot 3 = 6.
Exponentiating gives: L = e^6. (Alternatively, apply standard result lim_{u→0}(1 + u)^{1/u} = e).
Worked Example 2: Taylor Expansion & Truncation Error
Problem: Approximate f(x) = \sqrt{1 + x} up to the second degree around a = 0, and estimate \sqrt{1.2}.
Solution: Compute derivatives at 0:
f(0) = 1
f'(x) = \frac{1}{2}(1 + x)^{-1/2} ⇒ f'(0) = 1/2
f''(x) = -\frac{1}{4}(1 + x)^{-3/2} ⇒ f''(0) = -1/4
The second-order Taylor polynomial is: P_2(x) = 1 + \frac{1}{2}x - \frac{1}{8}x^2.
For x = 0.2: P_2(0.2) = 1 + 0.1 - \frac{0.04}{8} = 1 + 0.1 - 0.005 = 1.095.
Actual value: \sqrt{1.2} \approx 1.095445 (Relative error < 0.04%).
Worked Example 3: Global Optimization on a Closed Interval
Problem: Find the absolute minimum and maximum values of f(x) = 2x^3 - 3x^2 - 12x + 5 on the closed interval [-2, 4].
Solution:
1. Find interior critical points: f'(x) = 6x^2 - 6x - 12 = 6(x^2 - x - 2) = 6(x - 2)(x + 1) = 0 ⇒ x = -1, 2.
Both points lie within [-2, 4].
2. Evaluate f(x) at critical points and boundary endpoints:
f(-2) = 2(-8) - 3(4) - 12(-2) + 5 = -16 - 12 + 24 + 5 = 1f(-1) = 2(-1) - 3(1) - 12(-1) + 5 = -2 - 3 + 12 + 5 = 12f(2) = 2(8) - 3(4) - 12(2) + 5 = 16 - 12 - 24 + 5 = -15f(4) = 2(64) - 3(16) - 12(4) + 5 = 128 - 48 - 48 + 5 = 37
Conclusion: The absolute maximum is 37 (at x = 4) and absolute minimum is -15 (at x = 2).
7. GATE Exam Traps & Common Distractors
- Checking Endpoints: For optimization over a closed interval
[a, b], never forget to evaluate the boundary pointsf(a)andf(b). Extrema frequently lie on the boundaries wheref'(x) ≠ 0. - Local vs Global: Critical points with
f''(x) > 0only guarantee local minima. Global optimality requires either convexity over the domain or exhaustive comparison with boundary values. - L'Hôpital Preconditions: Do not apply L'Hôpital's rule unless you have explicitly verified that the expression is in indeterminate form (
0/0or∞/∞). Differentiating a determinate limit leads to wrong answers.
8. Syllabus Practice & Topic Tracking
Test your understanding with exam-pattern practice problems:
Practice Calculus Questions or review the GATE DA Syllabus Index.