Machine Learning

Concept: Machine learning algorithms are broadly categorized into supervised and unsupervised learning based on the availability of labeled data and the learning objective. Step 1: {\color{red}Supervised Learning}
Supervised learning involves training a model using labeled datasets:

• Each input has a known output (label)
• The model learns to map inputs to outputs

Examples:

• Email spam detection (spam vs non-spam)
• House price prediction based on features
• Medical diagnosis using patient data

Step 2: {\color{red}Unsupervised Learning}
Unsupervised learning works with unlabeled data:

• No predefined outputs
• The model identifies hidden patterns or structures

Examples:

• Customer segmentation in marketing
• Grouping similar news articles
• Anomaly detection in fraud analysis

Step 3: {\color{red}Key Differences}

• Data: Supervised uses labeled data; unsupervised uses unlabeled data
• Goal: Prediction vs pattern discovery
• Output: Known outcomes vs hidden structures

Step 4: {\color{red}Use Case Perspective}

• Use supervised learning when historical labeled data exists
• Use unsupervised learning for exploration and insights

Concept: Linear Regression is one of the simplest and most widely used machine learning algorithms for predicting a continuous output based on input features. It assumes a linear relationship between independent variables and the dependent variable. Step 1: {\color{red}What is Linear Regression?}
Linear regression models the relationship using a linear equation: where:

• = predicted output
• = input feature
• = slope (weight)
• = intercept (bias)

For multiple features, the equation extends to multiple linear regression.
Step 2: {\color{red}Goal of Linear Regression}
The objective is to:

• Find the best-fitting line
• Minimize the difference between predicted and actual values

This difference is called the error or residual.
Step 3: {\color{red}Mean Squared Error (MSE) Loss Function}
MSE is used to quantify prediction error: where:

• = actual value
• = predicted value
• = number of samples

Step 4: {\color{red}Role of MSE in Training}
MSE plays a critical role by:

• Penalizing larger errors more heavily (due to squaring)
• Providing a smooth, differentiable loss function
• Enabling optimization using gradient descent

Step 5: {\color{red}Why MSE is Preferred}

• Simple and mathematically convenient
• Works well with linear models
• Helps achieve a unique optimal solution

Concept: Gradient Descent is a widely used optimization algorithm in machine learning and deep learning. It helps find the optimal values of model parameters (weights and biases) by minimizing a loss function, thereby improving prediction accuracy. Step 1: {\color{red}What is Gradient Descent?}
Gradient Descent is an iterative optimization method that:

• Calculates the gradient (slope) of the loss function
• Updates parameters in the opposite direction of the gradient

This ensures movement toward the minimum loss.
Step 2: {\color{red}Basic Update Rule}
The parameter update is given by: where:

• = model parameters
• = learning rate
• = gradient of the loss function

Step 3: {\color{red}Role in Model Optimization}
Gradient Descent helps by:

• Reducing prediction errors
• Finding parameter values that minimize loss
• Improving model accuracy over iterations

Step 4: {\color{red}Learning Rate Importance}
The learning rate controls step size:

• Too large → overshooting the minimum
• Too small → slow convergence

Choosing the right value is critical for optimization. Step 5: {\color{red}Variants of Gradient Descent}

• Batch Gradient Descent — uses entire dataset
• Stochastic Gradient Descent (SGD) — updates per sample
• Mini-batch Gradient Descent — balance of speed and stability