Introduction to Machine Learning
Foundations and end-to-end workflow
What is Machine Learning?
Machine Learning is the study of algorithms that improve automatically through experience (data). Rather than hard-coding rules, we train models to map inputs to outputs.
- Goal: Learn f(x) ≈ y that generalizes to unseen data
- Data: Features (X) and sometimes labels (y)
- Model: Parametric or non-parametric hypothesis
- Learning: Optimize parameters to minimize loss
Types of Learning
- Supervised: Learn from labeled data (regression/classification)
- Unsupervised: Discover structure in unlabeled data (clustering, PCA)
- Semi-supervised: Mix of labeled and unlabeled data
- Reinforcement: Learn by interacting with environment via rewards
ML Pipeline
- Problem framing and data collection
- Data cleaning and preprocessing
- Feature engineering and selection
- Model training with cross-validation
- Evaluation on hold-out test set
- Deployment and monitoring (MLOps)
Data Preprocessing
- Handling missing values (drop, impute)
- Scaling and normalization (MinMax, StandardScaler)
- Encoding categorical variables (One-Hot, Ordinal)
- Outlier detection and treatment
- Train/validation/test split
Tip
Always fit preprocessing steps on training data only to avoid data leakage.
Evaluation Metrics
| Task | Metrics |
|---|---|
| Regression | MSE, RMSE, MAE, R² |
| Classification | Accuracy, Precision, Recall, F1, ROC-AUC |
| Clustering | Silhouette score, Davies–Bouldin, Calinski–Harabasz |
Bias–Variance Trade-off (Cheat Sheet)
Quick revision
- Bias: Error from wrong model assumptions (underfitting). High bias → simple model, poor train accuracy.
- Variance: Error from sensitivity to training data (overfitting). High variance → complex model, large gap between train and test accuracy.
- Goal: Find model complexity that minimizes total error = bias² + variance + noise.
- Practical tips: use cross-validation, regularization, more data, feature selection, ensemble methods.
- Visual cue: Learning curve: if both train & val errors are high → increase model complexity or features (reduce bias); if train low but val high → add regularization or more data (reduce variance).