1. Types of Learning
Supervised — labeled data (classification, regression)
Unsupervised — no labels (clustering, dimensionality reduction)
Reinforcement — agent learns via rewards
Pipeline: data → preprocess → train → validate → test → deploy
2. Regression & Classification
Linear Regression: y = w·x + b, minimize MSE
Logistic Regression: sigmoid for binary classification
Decision Trees, SVM, k-NN, Naive Bayes — classic algorithms
Gradient Descent: iterative weight update
η (learning rate) too high → diverge; too low → slow
3. Model Evaluation
Classification metrics
- Accuracy = (TP+TN)/total
- Precision = TP/(TP+FP)
- Recall = TP/(TP+FN)
- F1 = harmonic mean of P & R
- ROC-AUC for ranking quality
Regression: MSE, RMSE, MAE, R²
Cross-validation (k-fold) reduces variance in estimates.
4. Overfitting & Regularization
Overfitting: low train error, high test error
Underfitting: high train & test error
Fixes
- More data, simpler model
- Dropout, early stopping
- L1 (Lasso) / L2 (Ridge) regularization
- Cross-validation for hyperparams
Bias-Variance: high bias → underfit; high variance → overfit