What is the difference between parametric and non-parametric models? Provide examples.

• A. Parametric models assume a specific distribution with fixed parameters (linear regression, logistic regression, Gaussian Naive Bayes). Non-parametric models make few assumptions about the data distribution (k-NN, decision trees, Kaplan-Meier).
• B. Parametric models require non-linear relationships; Non-parametric models require linear relationships.
• C. Parametric models never overfit; Non-parametric models always overfit.
• D. Parametric models do not rely on training data; Non-parametric models do.

Answer: (A)

The essential idea is how much you rely on a fixed form for the relationship and how many parameters you estimate. Parametric models assume a specific functional form with a finite number of parameters. Once those parameters are learned, the model is fully described by that form, which makes learning efficient but can limit flexibility if the assumed shape isn’t close to reality. Examples include linear regression (a straight-line relationship with Gaussian errors), logistic regression (a linear combination of features mapped to probabilities via a logistic function), and Gaussian Naive Bayes (assumes features are Gaussian within each class and independent given the class).

Non-parametric models drop the fixed-form assumption, letting the data shape the model more freely. They usually make fewer distributional assumptions and can adapt to complex, irregular relationships, at the cost of needing more data and computation and a higher risk of overfitting if not regularized. Examples include k-nearest neighbors (predictions based on nearby data points without an assumed distribution), decision trees (learn splits directly from data), and Kaplan-Meier (estimates a survival function without assuming a parametric distribution).