This is the continuation of Understanding Bias-Variance Trade-Off (Part 1)
The trade-off between bias and variance can be depicted as, model with low bias will be flexible enough to fit the data well. But if it becomes too flexible it will memorize and overfit the data instead of generalizing it and it won’t perform well on a different but similar dataset i.e. error due to high variance. n the figure below, we see a plot of the model’s performance using prediction capability on the vertical axis as a function of model complexity on the horizontal axis. Here, we depict the case where we use a number of different orders of polynomial functions to approximate the target function. Shown in the figure are the calculated square bias, variance, and error on the test set for each of the estimator functions.
We see that as the model complexity increases, the variance slowly increases and the squared bias decreases. This points to the tradeoff between bias and variance due to model complexity, i.e. models that are too complex tend to have high variance and low bias, while models that are too simple will tend to have high bias and low variance. The best model will have both low bias and low variance.
Managing the bias-variance tradeoff is one great example of how experienced data scientists serve an integral part of an analytics-driven business.
Top comments (0)