It’s helpful to add a regularization term to your model so that as you’re building it, you penalize high parameter models against low parameter models. It’s commonly done in ML contexts but the idea is sound.
A similar idea in genetic algorithms is to penalize the length of the “genome” being optimized.
All sort of related to the bias variance trade off, overfitting, and the fact that with sufficient order of polynomial you can “perfectly” fit any number of points to create the worst approximator.
In case anyone is curious, the most aggressive form of regularization is sometimes called L0 regularization or sparse regression. Here, there is a penalty term for every additional parameter used in the model. This allows you to define a cost associated with model complexity (i.e. if an improvement to the fit is not greater than some value, the number of parameters is not increased). Solving for the best set of parameters can be tricky and thus LASSO (or L1 regularization) is often a good substitute.
A similar idea in genetic algorithms is to penalize the length of the “genome” being optimized.
All sort of related to the bias variance trade off, overfitting, and the fact that with sufficient order of polynomial you can “perfectly” fit any number of points to create the worst approximator.