Using scipy's genetic algorithm for initial parameter estimation in non-linear regression

[zunzun]

I see on github that you are writing optimization code using Python. When using non-linear solvers such as Levenberg-Marquardt or gradient descent to optimize parameters for non-linear regression, there is a general problem of initial starting parameters. With poor initial parameter values to start the "descent" into error space, the algorithms can stop in a local error minimum. For this reason, the authors of scipy have added a genetic algorithm for initial parameter estimation to use with scipy's non-linear solvers. The module is named scipy.optimize.differential_evolution. Scipy's implementation uses the Latin Hypercube algorithm to ensure a thorough search of parameter space.

I have used scipy's Differential Evolution genetic algorithm to determine initial parameters for fitting a double Lorentzian peak equation to Raman spectroscopy data of carbon nanotubes and found that the results were excellent. The GitHub project, with a test spectroscopy data file, is:

https://github.com/zunzun/RamanSpectroscopyFit

My background is in nuclear engineering and industrial radiation physics, and I love Python, so if you have any questions please let me know.

James Phillips

[vsmolyakov]

Hi James,

Thanks for your post! I'll check out SciPy's differential evolution for initial parameter estimation. In the past, I've used Bayesian Optimization that utilizes Gaussian Process to explore parameter space in a way that trades off exploration and exploitation. The goal is to minimize the number of parameter queries in the case where it's expensive to evaluate performance for a given set of initial parameters. Spearmint is another popular library for Bayesian Optimization.

Vadim

[zunzun]

Cool. I had not heard of Spearmint.