Numerical Optimization: Project List

Rich and Jason:  Implement and test a variety of nonlinear conjugate gradient algorithms.

Particle Swarm Optimization:

Jack, Alex, and Jason: Implement and test a particle swarm algorithm.  CUDA etc.

Robin, Wen, and Chad: Implement and test a particle swarm algorithm.  Quasi-Newton interactions.

Solving Nonlinear Equations

Ehsan, Hanif, Elaheh: Finding transfer orbits in matlab

Optimization tests in matlab

Changyu and ?: Optimizing MIMO radar stuff in matlab

Fitting accoustic impednace stuff in matlab.

Quasi-Newton schemes:

Implement and test Quasi-Newton schemes (single vector and block vector, and limited memory - BFGS, DFP, SR1, Broyden Class):  Convergence of (Subscript[B, k]) and (Subscript[H, k]); recovery from "bad" values, possibilities for safe guarding.

Implement and test Quasi-Newton schemes (single vector  and limited memory - BFGS, DFP, SR1, Broyden Class):  Convergence of (Subscript[B, k]) and (Subscript[H, k]); recovery from "bad" values, possibilities for safe guarding.

Implement and test a few Quasi-Newton schemes for contrained optimization.

Parallel Stuff

Implement a web search algorithm that performs a "p-direction" line search simultaneoulsy:

Compute (Subscript[f, i,j])=f((α^j)(Subscript[p, i]))   for -m≤j≤m 
It should slect the smallest and return suitable update values. 
Attempt to prove a thm that showing that the Z lemma holds provided the lowest value (Subscript[f, i,j]) satisfies Wolfe 1 and j<m.  I think the proof should look like Backtracking
Test various implementations of the p-directions.  For instance: you can zero out any entry you like, you can positively scale any individual entry. You can interchange any two components and flip one of the signs, .... 
Contemplate what this would mean for constrained optimization.

Implement a p-directional positivity scheme.  Roughly speaking the Cholesky decomposition trick for +def is to find a minimal change to the Hessian which preserves +def.  i.e. something like Min||(∇^2)f-(OverHat[A])|| and with (OverHat[A]) SPD and then solve (OverHat[A]).(Subscript[p, n])=-∇f to get our modifed Newton like direction.  To do this we need to generate the complete Hessian and then throw some of it away.  Maybe instead it might be a good idea to generate some bits of the Hessian and use them all.   Maybe something like:	 min ||(∇^2)f.p+∇f||  subject to the descent constraint p.∇f<0.  The bad news about this is that in most cases in high dimensions minimally constrained problems have tight constraints so one might need to modify the constraint.  We could also do the minimization over a restricted set of directions.  Maybe something like p=-(Subscript[α, 0])∇f+(Subscript[α, 1])(Subscript[r, 1])+(Subscript[α, 2])(Subscript[r, 2])+… where we construct the rs so that (Subscript[r, i]).∇f=0.  test the behavior of various possibilities.  Contemplate what this would mean for constrained optimzation. As an additional feature of this sort of problem we would not need all the components of (∇^2)f for such a computation.

AMBER Models:

Build interesting AMBER models of a range of sizes in matlab and test the behavior of the various optimization algorithms.  Ours and built-in

Build interesting AMBER models in mathematica of a range of sizes and test the behavior of the various optimization algorithms. Ours and built-in