Parallel Ray Search and QNewton updates

Allan Struthers

Description

This project is to implement and test a variety of potentially parallel search algorithms and QNewton update schemes.

Outline

Block Quasi Newton schemes where we attempt to update B (or more usefully H) using more than one {s,y}∈(ℝ^(2×n)) pair at a time.  This can be thought of as using matrices {S,Y} where S,Y∈(ℝ^(p×n)) where p is the number of pairs of values.

Explore convergence for random points on Quad SPD.

Attempt to implement limited memory analogues

Examine approximation schemes for the limited memory analogues.

Block evaluation schemes implementing a ray search algorithm that performs "p-direction" line searches:
Compute (Subscript[f, i,j])=f((α^j)(Subscript[p, i]))   for -m≤j≤m 
It should select the smallest and return suitable update values.

Attempt to prove a thm that showing that the Z lemma holds provided the lowest value (Subscript[f, i,j]) satisfies Wolfe 1 and j<m.  I think the proof should look like Backtracking

Test various implementations of the p-directions.  For instance: you can zero out any entry you like, you can positively scale any individual entry. You can interchange any two components and flip one of the signs, ....

References