- Ignizio and Cavalier: Linear Programming, Prentice Hall, 1994. (There are many good LP texts.)
- Baldick: Applied Optimization, Cambridge, 2006. (There are several powerpoint type presentations available on the web at Prof. Baldick's website.)
- Boyd and Vandenberghe, Convex Optimization, Cambridge, 2004.
- Jang, Sun and Mizutani, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Prentice Hall, 1997.
- Various journal papers and notes. See assignment page.

The following list of topics is way too long to cover in a semester so I will pick and choose among the latter topics. Students with a preference for certain topics should feel to make requests.

- Overview (Reading: Baldick Chapter 1 and 2)

- Linear optimization/programming (Reading: Baldick Chapter 4; ref. Ignizio and Cavalier, pp. 10-75; 167-182)
- General form
- Standard form
- Extreme points
- Feasible regions
- Convexity
- Optimality conditions
- Duality
- Relationship to graphs and network structure

- Linear programming solution methods (ref. Ignizio and Cavalier, pp. 80-93; 192-199)
- Simplex method
- Dual simplex method
- Network simplex method (brief overview)

- Interior point methods (brief overview)
- Ellipsoid algorithm
- Karmarkar's projective algorithm
- Computational complexity
- Example applications

- Shortest path and flow problems

- Weighted Least Absolute Value estimation
- WLAV vs Weighted Least Squares (WLS) estimation

- Integer programming (ref. Notes)
- Formulation
- Piece-wise linear transformations convex and non-convex
- Branch and bound method
- Binary programming
- Non-linear integer programming - transformation to linear programming
- Example applications

- Alarm processing using a set covering approach
- Unit commitment or other scheduling problems

- Graph formulations (ref. Notes)
- Network structure
- Search methods
- Depth-first
- Breadth-first
- Shortest path algorithms
- Heuristic search
- A* algorithm
- Relationship to linear programming, i.e., solving using LP
- Example applications

- Map problems

- Routing - system restoration

- Deeper concepts of duality (ref. Notes, Chapter 3 Baldick)
- General duality
- Karesh Kuhn Tucker conditions

- Convexity and the duality gap
- Lagrangian relaxation
- Example applications

- Scheduling

- Unit commitment
- Constrained network problems with Lagrangian Relations - my lightly annotated version on example from MIT Open Course

- Using derivative information - review of gradient and related methods
- Gauss/Steepest Descent
- Newton
- Levenberg-Marquardt modifications
- Relationship between zeros of a function and optimal solutions
- Alternatives to the derivative calculations (derivative free) (ref. Jang, et al, pp. 129-195).
- Secant method
- Line section methods (bisection)

- Downhill Simplex search
**Quadratic programming and successive quadratic programming**

- Formulation
- Solutions methods - reformulation as equivalent to a linear program
- Example applications

- Coming later in vector support machines

- Simple techniques to improve problem formulation (ref. Notes, Chapter 3 Baldick)
- Utility theory and goal programming
- Multi-objectives and soft constraints (ref. Notes, Ignizio and Cavalier, pp. 506-570)
- Objectives vs. constraints - levels of aspirations
- Multi-objectives
- Trade-offs
- Soft constraints: interval or fuzzy approaches
- Detailed example
- Vector optimization

**Data driven optimization - classification and constrained optimization; statistical methods**

- Vector support machines - linear program solutions
- Regression and traditional statistical methods
- Example applications

- Load forecasting

- Other data driven approaches - pattern matching as optimization; network and decision methods (ref. Notes, Jang, et al, pp. 226-238, 316-327)
- Pre-processing of data
- Principle component analysis
- Decision methods
- Bayesian
- Decision trees

- Network methods

- Hopfield neural net -
*skipped for now* - Stability
- Capacity
- Solving optimization problems
- Example: Hopfield net limits
- Feedforward neural networks
- Activation functions
- Learning methods and parameters
- Training as an optimization problem
- Example applications

- Electricity price forecasting
- Wind power forecasting

- Competing interests - game theory as
an optimization

- Traditional mathematical programming methods
- Relationship to linear programming and zero sum
- Traditional AI approaches to games -
*skipped for now* - Soft computational approaches -
*skipped for now* - Example application
- Bidding behavior in electricity markets

- New optimization approaches for complex domains (ref. Jang, et al, pp. 129-195).
- Stochastic methods
- Robust optimization

- Population based methods

- General framework
- Simulated Annealing
- Genetic algorithms
- Particle Swarm

- Example application

- Unit commitment

**Putting a problem into a solvable structure - Convexificaiton**

- Review of convexity
- Second order cone problems
- Linear matrix inequalities
- Interior point methods and barrier functions
- Example applications

- OPF

Midterm exam - 25%

Project - 25%