- Ignizio and Cavalier: Linear Programming, Prentice Hall, 1994. (There are many good LP texts.)
- Baldick: Applied Optimization, Cambridge, 2006. (There are several powerpoint type presentations available on the web at Prof. Baldick's website.)
- Boyd and Vandenberghe, Convex Optimization, Cambridge, 2004.
- Jang, Sun and Mizutani, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Prentice Hall, 1997.
- Various journal papers and notes. See assignment page.

The following list of topics is way too long to cover in a semester so I will pick and choose among the latter topics. Students with a preference for certain topics should feel to make requests.

- Overview (Reading: Baldick Chapter 1 and 2)

- Linear optimization/programming (Reading: Baldick Chapter 4; ref. Ignizio and Cavalier, pp. 10-75; 167-182)
- General form
- Standard form
- Extreme points
- Feasible regions
- Convexity
- Optimality conditions
- Duality
- Relationship to graphs and network structure

- Linear programming solution methods (ref. Ignizio and Cavalier, pp. 80-93; 192-199)
- Simplex method
- Dual simplex method
- Network simplex method (brief overview)

- Interior point methods (brief overview)
- Ellipsoid algorithm
- Karmarkar's projective algorithm
- Computational complexity
- Example applications

- Shortest path and flow problems

- Weighted Least Absolute Value estimation
- WLAV vs Weighted Least Squares (WLS) estimation

- Integer programming (ref. Notes)
- Formulation
- Piece-wise linear transformations convex and non-convex
- Branch and bound method
- Binary programming
- Non-linear integer programming - transformation to linear programming
- Example applications

- Alarm processing using a set covering approach
- Unit commitment or other scheduling problems

- More on Graph formulations (ref. Notes)
- Network structure
- Search methods
- Depth-first
- Breadth-first
- Shortest path algorithms
- Heuristic search
- A* algorithm
- Relationship to linear programming, i.e., solving using LP
- Example applications

- Map problems

- Routing - system restoration

- Deeper concepts of duality (ref. Notes, Chapter 3 Baldick)
- General duality
- Kuhn Tucker conditions

- Convexity and the duality gap
- Lagrangian relaxation
- Example applications

- Scheduling

- Unit commitment

- Using derivative information - review of gradient and related methods
- Gauss/Steepest Descent
- Newton
- Levenberg-Marquardt modifications
- Relationship between zeros of a function and optimal solutions
- Alternatives to the derivative calculations (derivative free) (ref. Jang, et al, pp. 129-195).
- Secant method
- Line section methods (bisection)

- Downhill Simplex search
- Example applications

- TBD

**Data driven optimization - classification and constrained optimization; statistical methods**

- Regression
- Vector support machines
- Example applications

- TBD

**Convexification**

- Review of convexity
- Second order cone problems
- Linear matrix inequalities
- Example applications

- TBD

II. From Optimization to Computational Intelligence - Formulating Useful Decision Problems

- Motivation - some simple techniques to improve problem formulation (ref. Notes, Chapter 3 Baldick)
- Utility theory and goal programming
- Vector optimization
- Multi-objective optimization with soft constraints (ref. Notes, Ignizio and Cavalier, pp. 506-570)
- Goal programming
- Soft constraints
- Interval and fuzzy approaches
- Detailed example
- Vector optimization

- Quadratic programming and successive linear programming -
skipped previously

- Formulation
- Solutions methods
- Example application
- Optimal power flow

- New optimization approaches for complex domains (ref. Jang, et al, pp. 129-195).
- Stochastic methods
- Robust optimization

- Population based methods

- General framework
- Simulated Annealing
- Genetic algorithms
- Particle Swarm

- Example application

- Unit commitment

- Data driven approaches - pattern matching as optimization; network and decision methods (ref. Notes, Jang, et al, pp. 226-238, 316-327)
- Pre-processing of data
- Principle component analysis
- Decision methods
- Bayesian
- Decision trees

- Network methods

- Hopfield neural net
- Stability
- Capacity
- Solving optimization problems
- Example: Hopfield net limits
- Feedforward neural networks
- Activation functions
- Learning methods and parameters
- Design methodologies
- Other nets - Kohonen, Radial basis networks
- Example application

- Electricity price forecasting
- Wind power forecasting

- Game theory
- Traditional mathematical programming methods
- Traditional AI approaches to games (didn't cover)

- Soft computational approaches (didn't cover)
- Example application
- Bidding behavior in electricity markets

Midterm exam - 25%

Project - 25%