This repository contains my implementations of various optimization methods, including Gradient Descent, Conjugate Gradient Descent, and a Cutting-Plane method. These implementations demonstrate the effectiveness of these algorithms through computational analysis and visualization.
- Purpose: An advanced optimization technique particularly effective for large-scale problems, utilizing direction conjugate to the previous direction.
- Implementation: Python code demonstrating conjugate gradient descent with iterative outputs of input and objective function.
- Purpose: To find local minima of functions using iterative steps proportional to the negative of the gradient.
- Implementation: Python code showcasing basic gradient descent steps along with performance analysis, using various sample functions and step sizes.
- Purpose: A method used in convex optimization that iteratively refines a feasible solution space.
- Implementation: Implementation of the cutting-plane method, illustrating its approach to reducing the feasible region and converging to an optimal solution.
The cutting plane and gradient descent methods are accompanied by visualizations (e.g., convergence plots, function descent paths) that illustrate the behavior and efficiency of the optimization techniques on various sample problems.
- Python: Primary programming language for all implementations.
- Matplotlib and NumPy: Used for data visualization and numerical operations.
- Demonstrated the practical application and comparative analysis of different optimization methods, step size, and objective functions..
- Provided insights into the scenarios where each method is most effective.
Gradient_Descent.ipynb: Code for the Gradient Descent method.Conjugate_Gradient_Gescent.ipynb: Code for the Conjugate Gradient Descent method.Cutting Plane Method (Center of Gravity).ipynb: Code for the Cutting-Plane method.