visualization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizations of the modified loss landscape $\\mathbf{f_\\lambda}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is used for generating plots of how the (linearized) piecewise constant loss $f$ changes with hyperparameter $\\lambda$. The method is proposed in [Differentiation of Blackbox Combinatorial Solvers](https://arxiv.org/abs/1912.02175). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Implemented are solvers for **GraphMatching**, **MultiGraphMatching**, **TravellingSalesman**, **Ranking**, and **ShortestPath.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The landscape is built by integrating the gradient calculated as $\\nabla f_\\lambda (w) = -\\frac{1}{\\lambda} [y(w) - y_\\lambda (w)]$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test iPyVolume plotting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run to test whether ipyvolume installation succeeded (and all required jupyter extentions are enabled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import ipyvolume as ipv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/homebrew/Caskroom/miniforge/base/envs/bb-backprop/lib/python3.12/site-packages/ipyvolume/serialize.py:102: RuntimeWarning: invalid value encountered in cast\n",
      "  subdata[..., i] = ((gradient[i][zindex] / 2.0 + 0.5) * 255).astype(np.uint8)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0e2bb9dbd4ed4fec89f1ebeb8f21f77d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Container(children=[VBox(children=(HBox(children=(Label(value='levels:'), FloatSlider(value=0.25, max=1.0, ste…"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V = np.zeros((128,128,128)) # our 3d array\n",
    "# outer box\n",
    "V[30:-30,30:-30,30:-30] = 0.75\n",
    "V[35:-35,35:-35,35:-35] = 0.0\n",
    "# inner box\n",
    "V[50:-50,50:-50,50:-50] = 0.25\n",
    "V[55:-55,55:-55,55:-55] = 0.0\n",
    "ipv.quickvolshow(V, level=[0.25, 0.75], opacity=0.03, level_width=0.1, data_min=0, data_max=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loss landscape plots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some example problems are generated in the following:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The magic constants are selected so that we obtain interesting sections of the high-dimensional function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "WARNING: It's 2D sections of high-D functions, sometimes may not be intuitive (interpolation happens in different dimension)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lpmp_py missing. Install it separately for using (multi)graph matching solvers\n",
      "GurobiPy missing, TSP module not available\n"
     ]
    }
   ],
   "source": [
    "from solvers_to_visualize import RankingBBSolver, TSPBBSolver, GraphMatchingBBSolver,  \\\n",
    "                                 ShortestPathBBSolver, MultiGraphMatchingBBSolver"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ranking"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# lambdas = [1e-6, 1, 2, 4, 8, 16]\n",
    "# rk_solv = RankingBBSolver(sequence_length=7, seed=45, w_normal_factor=30.0, w_shift_factor=20.0, y_addend=1)  # the last three arguments change the values of w and y_grad\n",
    "# rk_solv.plot_flambda(lambdas=lambdas, partitions=80, bounds1=(0.0, 1.0), bounds2=(0.0, 1.0), show_axes=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TSP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# tsp_solv = TSPBBSolver(num_nodes=5, seed=42, w_normal_factor=10, w_shift_factor=20)\n",
    "# tsp_solv.plot_flambda(lambdas=lambdas, partitions=10, bounds1=(-1.0, 1.0), bounds2=(-1.0, 1.0), show_axes=False, show_box=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph Matching"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# lambdas = [1e-6, 10, 20, 40, 80, 160]\n",
    "# gm_solv = GraphMatchingBBSolver(num_nodes_l=[3, 3], num_edges_l=[2, 2], seed=43, w_normal_factor=10, y_factor=0.1)\n",
    "# gm_solv.plot_flambda(lambdas=lambdas, partitions=80, bounds1=(-1.0, 1.0), bounds2=(-1.0, 1.0), show_axes=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shortest path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "66a009b5c2c341d39e94e1dc8e9bfa08",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Container(children=[HBox(children=(Play(value=0, interval=400, max=4), IntSlider(value=0, max=4)))], figure=Fi…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "lambdas = [1e-6, 0.5, 1, 2, 4]\n",
    "sp_solv = ShortestPathBBSolver(num_nodes=6, seed=30, w_normal_factor=10, y_factor=1, w_shift_addend=2)\n",
    "sp_solv.plot_flambda(lambdas=lambdas, partitions=10, bounds1=(0.0, 1.0), bounds2=(0.0, 1.0), show_axes=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multigraph matching"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# lambdas = [1e-6, 5, 10, 20, 40, 80]\n",
    "# mgm_solv = MultiGraphMatchingBBSolver(num_nodes_l=[3, 3], num_edges_l=[2, 2], seed=42, w_normal_factor=10, y_factor=0.1)\n",
    "# mgm_solv.plot_flambda(lambdas=lambdas, bounds1=(-2.0,2.0), bounds2=(-2.0,2.0), partitions=60, show_axes=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Customize! Run your own solvers and cuts!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Problems can be added by specifying the solver and the input-generation function as shown below.\n",
    "The input-generation should provide:\n",
    "- the config for the solver\n",
    "- a two-dimensional cut of a higher dimensional $w$-landscape \n",
    "- some gradient $\\frac{dL}{dy}$ of the loss with respect to $y$, for which the \"continuous interpolation\" $f_\\lambda$ should be plotted. \n",
    "\n",
    "For details on the provided helper functions to generate these inputs, as well as the calculations done for the plots, see utils.flamba_utils."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The challenge is always to find values for the $w$-cut and the $\\frac{dL}{dy}$ that produce interesting results, as in producing multiple different outputs of the solver. To find these regions there are various parameters that can be changed, including shifts and factors for changing the randomized inputs to values that \"fit\" the solver, as well as the seed. The boundaries for the plotting region can also be changed."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "bb-backprop",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}