Skip to content

Commit

Permalink
Added links to TomTom notebooks, run the pre-commit hooks and fixed s…
Browse files Browse the repository at this point in the history 
…ome typos.
  • Loading branch information
@batic
batic committed Sep 23, 2023
1 parent 3f2bae0 commit 2600caa
Show file tree
Hide file tree
Showing 3 changed files with 72 additions and 50 deletions.
4 changes: 3 additions & 1 deletion GEM-data/gem-datasets.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@
" * Results of LightGBM categorical built-up model\n",
" * Results of several TFCN regression and classification built-up models\n",
"\n",
"- [Map making use-case](GEM-map-making-usecase.ipynb)\n",
"- [Map making use-case](gem-map-making-usecase.ipynb)\n",
"\n",
"- [LC-CMS use-case](gem-lc-cms-usecase.ipynb)\n",
"\n",
"- [Continuous monitoring service](gem-NDWI-anomalies.ipynb)\n",
"\n",
Expand Down
79 changes: 50 additions & 29 deletions GEM-data/gem-lc-cms-usecase.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"source": [
"This notebook contains information about accessing the data for LC-CMS use case.\n",
"\n",
"The tutorial focusses on the accessing and visualizing the prediction results derived from the LC-CMS use case.\n",
"The tutorial focuses on the accessing and visualizing the prediction results derived from the LC-CMS use case.\n",
"The results are also available on the Map Viewer which can be accessed using this **[link](https://www.globalearthmonitor.eu/sites/default/files/LC_CMS/index.html)**.\n",
"\n",
"The predictions are stored in _[BYOC SentinelHub layers](https://docs.sentinel-hub.com/api/latest/data/byoc/)_. \n",
Expand Down Expand Up @@ -58,14 +58,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Predicitons over pilot Africa AOI\n",
"# Predictions over pilot Africa AOI\n",
"Following code snippets show how to access and visualize the predictions generated using LC-CMS pipeline. </br>\n",
"</br>\n",
"The predictions were generated using following input data and modelling technique : </br>\n",
"\n",
"**Input Data**\n",
"- Sentinel-2 L2A : Bands B1 to B12\n",
"- 120m with bilinear resampling​\n",
"- 120m with bi-linear resampling\n",
"- Sub-regions (AOIs) with different biomes and geographies \n",
"- Yearly data aggregated every two months\n",
"- Predictions generated yearly\n",
Expand All @@ -74,7 +74,7 @@
"- Pixel Based Classification using Random Forest Classifier\n",
"- 9 Classes\n",
"\n",
"Additionaly, LC-Classes are represented by following values in the prediction array: \n",
"Additionally, LC-Classes are represented by following values in the prediction array: \n",
"\n",
"| **Land Cover Class** | **Value** |\n",
"| -------------------- | --------- |\n",
Expand Down Expand Up @@ -138,7 +138,7 @@
],
"source": [
"nigeria_bbox = BBox((643000.0, 1105000.0, 715000.0, 1175000.0), CRS.UTM_31N)\n",
"eopatch = predictions_download_task.execute(bbox=nigeria_bbox) #, time_interval=[\"2019-01-01\", \"2023-01-01\"]"
"eopatch = predictions_download_task.execute(bbox=nigeria_bbox) # , time_interval=[\"2019-01-01\", \"2023-01-01\"]"
]
},
{
Expand All @@ -162,11 +162,22 @@
" \"\"\"\n",
" Function to transform the predictions to RGB for visualization\n",
" \"\"\"\n",
" ids_to_rgb = {0: [0, 0, 0], 1: [23, 91, 55], 3: [178, 223, 186], 4: [72, 234, 37], 6: [243, 245, 234], 7: [254, 247, 180], 8: [245, 42, 5], 9: [50, 74, 221], 10: [140, 203, 255], 11: [111, 124, 128]}\n",
" resulr_arr = np.zeros((preds.shape[0], preds.shape[1], 3), dtype=np.uint8)\n",
" for id, mapping in ids_to_rgb.items():\n",
" resulr_arr[np.where(preds == id)] = np.array(mapping)\n",
" return resulr_arr"
" ids_to_rgb = {\n",
" 0: [0, 0, 0],\n",
" 1: [23, 91, 55],\n",
" 3: [178, 223, 186],\n",
" 4: [72, 234, 37],\n",
" 6: [243, 245, 234],\n",
" 7: [254, 247, 180],\n",
" 8: [245, 42, 5],\n",
" 9: [50, 74, 221],\n",
" 10: [140, 203, 255],\n",
" 11: [111, 124, 128],\n",
" }\n",
" result_arr = np.zeros((preds.shape[0], preds.shape[1], 3), dtype=np.uint8)\n",
" for idx, mapping in ids_to_rgb.items():\n",
" result_arr[np.where(preds == idx)] = np.array(mapping)\n",
" return result_arr"
]
},
{
Expand Down Expand Up @@ -196,7 +207,7 @@
}
],
"source": [
"# Generate a plot for each year \n",
"# Generate a plot for each year\n",
"# we visualize the predictions using transform_to_rgb function\n",
"preds_data = eopatch.data[\"lcms_pred\"].astype(int).squeeze(axis=0)\n",
"y2019_preds = preds_data[:, :, 0]\n",
Expand All @@ -206,11 +217,11 @@
"fig, axes = plt.subplots(1, 3, figsize=(30, 10))\n",
"ax1, ax2, ax3 = axes\n",
"ax1.set_title(\"For year 2019\", fontweight=\"bold\")\n",
"ax1.imshow(transform_to_rgb(y2019_preds),interpolation=\"nearest\")\n",
"ax1.imshow(transform_to_rgb(y2019_preds), interpolation=\"nearest\")\n",
"ax2.set_title(\"For year 2020\", fontweight=\"bold\")\n",
"ax2.imshow(transform_to_rgb(y2020_preds),interpolation=\"nearest\")\n",
"ax2.imshow(transform_to_rgb(y2020_preds), interpolation=\"nearest\")\n",
"ax3.set_title(\"For year 2021\", fontweight=\"bold\")\n",
"ax3.imshow(transform_to_rgb(y2021_preds),interpolation=\"nearest\")\n",
"ax3.imshow(transform_to_rgb(y2021_preds), interpolation=\"nearest\")\n",
"plt.suptitle(\"LCMS Predictions for Kainji Lake, Nigeria in the pilot AOI Africa\", y=0.9, fontweight=\"bold\", fontsize=15)"
]
},
Expand All @@ -232,7 +243,7 @@
"# Similarly, we can visualize the data mask\n",
"def get_pred_diff(preds_from, preds_to):\n",
" \"\"\"\n",
" This function generates the diff between the the LC-CMS predictions taken over two different time periods.\n",
" This function generates the diff between the LC-CMS predictions taken over two different time periods.\n",
" \"\"\"\n",
" return np.where(preds_from != preds_to, preds_to, 0)"
]
Expand Down Expand Up @@ -277,13 +288,21 @@
"fig, axes = plt.subplots(1, 3, figsize=(30, 10))\n",
"ax1, ax2, ax3 = axes\n",
"ax1.set_title(\"Between 2019 to 2020\", fontweight=\"bold\")\n",
"ax1.imshow(transform_to_rgb(diff_2019_2020),interpolation=\"nearest\")\n",
"ax1.imshow(transform_to_rgb(diff_2019_2020), interpolation=\"nearest\")\n",
"ax2.set_title(\"Between 2020 to 2021\", fontweight=\"bold\")\n",
"ax2.imshow(transform_to_rgb(diff_2020_2021),interpolation=\"nearest\")\n",
"ax2.imshow(transform_to_rgb(diff_2020_2021), interpolation=\"nearest\")\n",
"ax3.set_title(\"Between 2019 to 2021\", fontweight=\"bold\")\n",
"ax3.imshow(transform_to_rgb(diff_2019_2021),interpolation=\"nearest\")\n",
"# plt.suptitle('LC-CMS Predictions for Nigeration in the pilot AOI Africa for years 2019, 2020, 2021')\n",
"plt.suptitle(\"LCMS Predictions Delta (1 and 2-year change) for the Kainji Lake, Nigeria from 2019 to 2021 with various combinations\", y=0.9, fontweight=\"bold\", fontsize=15)"
"ax3.imshow(transform_to_rgb(diff_2019_2021), interpolation=\"nearest\")\n",
"\n",
"plt.suptitle(\n",
" (\n",
" \"LCMS Predictions Delta (1 and 2-year change) for the Kainji Lake, Nigeria from 2019 to 2021 with various\"\n",
" \" combinations\"\n",
" ),\n",
" y=0.9,\n",
" fontweight=\"bold\",\n",
" fontsize=15,\n",
")"
]
},
{
Expand All @@ -302,12 +321,12 @@
"source": [
"# Predictions over experiment AOI of France\n",
"In the following code snippets, we will see how we can access the data for the France AOI. </br>\n",
"France AOI was primariliy used for experimentation and model tuning. </br>\n",
"France AOI was primarily used for experimentation and model tuning. </br>\n",
"\n",
"The final predictions that we will see were generated using following input data and modelling technique : </br>\n",
"**Input Data**\n",
"- Sentinel-2 L2A : Bands B1 to B12\n",
"- 120m with bilinear resampling​\n",
"- 120m with bi-linear resampling\n",
"- Sub-regions (AOIs) with different biomes and geographies \n",
"- Yearly data aggregated every two months\n",
"- Predictions generated yearly\n",
Expand Down Expand Up @@ -365,7 +384,7 @@
" max_threads=3,\n",
")\n",
"\n",
"eopatch_paris = predictions_download_task.execute(bbox=paris_bbox) #, time_interval=[\"2019-01-01\", \"2023-01-01\"]\n",
"eopatch_paris = predictions_download_task.execute(bbox=paris_bbox) # , time_interval=[\"2019-01-01\", \"2023-01-01\"]\n",
"paris_120m = eopatch_paris.data[\"lcms_pred\"].astype(int).squeeze(axis=0)"
]
},
Expand Down Expand Up @@ -404,7 +423,7 @@
" max_threads=3,\n",
")\n",
"\n",
"eopatch_paris = predictions_download_task.execute(bbox=paris_bbox) #, time_interval=[\"2019-01-01\", \"2023-01-01\"]\n",
"eopatch_paris = predictions_download_task.execute(bbox=paris_bbox) # , time_interval=[\"2019-01-01\", \"2023-01-01\"]\n",
"paris_60m = eopatch_paris.data[\"lcms_pred\"].astype(int).squeeze(axis=0)"
]
},
Expand Down Expand Up @@ -443,7 +462,7 @@
" max_threads=3,\n",
")\n",
"\n",
"eopatch_paris = predictions_download_task.execute(bbox=paris_bbox) #, time_interval=[\"2019-01-01\", \"2023-01-01\"]\n",
"eopatch_paris = predictions_download_task.execute(bbox=paris_bbox) # , time_interval=[\"2019-01-01\", \"2023-01-01\"]\n",
"paris_20m = eopatch_paris.data[\"lcms_pred\"].astype(int).squeeze(axis=0)"
]
},
Expand Down Expand Up @@ -486,12 +505,14 @@
"fig, axes = plt.subplots(1, 3, figsize=(30, 10))\n",
"ax1, ax2, ax3 = axes\n",
"ax1.set_title(\"120m resolution\", fontweight=\"bold\")\n",
"ax1.imshow(transform_to_rgb(paris_120m.squeeze(-1)),interpolation=\"nearest\")\n",
"ax1.imshow(transform_to_rgb(paris_120m.squeeze(-1)), interpolation=\"nearest\")\n",
"ax2.set_title(\"60m resolution\", fontweight=\"bold\")\n",
"ax2.imshow(transform_to_rgb(paris_60m.squeeze(-1)),interpolation=\"nearest\")\n",
"ax2.imshow(transform_to_rgb(paris_60m.squeeze(-1)), interpolation=\"nearest\")\n",
"ax3.set_title(\"20m resolution\", fontweight=\"bold\")\n",
"ax3.imshow(transform_to_rgb(paris_20m.squeeze(-1)),interpolation=\"nearest\")\n",
"plt.suptitle(\"LCMS Predictions for Paris region in France AOI for different resolutions\", y=0.8, fontweight=\"bold\", fontsize=15)"
"ax3.imshow(transform_to_rgb(paris_20m.squeeze(-1)), interpolation=\"nearest\")\n",
"plt.suptitle(\n",
" \"LCMS Predictions for Paris region in France AOI for different resolutions\", y=0.8, fontweight=\"bold\", fontsize=15\n",
")"
]
}
],
Expand Down
39 changes: 19 additions & 20 deletions GEM-data/gem-map-making-usecase.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,8 @@
"metadata": {},
"source": [
"# Map Making use case - Data Access\n",
"<!-- <img src=\"./figs/lccms_pipeline.png\" height=\"400\"/> -->\n",
"\n",
"The tutorial focusses on the accessing and visualizing the prediction results derived from the **Map Making** use case. </br>\n",
"The tutorial focuses on the accessing and visualizing the prediction results derived from the **Map Making** use case. </br>\n",
"The results are also available on the Map Viewer which can be accessed using this **[link](https://www.globalearthmonitor.eu/sites/default/files/water/index.html)**.\n",
"\n",
"The predictions are stored in _[BYOC SentinelHub layers](https://docs.sentinel-hub.com/api/latest/data/byoc/)_. \n",
Expand All @@ -30,13 +29,13 @@
"source": [
"# imports\n",
"%matplotlib inline\n",
"import matplotlib\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"\n",
"from eolearn.core import FeatureType\n",
"from eolearn.io import SentinelHubInputTask\n",
"from sentinelhub import CRS, Band, BBox, DataCollection, Unit\n",
"import matplotlib"
"from sentinelhub import CRS, Band, BBox, DataCollection, Unit"
]
},
{
Expand Down Expand Up @@ -115,7 +114,7 @@
"source": [
"# we choose a bounding box that covers a small area in Uruguay\n",
"uruguay_bbox = BBox((452100.0, 6328700.0, 474000.0, 6350700.0), CRS.UTM_21S)\n",
"eopatch = predictions_download_task.execute(bbox=uruguay_bbox) #, time_interval=[\"2019-01-01\", \"2023-01-01\"]"
"eopatch = predictions_download_task.execute(bbox=uruguay_bbox) # , time_interval=[\"2019-01-01\", \"2023-01-01\"]"
]
},
{
Expand Down Expand Up @@ -163,13 +162,13 @@
}
],
"source": [
"# Generate a plot for each year \n",
"# Generate a plot for each year\n",
"# we visualize the predictions using transform_to_rgb function\n",
"preds_data = eopatch.data[\"pred\"].astype(int).squeeze()\n",
"\n",
"fig, ax1 = plt.subplots(figsize=(10, 10))\n",
"ax1.set_title(\"Water Predictions for a lake in Uruguay AOI for year 2019\", fontweight=\"bold\")\n",
"ax1.imshow(transform_to_rgb(preds_data),interpolation=\"nearest\")"
"ax1.imshow(transform_to_rgb(preds_data), interpolation=\"nearest\")"
]
},
{
Expand All @@ -195,9 +194,9 @@
}
],
"source": [
"# Probabilities are stored in a separate collection with bands named 'PW' and 'IW' \n",
"# Probabilities are stored in a separate collection with bands named 'PW' and 'IW'\n",
"bands = [\"PW\", \"IW\"]\n",
"pred_collection_id = '6da384a4-1ea1-40a6-b855-0b4ad954671f'\n",
"pred_collection_id = \"6da384a4-1ea1-40a6-b855-0b4ad954671f\"\n",
"\n",
"byoc_collection = DataCollection.define_byoc(\n",
" pred_collection_id,\n",
Expand Down Expand Up @@ -230,10 +229,10 @@
"fig, axes = plt.subplots(1, 2, figsize=(14, 7))\n",
"ax1, ax2 = axes\n",
"# get colormap for reds\n",
"reds_cmap = matplotlib.cm.get_cmap('Reds')\n",
"ax1.imshow(proba_data[:, :, 0],interpolation=\"nearest\", cmap=reds_cmap)\n",
"reds_cmap = matplotlib.cm.get_cmap(\"Reds\")\n",
"ax1.imshow(proba_data[:, :, 0], interpolation=\"nearest\", cmap=reds_cmap)\n",
"ax1.set_title(\"Permanent Water probabilities\", fontweight=\"bold\")\n",
"ax2.imshow(proba_data[:, :, 1],interpolation=\"nearest\", cmap=reds_cmap)\n",
"ax2.imshow(proba_data[:, :, 1], interpolation=\"nearest\", cmap=reds_cmap)\n",
"ax2.set_title(\"Intermittent Water probabilities\", fontweight=\"bold\")\n",
"plt.suptitle(\"Water Predictions for a lake in Urugway AOI\", y=0.95, fontweight=\"bold\", fontsize=15)"
]
Expand All @@ -243,8 +242,8 @@
"metadata": {},
"source": [
"## Myanmar AOI experiments\n",
"We also performed various experiments in Maynmar for fine tuning and evaluating the Classifier performance. </br>\n",
"Following code snippets show how can we access and visualize the data.\n",
"We also performed various experiments in Maynmar for fine-tuning and evaluating the Classifier performance. </br>\n",
"Following code snippets show how the data can be accessed and visualized.\n",
"\n",
"### Gap Filling Model\n",
"In addition to Tuning the model, we also experimented with a Gap Filling model, which can be used to fill gaps after classification using a CNN Image Segmentation model. </br>"
Expand Down Expand Up @@ -278,7 +277,7 @@
"# The data for three years are stored as separate bands with following band names\n",
"bands = [\"PRED\"]\n",
"\n",
"pred_collection_id = 'b31d5c95-ddb6-440f-b732-5827f5f64fb7'\n",
"pred_collection_id = \"b31d5c95-ddb6-440f-b732-5827f5f64fb7\"\n",
"byoc_collection = DataCollection.define_byoc(\n",
" pred_collection_id,\n",
" service_url=\"https://services.sentinel-hub.com\",\n",
Expand Down Expand Up @@ -314,7 +313,7 @@
],
"source": [
"bands = [\"PRED\"]\n",
"pred_collection_id = '742682e8-ab52-46b6-bfa8-6fc068cb60cf'\n",
"pred_collection_id = \"742682e8-ab52-46b6-bfa8-6fc068cb60cf\"\n",
"byoc_collection = DataCollection.define_byoc(\n",
" pred_collection_id,\n",
" service_url=\"https://services.sentinel-hub.com\",\n",
Expand Down Expand Up @@ -369,16 +368,16 @@
}
],
"source": [
"# Generate a plot for each year \n",
"# Generate a plot for each year\n",
"# we visualize the predictions using transform_to_rgb function\n",
"preds_data = eopatch.data[\"pred\"].astype(int).squeeze()\n",
"\n",
"fig, axes = plt.subplots(1, 2, figsize=(14, 7))\n",
"ax1, ax2= axes\n",
"ax1, ax2 = axes\n",
"ax1.set_title(\"With LGBM\", fontweight=\"bold\")\n",
"ax1.imshow(transform_to_rgb(lgbm_pred),interpolation=\"nearest\")\n",
"ax1.imshow(transform_to_rgb(lgbm_pred), interpolation=\"nearest\")\n",
"ax2.set_title(\"With CNN Gap Filling model\", fontweight=\"bold\")\n",
"ax2.imshow(transform_to_rgb(cnn_pred),interpolation=\"nearest\")\n",
"ax2.imshow(transform_to_rgb(cnn_pred), interpolation=\"nearest\")\n",
"plt.suptitle(\"Water Predictions for a lake in Myanmar AOI\", y=0.95, fontweight=\"bold\", fontsize=15)"
]
},
Expand Down

0 comments on commit 2600caa

Please sign in to comment.