Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EE charting functions #1855

Merged
merged 40 commits into from
Aug 17, 2024
Merged

Add EE charting functions #1855

merged 40 commits into from
Aug 17, 2024

Conversation

giswqs
Copy link
Member

@giswqs giswqs commented Dec 19, 2023

This PR adds more charting functions to the chart module.

Reference: https://developers.google.com/earth-engine/guides/charts_image

chart.image_byRegion

ecoregions = ee.FeatureCollection('projects/google/charts_feature_example')
normClim = ee.ImageCollection('OREGONSTATE/PRISM/Norm81m').toBands().select('[0-9][0-9]_tmean')
labels = list(calendar.month_abbr)[1:] # a list of month labels, e.g. ['Jan', 'Feb', ...]

chart.image_byRegion(
    image=normClim, 
    regions=ecoregions, 
    reducer="mean", 
    scale=500, 
    xProperty='label', 
    xlabel='Ecoregion',
    ylabel='Temperature',
    labels=labels,
    # colors=['red', 'green', 'blue'],
    )


@giswqs giswqs requested a review from jdbcode December 19, 2023 21:14
Copy link

github-actions bot commented Dec 19, 2023

@github-actions github-actions bot temporarily deployed to pull request December 19, 2023 21:21 Inactive
@giswqs
Copy link
Member Author

giswqs commented Dec 19, 2023

chart.image_byClass

Reference: https://developers.google.com/earth-engine/guides/charts_image#uichartimagebyclass

bqplot example with line smoothing

import numpy as np
import bqplot.pyplot as plt

fig = plt.figure(title="Random Walks")
x = np.arange(100)
# three random walks
y = np.random.randn(3, 100).cumsum(axis=1)

lines = plt.plot(
    x, y,
    labels=["Line1", "Line2", "Line 3"],
    display_legend=True
)

# Plot the lines with interpolation
line1 = plt.plot(x, y[0], 'b-', label='Line 1', interpolation='basis')
line2 = plt.plot(x, y[1], 'r-', label='Line 2', interpolation='step-after')
line3 = plt.plot(x, y[2], 'g-', label='Line 3', interpolation='cardinal')

plt.show()

@jdbcode
Copy link
Collaborator

jdbcode commented Dec 20, 2023

Hi @giswqs, just a quick note to say I see the review request and will follow up shortly.

@giswqs
Copy link
Member Author

giswqs commented Dec 20, 2023

No rush. I will be adding more commit over the next few days.

@github-actions github-actions bot temporarily deployed to pull request December 22, 2023 19:59 Inactive
@github-actions github-actions bot temporarily deployed to pull request December 22, 2023 21:25 Inactive
@github-actions github-actions bot temporarily deployed to pull request December 25, 2023 04:38 Inactive
geemap/chart.py Outdated Show resolved Hide resolved
@jdbcode
Copy link
Collaborator

jdbcode commented Jan 3, 2024

It looks like you're maybe still working on chart.image_byClass?
I saw the new Image_byClass class but the image_byClass function is unchanged from todo. Do you want to add it to this PR, or would you like to merge the current changes?

@giswqs
Copy link
Member Author

giswqs commented Jan 3, 2024

@jdbcode Thanks fort testing it. I will be adding more classes and functions. Don't merge this PR yet.

@jdbcode
Copy link
Collaborator

jdbcode commented Jan 3, 2024

Sounds good

@github-actions github-actions bot temporarily deployed to pull request January 3, 2024 21:55 Inactive
@smith-kyle
Copy link

I'm a little late to the party, but I noticed you all aren't using a notebook review tool and wanted to invite you to review this pull request with GitNotebooks: https://gitnotebooks.com/gee-community/geemap/pull/1855

It lets you do things like comment on rendered markdown and code cells, so might be an easy win for your PR reviews.

@github-actions github-actions bot temporarily deployed to pull request February 15, 2024 23:09 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 25, 2024 12:29 Inactive
@giswqs giswqs changed the title Add image chart functions Add EE charting functions May 25, 2024
@github-actions github-actions bot temporarily deployed to pull request May 25, 2024 13:58 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 25, 2024 23:55 Inactive
@giswqs
Copy link
Member Author

giswqs commented May 26, 2024

chart.image_byClass

Reference: https://developers.google.com/earth-engine/guides/charts_image#uichartimagebyclass

import ee
import geemap
from geemap.chart import BaseChart
import geemap.chart as chart
from bqplot import Figure, Lines, Axis, LinearScale, ColorScale
from IPython.display import display

geemap.ee_initialize()

ecoregions = ee.FeatureCollection('projects/google/charts_feature_example')

image = ee.ImageCollection('MODIS/061/MOD09A1') \
    .filter(ee.Filter.date('2018-06-01', '2018-09-01')) \
    .select('sur_refl_b0[0-7]') \
    .mean() \
    .select([2, 3, 0, 1, 4, 5, 6])

wavelengths = [469, 555, 655, 858, 1240, 1640, 2130]

fc = geemap.zonal_stats(
    image, ecoregions, stat_type="MEAN", scale=500, verbose=False, return_fc=True
)
bands = image.bandNames().getInfo()
df = geemap.ee_to_df(fc)[bands + ["label"]]

# Define the scales
x_sc = LinearScale()
y_sc = LinearScale()
color_sc = ColorScale()

lines = []
colors =  ['#f0af07', '#0f8755', '#76b349'] 

x_values = list(range(len(df.columns) - 1))
x_values = [469, 555, 655, 858, 1240, 1640, 2130]
band_names = ['B03', 'B04', 'B01', 'B02', 'B05', 'B06', 'B07']

for i, label in enumerate(df['label']):
    line = Lines(
        x=x_values, 
        y=df.iloc[i, :-1].values.astype(float),  # Ensure y-values are float
        scales={'x': x_sc, 'y': y_sc},
        colors=[colors[i]],
        labels=[label],
        display_legend=True,
        # interpolation='basis',
        stroke_width=3
    )
    lines.append(line)

x_ax = Axis(scale=x_sc, label='Wavelength (nm)', tick_values=x_values)
y_ax = Axis(scale=y_sc, orientation='vertical', label='Reflectance')

fig = Figure(marks=lines, axes=[x_ax, y_ax], title='Ecoregion Spectral Reflectance', legend_location='top-right')

display(fig)

no interpolation
image

basis interpolation
image

@github-actions github-actions bot temporarily deployed to pull request August 10, 2024 03:06 Inactive
@giswqs
Copy link
Member Author

giswqs commented Aug 10, 2024

Add chart.array_values function.
Reference: https://developers.google.com/earth-engine/guides/charts_array

# Import the example feature collection and subset the forest feature.
forest = ee.FeatureCollection("projects/google/charts_feature_example").filter(
    ee.Filter.eq("label", "Forest")
)

# Define a MODIS surface reflectance composite.
modisSr = (
    ee.ImageCollection("MODIS/061/MOD09A1")
    .filter(ee.Filter.date("2018-06-01", "2018-09-01"))
    .select("sur_refl_b0[0-7]")
    .mean()
)

# Reduce MODIS reflectance bands by forest region; get a dictionary with
# band names as keys, pixel values as lists.
pixel_vals = modisSr.reduceRegion(
    **{"reducer": ee.Reducer.toList(), "geometry": forest.geometry(), "scale": 2000}
)

# Convert NIR and SWIR value lists to an array to be plotted along the y-axis.
y_values = pixel_vals.toArray(["sur_refl_b02", "sur_refl_b06"])


# Get the red band value list; to be plotted along the x-axis.
x_values = ee.List(pixel_vals.get("sur_refl_b01"))

title = "Relationship Among Spectral Bands for Forest Pixels"
colors = ["rgba(29,107,153,0.4)", "rgba(207,81,62,0.4)"]

fig = chart.array_values(
    y_values,
    axis=1,
    x_labels=x_values,
    series_names=["NIR", "SWIR"],
    chart_type="ScatterChart",
    colors=colors,
    title=title,
    x_label="Red reflectance (x1e4)",
    y_label="NIR & SWIR reflectance (x1e4)",
    default_size=15,
    xlim=(0, 800),
)
fig

image

@github-actions github-actions bot temporarily deployed to pull request August 10, 2024 14:20 Inactive
@giswqs
Copy link
Member Author

giswqs commented Aug 10, 2024

Add AreaChart type
Reference: https://developers.google.com/earth-engine/guides/charts_array#eelist_transect_line_plot

# Define a line across the Olympic Peninsula, USA.
transect = ee.Geometry.LineString([[-122.8, 47.8], [-124.5, 47.8]])

# Define a pixel coordinate image.
lat_lon_img = ee.Image.pixelLonLat()

# Import a digital surface model and add latitude and longitude bands.
elev_img = ee.Image("USGS/SRTMGL1_003").select("elevation").addBands(lat_lon_img)

# Reduce elevation and coordinate bands by transect line; get a dictionary with
# band names as keys, pixel values as lists.
elev_transect = elev_img.reduceRegion(
    reducer=ee.Reducer.toList(),
    geometry=transect,
    scale=1000,
)

# Get longitude and elevation value lists from the reduction dictionary.
lon = ee.List(elev_transect.get("longitude"))
elev = ee.List(elev_transect.get("elevation"))

# Sort the longitude and elevation values by ascending longitude.
lon_sort = lon.sort(lon)
elev_sort = elev.sort(lon)

fig = chart.array_values(
    elev_sort,
    x_labels=lon_sort,
    series_names=["Elevation"],
    chart_type="AreaChart",
    colors=["#1d6b99"],
    title="Elevation Profile Across Longitude",
    x_label="Longitude",
    y_label="Elevation (m)",
    stroke_width=5,
    fill="bottom",
    fill_opacities=[0.4],
    ylim=(0, 2500),
)
fig

image

@github-actions github-actions bot temporarily deployed to pull request August 10, 2024 14:58 Inactive
@giswqs
Copy link
Member Author

giswqs commented Aug 10, 2024

Add metadata scatter plot
Reference: https://developers.google.com/earth-engine/guides/charts_array#eelist_metadata_scatter_plot

# Import a Landsat 8 collection and filter to a single path/row.
col = ee.ImageCollection('LANDSAT/LC08/C02/T1_L2').filter(
    ee.Filter.expression('WRS_PATH ==  45 && WRS_ROW == 30')
)

# Reduce image properties to a series of lists; one for each selected property.
propVals = col.reduceColumns(
    reducer=ee.Reducer.toList().repeat(2),
    selectors=['CLOUD_COVER', 'GEOMETRIC_RMSE_MODEL'],
).get('list')

# Get selected image property value lists; to be plotted along x and y axes.
x = ee.List(ee.List(propVals).get(0))
y = ee.List(ee.List(propVals).get(1))
colors = [geemap.hex_to_rgba('#96356f', 0.4)]

fig = chart.array_values(
    y,
    x_labels=x,
    series_names=["RMSE"],
    chart_type="ScatterChart",
    colors=colors,
    title="Landsat 8 Image Collection Metadata (045030)",
    x_label="Cloud cover (%)",
    y_label="Geometric RMSE (m)",
    default_size=15,
)
fig

image

@giswqs
Copy link
Member Author

giswqs commented Aug 10, 2024

Add mapped function scatter & line plot
Reference: https://developers.google.com/earth-engine/guides/charts_array#eelist_mapped_function_scatter_line_plot

import math

start = -2 * math.pi
end = 2 * math.pi
points = ee.List.sequence(start, end, None, 50)

def sin_func(val):
    return ee.Number(val).sin()

values = points.map(sin_func)

fig = chart.array_values(
    values,
    points,
    chart_type="LineChart",
    colors=['#39a8a7'],
    title="Sine Function",
    x_label="radians",
    y_label="sin(x)",
     marker='circle',
)
fig

image

@github-actions github-actions bot temporarily deployed to pull request August 10, 2024 15:20 Inactive
@github-actions github-actions bot temporarily deployed to pull request August 10, 2024 22:48 Inactive
@giswqs
Copy link
Member Author

giswqs commented Aug 10, 2024

Add IntervalChart
Reference: https://developers.google.com/earth-engine/guides/charts_datatable#interval_chart

# Define a point to extract an NDVI time series for.
geometry = ee.Geometry.Point([-121.679, 36.479])

# Define a band of interest (NDVI), import the MODIS vegetation index dataset,
# and select the band.
band = "NDVI"
ndvi_col = ee.ImageCollection("MODIS/061/MOD13Q1").select(band)

# Map over the collection to add a day of year (doy) property to each image.


def set_doy(img):
    doy = ee.Date(img.get("system:time_start")).getRelative("day", "year")
    # Add 8 to day of year number so that the doy label represents the middle of
    # the 16-day MODIS NDVI composite.
    return img.set("doy", ee.Number(doy).add(8))


ndvi_col = ndvi_col.map(set_doy)

# Join all coincident day of year observations into a set of image collections.
distinct_doy = ndvi_col.filterDate("2013-01-01", "2014-01-01")
filter = ee.Filter.equals(**{"leftField": "doy", "rightField": "doy"})
join = ee.Join.saveAll("doy_matches")
join_col = ee.ImageCollection(join.apply(distinct_doy, ndvi_col, filter))

# Calculate the absolute range, interquartile range, and median for the set
# of images composing each coincident doy observation group. The result is
# an image collection with an image representative per unique doy observation
# with bands that describe the 0, 25, 50, 75, 100 percentiles for the set of
# coincident doy images.


def cal_percentiles(img):
    doyCol = ee.ImageCollection.fromImages(img.get("doy_matches"))

    return doyCol.reduce(
        ee.Reducer.percentile([0, 25, 50, 75, 100], ["p0", "p25", "p50", "p75", "p100"])
    ).set({"doy": img.get("doy")})


comp = ee.ImageCollection(join_col.map(cal_percentiles))

# Extract the inter-annual NDVI doy percentile statistics for the
# point of interest per unique doy representative. The result is
# is a feature collection where each feature is a doy representative that
# contains a property (row) describing the respective inter-annual NDVI
# variance, formatted as a list of values.


def order_percentiles(img):
    stats = ee.Dictionary(
        img.reduceRegion(
            **{"reducer": ee.Reducer.first(), "geometry": geometry, "scale": 250}
        )
    )

    # Order the percentile reduction elements according to how you want columns
    # in the DataTable arranged (x-axis values need to be first).
    row = ee.List(
        [
            img.get("doy"),
            stats.get(band + "_p50"),
            stats.get(band + "_p0"),
            stats.get(band + "_p25"),
            stats.get(band + "_p75"),
            stats.get(band + "_p100"),
        ]
    )

    # Return the row as a property of an ee.Feature.
    return ee.Feature(None, {"row": row})


reduction_table = comp.map(order_percentiles)

# Aggregate the 'row' properties to make a server-side 2-D array (DataTable).
data_table_server = reduction_table.aggregate_array("row")

# Define column names and properties for the DataTable. The order should
# correspond to the order in the construction of the 'row' property above.
column_header = ee.List([["DOY", "median", "p0", "p25", "p75", "p100"]])

# Concatenate the column header to the table.
data_table_server = column_header.cat(data_table_server)

df = chart.DataTable(data_table_server)

fig = chart.Chart(
    df,
    chart_type="IntervalChart",
    x_cols="DOY",
    y_cols=["p0", "p25", "median", "p75", "p100"],
    title="Annual NDVI Time Series with Inter-Annual Variance",
    x_label="Day of Year",
    y_label="Vegetation index (x1e4)",
    stroke_width=1,
    fill="between",
    fill_colors=["#b6d1c6", "#83b191", "#83b191", "#b6d1c6"],
    fill_opacities=[0.6] * 4,
    labels=["p0", "p25", "median", "p75", "p100"],
    display_legend=True,
    legend_location="top-right",
    ylim=(0, 10000),
)
fig

image

@github-actions github-actions bot temporarily deployed to pull request August 10, 2024 23:05 Inactive
@github-actions github-actions bot temporarily deployed to pull request August 11, 2024 03:14 Inactive
@giswqs
Copy link
Member Author

giswqs commented Aug 11, 2024

A collection of 21 charts.

Copy link
Collaborator

@jdbcode jdbcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome!! Great work, Qiusheng! I'm thinking about how to add them to the charting pages in the EE Docs. I think probably simply linking to the notebooks in this PR from the top of each respective DevSite page is the best for now (as opposed to adding the examples in each individual code block in the pages). Would that be okay with you - a link to your notebooks - the link would look like the buttons at the top of this page.

@jdbcode
Copy link
Collaborator

jdbcode commented Aug 13, 2024

Perfect, thanks!

@github-actions github-actions bot temporarily deployed to pull request August 14, 2024 15:14 Inactive
@giswqs giswqs merged commit bfb63d2 into master Aug 17, 2024
14 checks passed
@giswqs giswqs deleted the chart branch August 17, 2024 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Charts] Add a chart module for creating interactive charts
3 participants