Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📊 happiness: by age group #2867

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from
Draft

📊 happiness: by age group #2867

wants to merge 19 commits into from

Conversation

antea04
Copy link
Contributor

@antea04 antea04 commented Jun 21, 2024

(see also owid/owid-issues#1581)

Currently blocked as we are waiting on the full data set from the World happiness report. Description below only is relevant to the preliminary data (ca. 60 out of 200 countries)


Adding age group specific happiness data. Data comes from a .xls file I received from the World Happiness report team, it's the basis of Figure 2.1 here.

Tasks:

  • reach out to WHR to access data in usable format
  • import data into the etl
    • regional aggregates
  • draft charts with happiness data over ages

Questions for PR Review:

  • I made a chart for life satisfaction for each age group, because I did not find another way to easily compare happiness in an age group between countries. Is there a better alternative?
  • Since happiness data per age group is not available for all countries, regional aggregates sometimes include different subsets of countries, which can make this chart look weird (e.g. in "World" view). I added a footnote - is that enough?
  • Smaller questions/ things I wonder:
    • Is there an easier/ more elegant way for pivoting the data frame in the grapher step?
    • The source for the happiness data by age shows up as "Wellbeing Research Center (2024)", instead of "World Happiness Report (2024)" (like other graphs) because they are the producer of the 2024 report, which features this data. Is this discrepency confusing?
      I made that choice because previous reports have been produced by the "Sustainable Development Solutions Network", so for data which combines multiple World Happiness reports I just gave the source as "World Happiness Report (2012 - 2024)". I felt like this was the simplest solution while respecting the etl defaults where possible, but I can adjust it if its confusing.

@antea04 antea04 self-assigned this Jun 21, 2024
@owidbot
Copy link
Contributor

owidbot commented Jun 21, 2024

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-happiness-ages

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences
= Dataset garden/climate/2023-12-20/surface_temperature
  = Table surface_temperature
    ~ Column anomaly_above_0 (changed data)
        ~ Changed values: 14976 / 197535 (7.58%)
                                    country       time  anomaly_above_0 -  anomaly_above_0 +
                                    Morocco 1969-03-15           0.080668           0.080667
                                       Laos 2012-09-15           0.210032           0.210035
                                       Cuba 1940-11-15           0.107275           0.107277
          Heard Island and McDonald Islands 1997-10-15           0.095271           0.095267
                             American Samoa 1992-09-15           0.261990           0.261999
    ~ Column anomaly_below_0 (changed data)
        ~ Changed values: 21601 / 197535 (10.94%)
                               country       time  anomaly_below_0 -  anomaly_below_0 +
                                 Chile 2013-03-15          -0.164454          -0.164452
                            Luxembourg 1982-12-15          -0.207333          -0.207335
                               Myanmar 2005-11-15          -0.003508          -0.003511
          United States Virgin Islands 1942-07-15          -0.364595          -0.364576
                               Comoros 1984-11-15          -0.193058          -0.193054
    ~ Column temperature_2m (changed data)
        ~ Changed values: 2927 / 197535 (1.48%)
                                    country       time  temperature_2m -  temperature_2m +
                                    Belarus 1982-12-15          0.364824          0.364818
                                     Latvia 1989-11-15          0.613674          0.613666
                                    Andorra 2023-02-15         -0.239227         -0.239236
                                    Moldova 1952-01-15          0.520703          0.520697
          Heard Island and McDonald Islands 2009-05-15          0.567261          0.567243
    ~ Column temperature_anomaly (changed data)
        ~ Changed values: 36577 / 197535 (18.52%)
                     country       time  temperature_anomaly -  temperature_anomaly +
                   Palestine 2009-08-15               0.065123               0.065126
                     Somalia 1988-10-15              -0.147758              -0.147762
                  Tajikistan 1946-06-15               0.057419               0.057418
                    Thailand 1973-05-15              -0.653229              -0.653219
          Dominican Republic 2017-04-15               0.008354               0.008352
= Dataset garden/energy/latest/owid_energy
  = Table owid_energy
    ~ Dim country
-       - Removed values: 64 / 21748 (0.29%)
           year   country
           2023    Israel
           2023  Malaysia
           2023 OECD (EI)
           2023   Oceania
           2023     Yemen
    ~ Dim year
-       - Removed values: 64 / 21748 (0.29%)
            country  year
             Israel  2023
           Malaysia  2023
          OECD (EI)  2023
            Oceania  2023
              Yemen  2023
    ~ Column biofuel_cons_change_pct (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  biofuel_cons_change_pct
             Israel  2023                 9.905052
           Malaysia  2023                 7.547176
          OECD (EI)  2023                 9.952391
            Oceania  2023                -3.237998
              Yemen  2023                      NaN
        ~ Changed values: 817 / 21748 (3.76%)
                country  year  biofuel_cons_change_pct -  biofuel_cons_change_pct +
            Africa (EI)  2012                  18.688440                  20.213879
                   Asia  2016                  33.006786                  29.467726
                 Europe  2004                  43.541000                  32.972672
          Non-OECD (EI)  2004                  10.549747                  10.366488
                  World  2013                   8.340359                   8.679522
    ~ Column biofuel_cons_change_twh (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  biofuel_cons_change_twh
             Israel  2023                 0.073541
           Malaysia  2023                 0.781429
          OECD (EI)  2023                71.833069
            Oceania  2023                -0.045253
              Yemen  2023                      NaN
        ~ Changed values: 858 / 21748 (3.95%)
                       country  year  biofuel_cons_change_twh -  biofuel_cons_change_twh +
          Central America (EI)  2019                  -0.029848                   0.022644
           European Union (27)  1996                   1.143240                   1.140712
                      Thailand  2012                   2.656236                   2.628345
                        Turkey  2017                   0.075500                   0.074883
                         World  2013                  63.185730                  66.178894
    ~ Column biofuel_cons_per_capita (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  biofuel_cons_per_capita
             Israel  2023                88.941887
           Malaysia  2023               324.565491
          OECD (EI)  2023                      NaN
            Oceania  2023                29.671305
              Yemen  2023                      NaN
        ~ Changed values: 664 / 21748 (3.05%)
          country  year  biofuel_cons_per_capita -  biofuel_cons_per_capita +
           Africa  2018                   0.783870                   0.762708
          Austria  2018                 584.883789                 658.023743
            Japan  2015                  29.640976                  31.051037
           Norway  2017                1099.903564                1052.604614
            World  2000                  21.550003                  21.616526
    ~ Column biofuel_consumption (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  biofuel_consumption
             Israel  2023             0.815999
           Malaysia  2023            11.135365
          OECD (EI)  2023           793.600342
            Oceania  2023             1.352302
              Yemen  2023                  NaN
        ~ Changed values: 832 / 21748 (3.83%)
                        country  year  biofuel_consumption -  biofuel_consumption +
                         Africa  2012               0.599225               0.606699
                    Europe (EI)  2004              19.695684              20.481251
          High-income countries  2013             542.591797             540.465454
                      Lithuania  2012               0.703167               0.702778
               Middle East (EI)  2022               0.742458               0.631898
    ~ Column biofuel_share_elec (changed metadata)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  biofuel_share_elec
             Israel  2023                 NaN
           Malaysia  2023                 NaN
          OECD (EI)  2023                 NaN
            Oceania  2023                 NaN
              Yemen  2023                 NaN
    ~ Column biofuel_share_energy (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  biofuel_share_energy
             Israel  2023              0.263686
           Malaysia  2023              0.833916
          OECD (EI)  2023              1.242715
            Oceania  2023              0.070742
              Yemen  2023                   NaN
        ~ Changed values: 1408 / 21748 (6.47%)
                                country  year  biofuel_share_energy -  biofuel_share_energy +
                            Europe (EI)  1998                0.017912                0.018555
                              Lithuania  2022                2.137053                2.080729
          Lower-middle-income countries  2021                0.131277                0.562367
                         United Kingdom  1999                0.000000                     NaN
          Upper-middle-income countries  2002                0.238657                0.247865
    ~ Column carbon_intensity_elec (changed metadata)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  carbon_intensity_elec
             Israel  2023                    NaN
           Malaysia  2023                    NaN
          OECD (EI)  2023                    NaN
            Oceania  2023                    NaN
              Yemen  2023                    NaN
    ~ Column coal_cons_change_pct (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_cons_change_pct
             Israel  2023            -13.529128
           Malaysia  2023              3.761744
          OECD (EI)  2023            -12.918674
            Oceania  2023             -2.569598
              Yemen  2023                   NaN
        ~ Changed values: 1005 / 21748 (4.62%)
                      country  year  coal_cons_change_pct -  coal_cons_change_pct +
          European Union (27)  1971               -3.007752               -2.851605
                       France  1968               -4.126936               -3.836381
           Middle Africa (EI)  2015              319.450897              -14.243496
                North America  2016               -7.888305               -8.187544
                    OECD (EI)  1989                0.722003                0.727367
    ~ Column coal_cons_change_twh (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_cons_change_twh
             Israel  2023             -6.010689
           Malaysia  2023              9.895203
          OECD (EI)  2023          -1036.388184
            Oceania  2023            -11.351257
              Yemen  2023                   NaN
        ~ Changed values: 1000 / 21748 (4.60%)
                                country  year  coal_cons_change_twh -  coal_cons_change_twh +
                    European Union (27)  1972              -71.710938              -64.994629
          Lower-middle-income countries  2016              202.411133              214.453613
                     North America (EI)  2018             -222.987305             -222.223633
          Upper-middle-income countries  1966              223.687012              223.349121
                                  World  2017              456.417969              414.726562
    ~ Column coal_cons_per_capita (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_cons_per_capita
             Israel  2023           4187.365723
           Malaysia  2023           7955.551758
          OECD (EI)  2023                   NaN
            Oceania  2023           9443.565430
              Yemen  2023                   NaN
        ~ Changed values: 705 / 21748 (3.24%)
                                country  year  coal_cons_per_capita -  coal_cons_per_capita +
                                Hungary  2023              994.128479                     NaN
          Lower-middle-income countries  2019             1936.910400             2177.806152
                             Luxembourg  2022              737.595337              794.275940
                                 Sweden  2020             1794.966797             1825.810303
                                  World  2020             5394.329590             5386.257324
    ~ Column coal_consumption (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_consumption
             Israel  2023         38.417072
           Malaysia  2023        272.943268
          OECD (EI)  2023       6986.013184
            Oceania  2023        430.400818
              Yemen  2023               NaN
        ~ Changed values: 974 / 21748 (4.48%)
                                country  year  coal_consumption -  coal_consumption +
                                Estonia  2004           38.469120           38.464703
                    European Union (27)  2001         3285.362305         3285.511475
          Lower-middle-income countries  1996         2279.926270         2355.115234
                            Switzerland  2022            1.069444            1.027778
          Upper-middle-income countries  1969         5710.318359         5692.264160
    ~ Column coal_elec_per_capita (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_elec_per_capita
             Israel  2023                   NaN
           Malaysia  2023           2372.542236
          OECD (EI)  2023                   NaN
            Oceania  2023           2756.046631
              Yemen  2023                   NaN
        ~ Changed values: 63 / 21748 (0.29%)
                                country  year  coal_elec_per_capita -  coal_elec_per_capita +
          Lower-middle-income countries  1999              174.644577              185.526779
          Upper-middle-income countries  1987              393.385468              390.101959
          Upper-middle-income countries  1993              500.496490              495.297974
          Upper-middle-income countries  1995              575.859558              569.011719
                                  World  1999              927.965515              927.996521
    ~ Column coal_electricity (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_electricity
             Israel  2023               NaN
           Malaysia  2023         81.398430
          OECD (EI)  2023       1904.473755
            Oceania  2023        125.609848
              Yemen  2023               NaN
        ~ Changed values: 191 / 21748 (0.88%)
                                country  year  coal_electricity -  coal_electricity +
                            Europe (EI)  2017          889.590515          889.500244
          Lower-middle-income countries  1992          336.177002          347.538696
                          Non-OECD (EI)  1992         1482.467896         1482.574219
                          Non-OECD (EI)  2013         6041.945801         6041.482910
                                 Taiwan  1993           44.820042           44.944958
    ~ Column coal_prod_change_pct (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_prod_change_pct
             Israel  2023                   NaN
           Malaysia  2023                   NaN
          OECD (EI)  2023             -2.332038
            Oceania  2023              3.787732
              Yemen  2023                   NaN
        ~ Changed values: 388 / 21748 (1.78%)
                                country  year  coal_prod_change_pct -  coal_prod_change_pct +
                               CIS (EI)  2011                4.573560                4.561710
                            Europe (EI)  2009               -5.542719               -5.583382
          Lower-middle-income countries  1993               -3.117007               -1.496828
                                Romania  2023              -18.758846                     NaN
                          South America  2017               -1.532805               -1.710314
    ~ Column coal_prod_change_twh (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_prod_change_twh
             Israel  2023                   NaN
           Malaysia  2023                   NaN
          OECD (EI)  2023           -200.340820
            Oceania  2023            118.918945
              Yemen  2023                   NaN
        ~ Changed values: 389 / 21748 (1.79%)
                                country  year  coal_prod_change_twh -  coal_prod_change_twh +
                               CIS (EI)  2010              154.398438              154.740234
                            Europe (EI)  2008              -69.319092              -71.225342
          Lower-middle-income countries  1992               21.680420               80.069092
                                Romania  2023               -6.056734                     NaN
                          South America  2017              -12.097778              -13.489075
    ~ Column coal_prod_per_capita (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_prod_per_capita
             Israel  2023                   NaN
           Malaysia  2023                   NaN
          OECD (EI)  2023                   NaN
            Oceania  2023          71495.921875
              Yemen  2023                   NaN
        ~ Changed values: 271 / 21748 (1.25%)
                                country  year  coal_prod_per_capita -  coal_prod_per_capita +
                                 Europe  2008             5990.598145             5970.875488
          Lower-middle-income countries  2007             1159.381836             1689.274536
          Lower-middle-income countries  2014             1261.394531             2171.547852
          Upper-middle-income countries  2008            10340.537109             9633.823242
                                  World  2007             5717.166504             5715.296875
    ~ Column coal_production (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_production
             Israel  2023              NaN
           Malaysia  2023              NaN
          OECD (EI)  2023      8390.475586
            Oceania  2023      3258.504883
              Yemen  2023              NaN
        ~ Changed values: 380 / 21748 (1.75%)
                                country  year  coal_production -  coal_production +
                                Hungary  2023           7.485831                NaN
                          Non-OECD (EI)  2003       18549.837891       18538.021484
                          South America  2012         773.729553         773.193604
                          South America  2017         777.159729         775.202026
          Upper-middle-income countries  1983        9459.282227        9454.828125
    ~ Column coal_share_elec (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_share_elec
             Israel  2023              NaN
           Malaysia  2023        43.344757
          OECD (EI)  2023        17.036781
            Oceania  2023        39.534286
              Yemen  2023              NaN
        ~ Changed values: 249 / 21748 (1.14%)
                country  year  coal_share_elec -  coal_share_elec +
               CIS (EI)  2018          17.874310          18.089489
          Non-OECD (EI)  2004          42.290844          42.291775
          Non-OECD (EI)  2009          46.709461          46.712364
              OECD (EI)  2004          37.419418          37.420006
                  World  1999          37.723404          37.725464
    ~ Column coal_share_energy (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  coal_share_energy
             Israel  2023          12.414269
           Malaysia  2023          20.440435
          OECD (EI)  2023          10.939544
            Oceania  2023          22.515324
              Yemen  2023                NaN
        ~ Changed values: 2209 / 21748 (10.16%)
                        country  year  coal_share_energy -  coal_share_energy +
                          China  2008            71.927696            71.931747
            European Union (27)  1982            30.939507            30.894846
          High-income countries  2005            19.513409            19.529028
               Middle East (EI)  2017             1.027145             1.020232
                         Poland  2018            47.392487            47.487602
    ~ Column electricity_generation (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  electricity_generation
             Israel  2023               77.176811
           Malaysia  2023              187.793015
          OECD (EI)  2023            11178.600586
            Oceania  2023              317.723816
              Yemen  2023                     NaN
        ~ Changed values: 300 / 21748 (1.38%)
                                 country  year  electricity_generation -  electricity_generation +
                             Africa (EI)  2006                586.256897                586.149536
                           Non-OECD (EI)  2006               8335.224609               8334.709961
                      North America (EI)  2022               5536.817383               5547.990723
                               OECD (EI)  2019              11235.717773              11212.098633
          South and Central America (EI)  2004                902.045410                901.977600
    ~ Column electricity_share_energy (changed metadata, changed data)
-       -     citation_full: Energy Institute - Statistical Review of World Energy (2024).
        ?                                                                              ^
+       +     citation_full: Energy Institute - Statistical Review of World Energy (2023).
        ?                                                                              ^
-       -     attribution: Energy Institute - Statistical Review of World Energy (2024)
        ?                                                                            ^
+       +     attribution: Energy Institute - Statistical Review of World Energy (2023)
        ?                                                                            ^
-       -     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/merged_panel.csv
        ?                                                                              ^^^^  ^^
+       +     url_download: https://www.energyinst.org/__data/assets/file/0007/1055761/Consolidated-Dataset-Panel-format-CSV.csv
        ?                                                                              ^^^^^^^^^^  ^^^^^^^^^^    +++++++++++
-       -     date_accessed: '2024-06-20'
        ?                        ^ ^^  -
+       +     date_accessed: '2023-12-12'
        ?                        ^ ^^ +
-       -     date_published: '2024-06-20'
        ?                         ^     ^
+       +     date_published: '2023-06-26'
        ?                         ^     ^
-       -       name: © Energy Institute 2024
        ?                                   ^
+       +       name: © Energy Institute 2023
        ?                                   ^
-       -       url: https://www.energyinst.org/__data/assets/pdf_file/0006/1542714/EI_Stats_Review_2024_single_pages.pdf
+       +       url: |-
+       +         https://web.archive.org/web/20230626200231/https://www.energyinst.org/__data/assets/pdf_file/0004/1055542/EI_Stat_Review_PDF_single-2.pdf

-       - Removed values: 64 / 21748 (0.29%)
            country  year  electricity_share_energy
             Israel  2023                 26.030266
           Malaysia  2023                 14.704694
          OECD (EI)  2023                 20.300945
            Oceania  2023                 18.738131
              Yemen  2023                       NaN
        ~ Changed values: 2069 / 21748 (9.51%)
                      country  year  electricity_share_energy -  electricity_share_energy +
                      Belarus  2021                   14.003945                   13.978733
                        Japan  1990                   19.028790                   18.888668
                 South Africa  2015                   18.059338                   18.073540
                 Turkmenistan  2021                    5.087101                    5.015595
          Western Africa (EI)  2002  

...diff too long, truncated...

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2024-11-20 10:38:18 UTC
Execution time: 4.25 seconds

@antea04 antea04 requested a review from spoonerf June 26, 2024 14:13
@spoonerf
Copy link
Contributor

Hey @antea04 - nice work!

I've made some suggestions, and I think it would be good to clarify whether they are able to share all the country's data before merging this.

To answer your questions:

I made a chart for life satisfaction for each age group, because I did not find another way to easily compare happiness in an age group between countries. Is there a better alternative?

I think perhaps a good way to present this would be to change this chart to a discrete bar chart, this way the differences between the age-groups are more obvious and we often opt for this when the time-series is fairly flat over time, like it is here. (I would also change the labels so they are a bit shorter e.g. '<30 years', let me know if you aren't sure how to do this.)

The scatterplot you've made is also a really nice way of showing this too. I think just having these two charts is plenty, and I would opt not to create individual maps of the age groups.


Since happiness data per age group is not available for all countries, regional aggregates sometimes include different subsets of countries, which can make this chart look weird (e.g. in "World" view). I added a footnote - is that enough?

Do you mean weird in that for 'World' the value for All ages is lower than each of the individual age-groups? I think perhaps you should set a threshold of around 70% coverage for regions to be calculated. I've suggested a way of doing this in the code review. Hopefully, that will help, if not we should reconsider if calculating regions for the age-group data is helpful. If they are able to share more of the data, this shouldn't be a problem.


Is there an easier/ more elegant way for pivoting the data frame in the grapher step?

I've suggested a way in the code review - you can use age_group as an index, which should mean the pivoting is done automatically. You can also use Jinja templates to give specific metadata to each age-group, when it is included in the index.


The source for the happiness data by age shows up as "Wellbeing Research Center (2024)", instead of "World Happiness Report (2024)" (like other graphs) because they are the producer of the 2024 report, which features this data. Is this discrepency confusing?
I made that choice because previous reports have been produced by the "Sustainable Development Solutions Network", so for data which combines multiple World Happiness reports I just gave the source as "World Happiness Report (2012 - 2024)". I felt like this was the simplest solution while respecting the etl defaults where possible, but I can adjust it if its confusing.

I think what you suggest is a good solution!

etl/steps/data/garden/happiness/2024-06-09/happiness.py Outdated Show resolved Hide resolved
etl/steps/data/garden/happiness/2024-06-09/happiness.py Outdated Show resolved Hide resolved
etl/steps/data/grapher/happiness/2024-06-09/happiness.py Outdated Show resolved Hide resolved
@@ -13,7 +13,7 @@ dataset:
tables:
happiness:
variables:
cantril_ladder_score:
happiness_score:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use some Jinja templating here to add metadata to specific age-groups, if you want. Using this you can add index values into the metadata. Here is a slightly complicated example of how you can do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would help with the question you had about pivoting in the grapher step.

@lucasrodes lucasrodes changed the title 📊 DRAFT: Happiness by age group 📊 happiness: by age group Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants