Change reference cost year to 2020 #121

lisazeyen · 2024-02-05T11:54:12Z

Closes # (if applicable).

Changes proposed in this Pull Request

Change the reference cost year to 2020, closes issue #118

a new column currency_year is added which represents the currency year of the original input data
all input data is adjusted for inflation
for input data where currency year was not clear, year of publication is assumed
for old pypsa cost assumptions currency year 2015 is assumed

Checklist

add annual inflation from eurostat
@JulianGeis one would need to go through all the techs listed here and double check the reference cost year. All the techs with cost year 2020 should be added to the list here
add reference year for vehicle costs
Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
Data source for new technologies is cleary stated.
Newly introduced dependencies are added to environment.yaml (if applicable).
A note for the release notes doc/release_notes.rst of the upcoming release is included.
I consent to the release of this PR's code under the GPLv3 license.

fneum · 2024-02-05T12:35:52Z

@aleks-g and @koen-vg have used real inflation data instead of snakemake.config['rate_inflation'] from https://ec.europa.eu/eurostat/databrowser/view/teicp000/default/table?lang=en in https://github.com/koen-vg/enabling-agency / https://arxiv.org/abs/2312.11264.

While costs in the above repository are given in 2015 EUR, we have converted all cost data to 2023 EUR for the purposes of this study, using inflation data for the Euro area up to October 2023 [13], amounting to a 24.5% increase compared to technology-data.

Don't know though if that was done afterwards or directly in a fork of technology-data.

aleks-g · 2024-02-05T12:53:23Z

@aleks-g and @koen-vg have used real inflation data instead of snakemake.config['rate_inflation'] from https://ec.europa.eu/eurostat/databrowser/view/teicp000/default/table?lang=en in https://github.com/koen-vg/enabling-agency / https://arxiv.org/abs/2312.11264.

While costs in the above repository are given in 2015 EUR, we have converted all cost data to 2023 EUR for the purposes of this study, using inflation data for the Euro area up to October 2023 [13], amounting to a 24.5% increase compared to technology-data.

Don't know though if that was done afterwards or directly in a fork of technology-data.

Hi Fabian, we've only included inflation since 2015 in the analysis/post-processing afterwards, without adapting anything in technology-data itself. :)

lisazeyen · 2024-02-05T16:23:42Z

@aleks-g and @koen-vg have used real inflation data instead of snakemake.config['rate_inflation'] from https://ec.europa.eu/eurostat/databrowser/view/teicp000/default/table?lang=en in https://github.com/koen-vg/enabling-agency / https://arxiv.org/abs/2312.11264.

While costs in the above repository are given in 2015 EUR, we have converted all cost data to 2023 EUR for the purposes of this study, using inflation data for the Euro area up to October 2023 [13], amounting to a 24.5% increase compared to technology-data.

Don't know though if that was done afterwards or directly in a fork of technology-data.

I guess it make sense to take the actual inflation. I downloaded the annual inflation from eurostat and would add this instead of assuming 2% inflation for every year

lisazeyen · 2024-02-16T09:16:08Z

We haven't checked all technologies yet in DEA, but I think we can already pull it in and open another issue that all single DEA technologies should be double checked for the correct currency year.

koen-vg · 2024-02-16T09:25:27Z

I have to say I'm a little bit confused by the Eurostat dataset browser sometimes because it sometimes includes and sometimes doesn't include 2023. But here it seems possible to get a dataset in the same format with 2023 included? https://ec.europa.eu/eurostat/databrowser/view/prc_hicp_aind__custom_9900786/default/table?lang=en

I think it would be nice to have as recent as possible data available, and there are now inflation values for 2023 (namely, 6.4%).

Apart from that, thanks a bunch for this work!

lisazeyen · 2024-02-16T12:53:58Z

I have to say I'm a little bit confused by the Eurostat dataset browser sometimes because it sometimes includes and sometimes doesn't include 2023. But here it seems possible to get a dataset in the same format with 2023 included? https://ec.europa.eu/eurostat/databrowser/view/prc_hicp_aind__custom_9900786/default/table?lang=en

I think it would be nice to have as recent as possible data available, and there are now inflation values for 2023 (namely, 6.4%).

Apart from that, thanks a bunch for this work!

Good idea to include 2023, unfortunately the data you are linking to is only dating back to 2015 and it would be nice to have the earlier years in there as well. I have added 2023 now within the function when preparing the data (not the nicest way, but at least 2023 is included now as well)...

fneum

Looks mostly good. Some column headers were removed and there were some local file paths added, and we should link the source for the inflation data.

Did you do a few spot checks whether the adjustments made sense in outputs? If yes, I am happy for this to be merged.

Regarding the range of inflation data available, if the other dataset (dating back pre-2015) is not updated, we can think about merging the two sources.

docs/release_notes.rst

environment.yaml

fneum · 2024-02-16T15:15:45Z

Let's do the following spot checks:

H2 (g) pipeline
HVDC overhead
nuclear
methanolisation

lisazeyen · 2024-02-16T16:42:29Z

Looking at projections for 2020 and comparing data as named in the original source with the outputs of the technology data

                            original            new  currency_year
H2 (g) pipeline investment    363.08     303.684542         2023.0
HVDC overhead   investment     400.0     442.141439         2011.0
nuclear         VOM             4.24       3.546388         2023.0
                fuel             3.0       3.412227         2010.0
                investment   10275.0    8594.135356         2023.0
methanolisation investment  788000.0  819108.478032         2017.0

Let's do some example calculations e.g. for
methanolisation
investment costs 788 000 Eur_2017 -> inflation in 2018 1.8%, 2019 1.4%, 2020 0.7%
788 000 * (1.018 * 1.014 * 1.007) = 788 000 * 1.039478 = 819 108.47 Eur_2020

H2 pipelines
original 363.08 Eur_2023 -> inflation in 2021 2.9%, 2022 9.2% and 2023 6.4%
363.08 / (1.029 * 1.092 * 1.064) = 363.08 / 1.1955 = 303.6845

euronion · 2024-02-19T07:16:56Z

I have to say I'm a little bit confused by the Eurostat dataset browser sometimes because it sometimes includes and sometimes doesn't include 2023. But here it seems possible to get a dataset in the same format with 2023 included? https://ec.europa.eu/eurostat/databrowser/view/prc_hicp_aind__custom_9900786/default/table?lang=en
I think it would be nice to have as recent as possible data available, and there are now inflation values for 2023 (namely, 6.4%).
Apart from that, thanks a bunch for this work!

Good idea to include 2023, unfortunately the data you are linking to is only dating back to 2015 and it would be nice to have the earlier years in there as well. I have added 2023 now within the function when preparing the data (not the nicest way, but at least 2023 is included now as well)...

You could use the OECD data repository instead, they list HICP since 1991 for the current EURO area and allow for download as CSV or by API access:

https://data-explorer.oecd.org/vis?tm=HICP&pg=0&snb=5&vw=tb&df[ds]=dsDisseminateFinalDMZ&df[id]=DSD_PRICES%40DF_PRICES_HICP&df[ag]=OECD.SDD.TPS&df[vs]=1.0&pd=%2C&dq=.A.HICP.CPI.PA._T.N.GY&ly[rw]=REF_AREA&ly[cl]=TIME_PERIOD&to[TIME_PERIOD]=false&lo=100&lom=LASTNPERIODS

lisazeyen · 2024-02-19T09:54:20Z

I have to say I'm a little bit confused by the Eurostat dataset browser sometimes because it sometimes includes and sometimes doesn't include 2023. But here it seems possible to get a dataset in the same format with 2023 included? https://ec.europa.eu/eurostat/databrowser/view/prc_hicp_aind__custom_9900786/default/table?lang=en
I think it would be nice to have as recent as possible data available, and there are now inflation values for 2023 (namely, 6.4%).
Apart from that, thanks a bunch for this work!

Good idea to include 2023, unfortunately the data you are linking to is only dating back to 2015 and it would be nice to have the earlier years in there as well. I have added 2023 now within the function when preparing the data (not the nicest way, but at least 2023 is included now as well)...

You could use the OECD data repository instead, they list HICP since 1991 for the current EURO area and allow for download as CSV or by API access:

https://data-explorer.oecd.org/vis?tm=HICP&pg=0&snb=5&vw=tb&df[ds]=dsDisseminateFinalDMZ&df[id]=DSD_PRICES%40DF_PRICES_HICP&df[ag]=OECD.SDD.TPS&df[vs]=1.0&pd=%2C&dq=.A.HICP.CPI.PA._T.N.GY&ly[rw]=REF_AREA&ly[cl]=TIME_PERIOD&to[TIME_PERIOD]=false&lo=100&lom=LASTNPERIODS

they are mentioning as source Eurostat (=.
I found a Eurostat table now with data from 2001-2023 and added that one as well as the API link to the source

lisazeyen · 2024-02-19T11:00:30Z

@fneum I compared now the original data input with the cost adjusted values. Comparing the old and new costs is not very insightful since many of the input data previously was not inflation adjusted.

One thing to double check, I am basically assuming now when converting e.g. Eur_2017 to Eur_2020 the inflation of the three year 2018, 2019, 2020, since Eur_2017 should consider already the inflation for 2017 right?

fneum · 2024-02-19T12:57:13Z

One thing to double check, I am basically assuming now when converting e.g. Eur_2017 to Eur_2020 the inflation of the three year 2018, 2019, 2020, since Eur_2017 should consider already the inflation for 2017 right?

I think either way is fine, just it cannot be 2017, 2018, 2019, 2020.

lisazeyen added 2 commits February 5, 2024 12:52

change reference year to 2020

c3766b2

also adjust VOM and fixed to inflation

580fa14

add annual inflation rate

e9e2087

lisazeyen added 13 commits February 5, 2024 17:24

adjust DEA 2015 for inflation

3365684

merge latest changes

be7e6e8

fix some future warnings

c1a8bcc

first steps to add currency_year everywhere

e809090

remove upper version limit for pandas

595ec7e

remove breakpoint, remove FOM adjustment

155ddea

fixes adjustment inflation

c8c711d

set type to float

ab6b487

fix python version below 3.12

39d84ae

add annual inflation rate

8354226

remove fixed annual inflation rate

c2d9c7b

merge master

ac67edb

add release note

a3aa3c3

lisazeyen marked this pull request as ready for review February 16, 2024 09:14

lisazeyen requested a review from fneum February 16, 2024 09:14

add inflation rate for 2023

8788cac

lisazeyen and others added 2 commits February 16, 2024 13:58

fix bug

ea2ec91

add back in column headers and absolute local file path references

97c18f8

fneum requested changes Feb 16, 2024

View reviewed changes

docs/release_notes.rst Outdated Show resolved Hide resolved

environment.yaml Show resolved Hide resolved

lisazeyen added 2 commits February 19, 2024 10:49

change inflation rate source

d1167cb

update output files

286a3cb

adjust inflation function

a59fab0

lisazeyen requested a review from fneum February 19, 2024 11:00

update output files

11d382f

fneum merged commit 3c39918 into master Feb 19, 2024
1 of 2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change reference cost year to 2020 #121

Change reference cost year to 2020 #121

lisazeyen commented Feb 5, 2024 •

edited

Loading

fneum commented Feb 5, 2024

aleks-g commented Feb 5, 2024 •

edited

Loading

lisazeyen commented Feb 5, 2024

lisazeyen commented Feb 16, 2024

koen-vg commented Feb 16, 2024 •

edited

Loading

lisazeyen commented Feb 16, 2024

fneum left a comment

fneum commented Feb 16, 2024 •

edited by lisazeyen

Loading

lisazeyen commented Feb 16, 2024 •

edited

Loading

euronion commented Feb 19, 2024

lisazeyen commented Feb 19, 2024

lisazeyen commented Feb 19, 2024

fneum commented Feb 19, 2024

Change reference cost year to 2020 #121

Change reference cost year to 2020 #121

Conversation

lisazeyen commented Feb 5, 2024 • edited Loading

Changes proposed in this Pull Request

Checklist

fneum commented Feb 5, 2024

aleks-g commented Feb 5, 2024 • edited Loading

lisazeyen commented Feb 5, 2024

lisazeyen commented Feb 16, 2024

koen-vg commented Feb 16, 2024 • edited Loading

lisazeyen commented Feb 16, 2024

fneum left a comment

Choose a reason for hiding this comment

fneum commented Feb 16, 2024 • edited by lisazeyen Loading

lisazeyen commented Feb 16, 2024 • edited Loading

euronion commented Feb 19, 2024

lisazeyen commented Feb 19, 2024

lisazeyen commented Feb 19, 2024

fneum commented Feb 19, 2024

lisazeyen commented Feb 5, 2024 •

edited

Loading

aleks-g commented Feb 5, 2024 •

edited

Loading

koen-vg commented Feb 16, 2024 •

edited

Loading

fneum commented Feb 16, 2024 •

edited by lisazeyen

Loading

lisazeyen commented Feb 16, 2024 •

edited

Loading