Merge branch 'dev' into features/#450-add-metadata

CHANGELOG.rst

@@ -252,6 +252,8 @@ Added
   `PR #903 <https://github.com/openego/eGon-data/pull/903>`_
 * Add low flex scenario 'eGon2035_lowflex'
   `#822 <https://github.com/openego/eGon-data/issues/822>`_
+* Add MaStR geocoding and handling of conventional generators
+  `#1095 <https://github.com/openego/eGon-data/issues/1095>`_
 
 .. _PR #159: https://github.com/openego/eGon-data/pull/159
 .. _PR #703: https://github.com/openego/eGon-data/pull/703
@@ -456,6 +458,8 @@ Changed
   `#882 <https://github.com/openego/eGon-data/issues/882>`_
 * Insert crossboarding gas pipeline with Germany in eGon100RE
   `#881 <https://github.com/openego/eGon-data/issues/881>`_
+* Harmonize H2 carrier names in eGon100RE
+  `#929 <https://github.com/openego/eGon-data/issues/929>`_
 * Rename noflex to lowflex scenario for motorized individual travel
   `#921 <https://github.com/openego/eGon-data/issues/921>`_
 * Update creation of heat demand timeseries
@@ -475,6 +479,8 @@ Changed
   `#519 <https://github.com/openego/eGon-data/issues/519>`_
 * Add missing VOM costs for heat sector components
   `#942 <https://github.com/openego/eGon-data/issues/942>`_
+* Add sanity checks for gas sector in eGon2035
+  `#864 <https://github.com/openego/eGon-data/issues/864>`_
 * Desaggregate industry demands to OSM areas and industrial sites
   `#1001 <https://github.com/openego/eGon-data/issues/1001>`_
 * Add gas generator in Norway
@@ -484,11 +490,20 @@ Changed
   created for a single process. This fixes issue `#799`_.
 * Insert rural heat per supply technology
   `#1026 <https://github.com/openego/eGon-data/issues/1026>`_
+* Insert lifetime for components from p-e-s in eGon100RE
+  `#1073 <https://github.com/openego/eGon-data/issues/1073>`_
 * Change hgv data source to use database
   `#1086 <https://github.com/openego/eGon-data/issues/1086>`_
+* Change desposit ID for data_bundle download from zenodo sandbox
+  `#1110 <https://github.com/openego/eGon-data/issues/1110>`_
+* Use MaStR geocoding results for pv rooftop to buildings mapping workflow
+  `#1095 <https://github.com/openego/eGon-data/issues/1095>`_
+* Rename eMob MIT carrier names (use underscores)
+  `#1105 <https://github.com/openego/eGon-data/issues/1105>`_
 
 .. _#799: https://github.com/openego/eGon-data/issues/799
 
+
 Bug Fixes
 ---------
 
@@ -722,13 +737,17 @@ Bug Fixes
 * Fix URL of TYNDP scenario dataset
 * Automatically generated tasks now get unique :code:`task_id`\s.
   Fixes issue `#985`_ via PR `#986`_.
+* Adjust capcities of German CH4 stores
+  `#1096 <https://github.com/openego/eGon-data/issues/1096>`_
 * Fix faulty DSM time series
   `#1088 <https://github.com/openego/eGon-data/issues/1088>`_
 * Set upper limit on commissioning date for units from MaStR
   dataset
   `#1098 <https://github.com/openego/eGon-data/issues/1098>`_
 * Fix conversion factor for CH4 loads abroad in eGon2035
   `#1104 <https://github.com/openego/eGon-data/issues/1104>`_
+* Change structure of documentation in rtd
+  `#11126 <https://github.com/openego/eGon-data/issues/1126>`_
 
 .. _PR #692: https://github.com/openego/eGon-data/pull/692
 .. _#343: https://github.com/openego/eGon-data/issues/343

docs/_templates/module.rst_t

@@ -0,0 +1,9 @@
+{%- if show_headings %}
+{{- basename.split(".")[-1] | e | heading }}
+
+{% endif -%}
+.. automodule:: {{ qualname }}
+{%- for option in automodule_options %}
+   :{{ option }}:
+{%- endfor %}
+

docs/_templates/package.rst_t

@@ -0,0 +1,47 @@
+{% macro automodule(modname, options) %}
+
+.. automodule:: {{ modname }}
+{%- for option in options %}
+   :{{ option }}:
+{%- endfor %}
+{%- endmacro %}
+
+{% macro toctree(docnames) -%}
+
+.. toctree::
+   :maxdepth: {{ maxdepth }}
+{% for docname in docnames %}
+   {{ docname }}
+{%- endfor %}
+{%- endmacro %}
+
+{{- pkgname.split(".")[-1] | e | heading }}
+
+{%- if is_namespace %}
+
+.. py:module:: {{ pkgname }}
+{% endif %}
+
+{%- if modulefirst and not is_namespace %}
+{{ automodule(pkgname, automodule_options) }}
+{% endif %}
+
+{%- if submodules %}
+{% if separatemodules %}
+{{- toctree(submodules + subpackages) }}
+{% else %}
+{%- if subpackages %}
+{{ toctree(subpackages) }}
+{% endif %}
+{%- for submodule in submodules %}
+{% if show_headings %}
+{{- [submodule, "module"] | join(" ") | e | heading(2) }}
+{% endif %}
+{{ automodule(submodule, automodule_options) }}
+{% endfor %}
+{%- endif %}
+{%- endif %}
+
+{%- if not modulefirst and not is_namespace %}
+{{ automodule(pkgname, automodule_options) }}
+{% endif %}

docs/_templates/toc.rst_t

@@ -0,0 +1,7 @@
+{{ header | heading }}
+
+.. toctree::
+   :maxdepth: {{ maxdepth }}
+{% for docname in docnames %}
+   {{ docname }}
+{%- endfor %}

docs/about.rst

@@ -0,0 +1,111 @@
+***************
+About eGon-data
+***************
+
+Project background
+==================
+
+egon-data provides a transparent and reproducible open data-based data processing pipeline for generating data models suitable for energy system modeling. The data is customized for the requirements of the research project eGo_n. The research project aims to develop tools for open and cross-sectoral planning of transmission and distribution grids. For further information please visit the `eGo_n project website <https://ego-n.org/>`_.
+egon-data is a further development of the `Data processing <https://github.com/openego/data_processing>`_ developed in the former research project `open_eGo <https://openegoproject.wordpress.com/>`_. It aims to extend the data models as well as improve the replicability and manageability of the data preparation and processing. 
+The resulting data set serves as an input for the optimization tools `eTraGo <https://github.com/openego/eTraGo>`_, `ding0 <https://github.com/openego/ding0>`_ and `eDisGo <https://github.com/openego/eDisGo>`_ and delivers, for example, data on grid topologies, demands/demand curves and generation capacities in a high spatial resolution. The outputs of egon-data are published under open-source and open-data licenses.  
+
+
+Objectives of the project
+=========================
+
+Driven by the expansion of renewable generation capacity and the progressing electrification of other energy sectors, the electrical grid increasingly faces new challenges: fluctuating supply of renewable energy and simultaneously a changing demand pattern caused by sector coupling. However, the integration of non-electric sectors such as gas, heat, and e-mobility enables more flexibility options. The eGo_n project aims to investigate the effects of sector coupling on the electrical grid and the benefits of new flexibility options. This requires the creation of a spatially and temporally highly resolved database for all sectors considered. 
+
+Project consortium and funding
+==================================
+
+The following universities and research institutes were involved in the creation of eGon-data: 
+
+* University of Applied Sciences Flensburg
+* Reiner Lemoine Institut
+* Otto von Guericke University Magdeburg
+* DLR Institute of Networked Energy Systems
+* Europa-Universität Flensburg 
+
+The eGo_n project (FKZ: 03EI1002) is supported by the Federal Ministry for Economic Affairs and Climate Action (BMWK) on the basis of a decision by the German Bundestag.
+
+.. image:: images/Logos_Projektpartner_egon_data.png
+  :width: 400
+  :alt: Logos of project partners
+
+
+eGon-data as one element of the eGo-Toolchain
+=============================================
+
+In the eGo_n project different tools were developed, which are in exchange with each other and have to serve the respective requirements on data scope, resolution, and format. The results of the data model creation have to be especially adapted to the requirements of the tools eTraGo and eDisGo for power grid optimization on different grid levels. 
+A PostgreSQL database serves as an interface between the data model creation and the optimization tools.
+The figure below visualizes the interdependencies between the different tools. 
+
+.. image:: images/Toolchain_web_desktop.svg
+  :width: 800
+  :alt: eGon-data tool chain
+
+.. _concept-and-scenarios-ref:
+
+Modeling concept and scenarios
+===============================
+
+eGon-data provides a data model suitable for calculations and optimizations with the tools eTraGo, eDisGo and eGo and therefore aims to satisfy all requirements regarding the scope and temporal as well as spatial granularity of the resulting data model.
+The following image visualizes the different components considered in scenario ``eGon2035``.
+
+.. image:: images/egon-modell-szenario-egon2035.png
+  :width: 800
+  :alt: Components of the data models
+
+eGon-data aims to create different scenarios, which differ in terms of RE penetration or the availability of flexibility options. Currently, the following scenarios are available or in progress. 
+
+* ``eGon2035`` Mid-termin scenario based on assumptions from the German network expansion plan 'scenario C2035', version 2021 and TYNDP
+* ``eGon2035_lowflex`` Mid-termin scenario similar to 'eGon2035', but with a limited availability of flexibility options
+* ``eGon100RE`` Long-term scenario with a 100% RE penetration, based on optimization results with PyPSA-Eur-Sec and additional data inputs (work-in-progress)
+
+.. list-table:: Installed capacities of German power park in scenario ``eGon2035`` and ``eGon2035_lowflex``
+   :widths: 50 50
+   :header-rows: 1
+
+   * - carrier
+     - Installed capacities
+   * - gas
+     - 46.7 GW
+   * - oil
+     - 1.3 GW
+   * - pumped hydro
+     - 10.2 GW
+   * - wind onshore
+     - 90.9 GW
+   * - wind offshore
+     - 34.0 GW
+   * - solar
+     - 120.1 GW
+   * - biomass
+     - 8.7 GW
+   * - others
+     - 5.4 GW
+
+
+.. list-table:: German energy demands in scenarios ``eGon2035`` and ``eGon2035_lowflex``
+   :widths: 50 50
+   :header-rows: 1
+
+   * - Demand sector
+     - Energy demand
+   * - MIT transport
+     - 41.4 TWh :subscript:`el`
+   * - central heat
+     - 68.9 TWh :subscript:`th`
+   * - rural heat
+     - 423.2 TWh :subscript:`th`
+   * - electricity
+     - 498.1 TWh :subscript:`el`
+   * - Methane industry
+     - 196.0 TWh :subscript:`CH4`
+   * - Hydrogen industry
+     - 16.1 TWh :subscript:`H2`
+   * - Hydrogen transport
+     - 26.5 TWh :subscript:`H2`
+
+
+

docs/conf.py

@@ -44,3 +44,11 @@
 napoleon_use_ivar = True
 napoleon_use_rtype = False
 napoleon_use_param = False
+
+add_module_names = False
+modindex_common_prefix = ["egon.data.", "egon.data.datasets."]
+
+autodoc_type_aliases = {
+    "Dependencies": "egon.data.datasets.Dependencies",
+    "Tasks": "egon.data.datasets.Tasks"
+}

docs/data.rst

@@ -1,9 +1,124 @@
 ****
 Data
 ****
+The description of the methods, input data and results of the eGon-data pipeline is given in the following section.
+References to datasets and functions are integrated if more detailed information is required.
+
+Main input data and their processing
+====================================
+
+All methods in the eGon-data workflow rely on public and freely available data from different external sources. The most important data sources
+and their processing within the eGon-data pipeline are described here.
+
+.. include:: data/input_data.rst
+
+Grid models
+===========
+
+Power grid models of different voltage levels form a central part of the eGon data model, which is required for cross-grid-level optimization.
+In addition, sector coupling necessitates the representation of the gas grid infrastructure, which is also described in this section.
+
+Electricity grid
+----------------
+
+.. include:: data/electricity_grids.rst
+
+Gas grid
+--------
+
+.. include:: data/gas_grids.rst
+
+Demand
+======
+
+Electricity, heat and gas demands from different consumption sectors are taken into account in eGon-data. The related methods to distribute and
+process the demand data are described in the following chapters for the different consumption sectors separately.
+
+.. _electricity-demand-ref:
+
+Electricity
+-----------
+
+.. include:: data/electricity_demand.rst
+
+Heat
+----
+
+.. include:: data/heat_demand.rst
+
+Gas
+---
+
+.. include:: data/gas_demand.rst
+
+.. _mobility-demand-ref:
+
+Mobility
+--------
+
+.. include:: data/mobility_demand.rst
+
+
+Supply
+======
+
+The distribution and assignment of supply capacities or potentials are carried out technology-specific. The different methods are described in the
+following chapters.
+
+Electricity
+-----------
+
+.. include:: data/electricity_supply.rst
+
+Heat
+----
+
+.. include:: data/heat_supply.rst
+
+Gas
+---
+.. include:: data/gas_supply.rst
+
+
+Flexibility options
+===================
+
+Different flexibility options are part of the model and can be utilized in the optimization of the energy system. Therefore detailed information about
+flexibility potentials and their distribution are needed. The considered technologies described in the following chapters range from different storage units,
+through dynamic line rating to Demand-Side-Management measures.
+
+Demand-Side-Management
+----------------------
+
+.. include:: data/DSM.rst
+
+Dynamic line rating
+-------------------
+
+.. include:: data/DLR.rst
+
+E-Mobility
+----------
+
+.. include:: data/e-mobility.rst
+
+Battery stores
+----------------
+
+.. include:: data/batteries.rst
+
+Gas stores
+-----------------
+
+.. include:: data/gas_stores.rst
+
+Heat stores
+-------------
+
+.. include:: data/heat_stores.rst
 
-Scenarios
-=========
 
 Published data
 ==============
+
+