WIP: Helper Methods in SLC BaseClass

[elenya-grant]

Helper Methods in SLC BaseClass

Added some hopefully useful helper methods in the SLC baseclass and intended to resolve Issue #741. I also view this as a pre-requisite to tackling Issue #747. These methods are intended to help future control development be more easily expanded to varying system architectures and make them more friendly to multi-commodity systems (systems that contain converters such as an electrolyzer or Haber Bosch system). To me, this meant being able to understand complex systems in a more understandable way and to be able to easily translate the demanded commodity into the demand of each upstream technology. For example, how to translate the ammonia demand into hydrogen demand, and also translate that hydrogen demand into electricity demand and water demand.

The methods in the SLC baseclass do work for more complex system architectures than we can currently represent with the models in H2I. I think that the methods would be able to handle architectures that include splitters (and if not, I think the changes to them would be more simple). I think that integrating splitters may just change how some of the methods called by controllers that inherit the baseclass provide the necessary inputs. Aka - I think that _post_setup_multi_commodity() should work for splitters (or require minor changes).

This PR also includes some updates to the DemandFollowingControl to show-case how to use some of the methods in the SLC baseclass. I am still working on adding all the necessary doc-strings and cleaning up the code.

Please do not yet push any changes to this branch until the TODOs are complete

A note to reviewers:

Hi reviewer! First, I tried to put some "worked examples" in the doc strings of the methods I added, and these can be seen in a nice way if you look at the read the docs build for the SLC base class. All the methods I added are at the end of the file. I also showed examples of the main attributes instantiated in the _post_setup_multi_commodity() method in the docs page for the SLC baseclass under the section titled "Heterogeneous Commodity Hybrid System". Give those a look!

What would be very helpful to me is if you went through the methods in the SLC baseclass that I call out in "Summary of methods added in the SLC base class" (only look at the ones in the demand following controller if it helps you understand how the ones in the SLC baseclass are used). The most complicated methods are the ones noted in Step 1. I would really love to know which of the methods called in _post_setup_multi_commodity are the most confusing/hard to follow (and perhaps some more detail on what parts are confusing). I would very much recommend reading Step 1 as you look at the code and the worked out examples in the doc page. I think that understanding the variables at each substep of Step 1 are crucial to understanding what's happening in _post_setup_multi_commodity(), the methods used in _post_setup_multi_commodity(), and the attributes/methods that are used (or useful) in Step 2 and Step 3.

Please communicate with me if you want to push up any changes to the code. In some places of the code, the code may seem overly complex because it has to be that complicated to work properly (but sometimes it may be overly complex for no good reason - and we should figure that out together). Also - sometimes in the code you may realize that things could be make 1-liners, but for the sake of readability I didn't do that. There are also errors and warnings that aren't very descriptive - they aren't very descriptive because usually it means that some unexpected system design was encountered or that something else "upstream" may have broke. Also - I totally expect to rename some methods and variable names - please feel free to recommend ideas for different naming, but don't focus on that (yet).

Some of the tests in h2integrate/control/control_strategies/system_level/test/test_slc_baseclass.py are a bit confusing. I can/will clean these up before this PR is merged in. Right now, this mess of tests is to ensure that nothing breaks if changes are made.

Please focus on the new methods added to the SLC baseclass, specifically the ones called in _post_setup_multi_commodity. I did make changes to the demand following control and the demand following controller still has its limitation. The goal of this PR was to add in helpful methods to the SLC baseclass and showcase their utilization - not to address the short falls of the demand following controller. Future PRs can work on making the demand following controller more robust and flexible - but not this one!

I think that the conversion_recipes attribute thats made at the end of the post_setup_multi_commodity() is the most confusing and hard-to-explain attribute. I tried to make using this attribute more developer-friendly with the methods get_conversion_from_recipe and get_techs_to_demand_from_recipe.

The only other part of this PR body that may be good to look at (outside of Section 1) is the TODO list. Besides that, the only thing you should read right now (assuming you've read everything above here) is "Summary of methods added in the SLC base class".

Summary of methods added in the SLC base class

The methods included in this PR that were added to SystemLevelControlBase perhaps fall into 3 buckets, this is summarized below. Please note that the term "converter" I'm using throughout this PR body is used to refer to components that convert at least one commodity type into at least one other commodity type. Resource (such as wind and solar) is not the same as a commodity. An electrolyzer is a converter, a wind farm is not.

1. "Setup" methods to understand the system described by the technology_graph

This step is a pre-cursor to 2 and 3. The top-level method of this is the _post_setup_multi_commodity (which is called at the end of the setup() method of the SystemLevelControlBase). The goals of this method are to:

• find the converter technologies within the system and their relationships to other technologies. This is done with the method _find_converter_techs(), which outputs converters and converter_upstreams.

  • _find_converter_techs() outputs converters and converter_upstreams, which are used in the following steps and are saved in the HCHSConfig (aka - these are important)
  • _find_converter_techs() calls the method get_successors_for_tech_with_input_cmod(). get_successors_for_tech_with_input_cmod() is used elsewhere in the code and may be a good option to replace or be used within the method get_upstream_techs_for_commodity()
  • methods called:
    • _find_converter_techs(): only called in _post_setup_multi_commodity()
    • get_successors_for_tech_with_input_cmod(): currently only used in the context of _post_setup_multi_commodity(), but may be useful elsewhere.
  • HCHSConfig attributes created: converters and converter_upstreams

• simplify the technology graph into "subsystems". Each "subsystem" is defined by its relationship to converters and its output commodities. This simplification is done within _post_setup_multi_commodity(). It starts with creating grouped_techs and grouped_commodities. Then it creates a more "easily searchable" version of grouped_techs and grouped_commodities called reversed_grouped_techs and reversed_group_commodities. grouped_techs is used later on, but grouped_commodities, reversed_grouped_techs, and reversed_group_commodities are only used to make simple_graph. simple_graph is directional graph representation of the subsystems and their relationships to each other.

  • methods called: none
  • variables needed: converters and converter_upstreams
  • HCHSConfig attributes created: grouped_techs and simple_graph

• make instructions on how to translate the "system demand" (user-defined in the SLC topology) to the demand of each subsystem. I.e., convert ammonia demand into hydrogen demand, then also convert that hydrogen demand into electricity demand (there are two recipes - one for the hydrogen subsystem and one for the electricity subsystem that feeds into the hydrogen subsystem). This is done in the method _make_conversion_factor_recipes() which outputs conversion_recipes.

  • methods called:
    • _make_conversion_factor_recipes(): only called in _post_setup_multi_commodity()
      • this uses another method _make_recipe_from_grouped_path(). _make_recipe_from_grouped_path() is only called from _make_conversion_factor_recipes()
  • variables needed: converters, simple_graph, and grouped_techs
  • HCHSConfig attributes created: conversion_recipes

• other information that is summarized in _post_setup_multi_commodity but not necessary to the main functionality this PR offers are the parameters non_converter_keys and converter_tech_names. These are stored as attributes in the HCHSConfig to partially simplify some stuff later on. non_converter_keys is stored in HCHSConfig as the attribute name non_converter_conversion_factor_keys.

2. "Compute" methods to calculate ratios (production and capacity) of different "subsystems". These methods are intended to calculate some input/output ratio from the inputs. The logic within these methods are independent of anything done in Step 1 - but providing the inputs to them is made substantially easier with the use of the converters and converter_upstreams attributes used in Step 1. The usage of these methods is showcased in the get_conversion_factors() method added to the DemandFollowingControl controller. Note that within the DemandFollowingControl.get_conversion_factors() method, we also see where the non_converter_conversion_factor_keys comes in handy. The two methods are:

   • get_converter_conversion_ratio(): calculates the ratio of input commodity production to output commodity production
   • get_converter_capacity_ratio(): calculates the ratio of input commodity capacity to output commodity capacity

3. "Compute" methods for getting information about a "subsystem" and the relationship between that "subsystem" and the demand. These methods would be called from a class that inherits SystemLevelControlBase in the compute() method. These methods are intended to make the conversion_recipes created in Step 1 to be easily usable by classes that inherit it. conversion_recipes is perhaps the most confusing variable made in Step 1 - but these two methods are helpful at extracting the important information from it. Similar to Step 2, the usage of these are showcased in the compute() method of DemandFollowingControl. The methods are get_techs_to_demand_from_recipe() and get_conversion_from_recipe().

   • get_techs_to_demand_from_recipe() will take in a "recipe name" (key of conversion_recipes) and return the technologies that should be "dispatched" based on that recipe. This method uses the attributes simple_graph and grouped_techs created in Step 1.
   • get_conversion_from_recipe() takes in a dictionary of conversion factors (like those created in DemandFollowingControl.get_conversion_factors() and a "recipe" (value from conversion_recipes). It uses the recipe to determine which of the conversion_factors to multiply together in order to calculate the compounding conversion ratio.

Section 1: Type of Contribution

• Feature Enhancement
  • Framework
  • New Model
  • Updated Model
  • Tools/Utilities
  • Other (please describe):
• Bug Fix
• Documentation Update
• CI Changes
• Other (please describe):

Section 2: Draft PR Checklist

• Open draft PR
• Describe the feature that will be added
• Fill out TODO list steps
• Describe requested feedback from reviewers on draft PR
• Complete Section 7: New Model Checklist (if applicable)

TODO:

• Split out the new example into two separate folders
• Finish adding doc strings to the methods in the SLC baseclass
• Add another example or test with using non-averaged conversion factors
• Move new attributes set in _post_setup_multi_commodity to a separate class
• Rename ChangeNameAttributeClass
• Rename slc attribute rename_me_config to something else
• Add subtests into test_find_converter_techs_fake_system
• Rename some methods
• Re-do subtests that are commented out in h2integrate/control/control_strategies/system_level/test/test_slc_baseclass.py

Type of Reviewer Feedback Requested (on Draft PR)

Structural feedback:

Implementation feedback:

• Should methods that are only called in _post_setup_multi_commodity() be updated to have the format __method_name__? Or no?
• Should methods that are called from the DemandFollowingControl class be formatted as _method_name or just method_name?

Other feedback:

• Any tips on formatting some pretty detailed variables in docstrings? Some of the doc strings are not fun to format and I'm struggling to explain some of the format of the input and output variables well. A lot of variables are dictionaries with lists as a tuple of strings and values that are equally as complex. For example, one variable, converter_upstreams, is a dictionary like:

converter_upstreams[("electricity", "electrolyzer")] = {"solar", "battery", "wind"}

Section 3: General PR Checklist

• PR description thoroughly describes the new feature, bug fix, etc.
• Added tests for new functionality or bug fixes
• Tests pass (If not, and this is expected, please elaborate in the Section 6: Test Results)
• Documentation
  • Docstrings are up-to-date
  • Related docs/ files are up-to-date, or added when necessary
  • Documentation has been rebuilt successfully
  • Examples have been updated (if applicable)
• CHANGELOG.md
  • At least one complete sentence has been provided to describe the changes made in this PR
  • After the above, a hyperlink has been provided to the PR using the following format:
    "A complete thought. [PR XYZ]((https://github.com/NatLabRockies/H2Integrate/pull/XYZ)", where
    XYZ should be replaced with the actual number.

Section 4: Related Issues

Intended to resolve Issue #741

Section 5: Impacted Areas of the Software

Section 5.1: New Files

• h2integrate/control/control_strategies/system_level/test/test_slc_baseclass.py: new test file for some methods introduced in the SLC baseclass

Section 5.2: Modified Files

• h2integrate/core/h2integrate_model.py._classify_slc_technologies(): updated so that other inputs (such as tech_control_classifiers and storage_techs_to_control) only include technologies that are upstream of the demand component.
• h2integrate/control/control_strategies/system_level/demand_following_control.py
  • DemandFollowingControlConfig: new configuration class for the demand following controller
  • DemandFollowingControl
    • Basically wrapped a lot of what was in compute() in a method named get_setpoints_for_commodity_subset()
    • get_conversion_factors(): new method
    • compute(): changed a bunch

Many new methods were added to h2integrate/control/control_strategies/system_level/system_level_control_base.py. Below is a summary of the major new methods and the other methods that they use

• _post_setup_multi_commodity(): called in setup() and uses the below methods
  • _find_converter_techs: only called during _post_setup_multi_commodity() and uses the below methods
    • get_successors_for_tech_with_input_cmod: called elsewhere - not specific to this method.
  • _make_conversion_factor_recipes: only called during _post_setup_multi_commodity()
    • _make_recipe_from_grouped_path: only called in _make_conversion_factor_recipes()
  • sets attributes using the ChangeNameAttributeClass
    • simple_graph
    • non_converter_conversion_factor_keys:
    • grouped_techs:
    • converters:
    • converter_upstreams:
    • converter_tech_names: could perhaps remove
    • conversion_recipes:

New methods in the SLC baseclass that are used by the DemandFollowingControl.compute() method are:

• get_techs_to_demand_from_recipe
• get_conversion_from_recipe
• get_converter_capacity_ratio
• get_converter_conversion_ratio

Section 6: Additional Supporting Information

Section 7: Test Results, if applicable

Section 8 (Optional): New Model Checklist

• Model Structure:
  • Follows established naming conventions outlined in docs/developer_guide/coding_guidelines.md
  • Used attrs class to define the Config to load in attributes for the model
    • If applicable: inherit from BaseConfig or CostModelBaseConfig
  • Added: initialize() method, setup() method, compute() method
    • If applicable: inherit from CostModelBaseClass
• Integration: Model has been properly integrated into H2Integrate
  • Added to supported_models.py
  • If a new commodity_type is added, update create_financial_model in h2integrate_model.py
• Tests: Unit tests have been added for the new model
  • Pytest-style unit tests
  • Unit tests are in a "test" folder within the folder a new model was added to
  • If applicable add integration tests
• Example: If applicable, a working example demonstrating the new model has been created
  • Input file comments
  • Run file comments
  • Example has been tested and runs successfully in test_all_examples.py
• Documentation:
  • Write docstrings using the Google style
  • Model added to the main models list in docs/user_guide/model_overview.md
    • Model documentation page added to the appropriate docs/ section
    • <model_name>.md is added to the _toc.yml
  • Run generate_class_hierarchy.py to update the class hierarchy diagram in docs/developer_guide/class_structure.md