epic: add warehouse by atravitz · Pull Request #1864 · OpenFreeEnergy/openfe

atravitz · 2026-03-05T21:58:02Z

This introduces the concept of a warehouse (name to be workshopped?), which replaces the need for metadatastore , resultclient and resultserver functionality.

I do not consider this to be breaking compatibility, since we're removing unused code (that was not in our docs) and adding new functionality.

Checklist

All new code is appropriately documented (user-facing code must have complete docstrings): see warehouse: docs #1988 for separate docs work
Added a news entry, or the changes are not user-facing: this will come when merging into main
Ran pre-commit: you can run pre-commit locally or comment on this PR with pre-commit.ci autofix.

Manual Tests: these are slow so don't need to be run every commit, only before merging and when relevant changes are made (generally at reviewer-discretion).

GPU integration tests
example notebook testing
packaging tests: run this for any large feature PRs or PRs that add test data.

Developers certificate of origin

I certify that this contribution is covered by the MIT License here and the Developer Certificate of Origin at https://developercertificate.org/.

…orage (#1632) * feat: remove result_server references in favor of gufe external_storage Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * fix: remove extra external_store * test: change from result_server to external_storage * test: remove external_store reference * test: change extension to item per class changes * refactor: remove result server import * chore: remove metadata store from resultclient Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * refactor: remove result server * refactor: remove metadata store --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* refactor(warehouse): rename to warehouse * refactor: rename _ResultContainerTest to _ContainerTest * refactor: rename _Container instances to _DataContainer

…ehouse

* refactor: remove extra implementations We may end up adding these back later but for the time being, we are going to remove them to keep life easy. * refactor: remove load_* and store_* functions Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * refactor: keys are being stored really strangely * refactor(warehouse): huge refactor to simplify working with Warehouse * refactor(warehouse): remove dead code * fix: delete failed due to incorrect object * test: add a test for the FileSystemWarehouse * refactor: remove dead code * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs: update warehouse docstrings * feat(warehouse): leverage keyed_chain for object dedup * Update openfe/storage/warehouse.py * import Literal * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs(warehouse): change from return, to raises * refactor(warehouse): move _key_exists to exists * chore(warehouse): remove extra todo * chore(warehouse): remove _load_stream * fix(warehouse): deduplicate objects on the filesystem * chore(warehouse): add typehint to exists --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>

* build with pooch@main to see if hotfix works * add link

…ehouse

* feat(warehouse): add result tokenizable store * docs(warehouse): add docs on the result store * chore: add property for result store * fix: use the correct function signature for handling setup --------- Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>

…ehouse

codecov · 2026-03-05T23:35:00Z

Codecov Report

❌ Patch coverage is 97.39583% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.24%. Comparing base (b3becad) to head (755cfd1).
⚠️ Report is 2 commits behind head on epic/execution_improvements.

Files with missing lines	Patch %	Lines
src/openfe/storage/warehouse.py	92.95%	5 Missing ⚠️

Additional details and impacted files

@@                       Coverage Diff                       @@
##           epic/execution_improvements    #1864      +/-   ##
===============================================================
- Coverage                        95.13%   90.24%   -4.90%     
===============================================================
  Files                              213      209       -4     
  Lines                            20871    20475     -396     
===============================================================
- Hits                             19855    18477    -1378     
- Misses                            1016     1998     +982

Flag	Coverage Δ
fast-tests	`90.24% <97.39%> (?)`
slow-tests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ehouse

atravitz · 2026-07-14T21:32:25Z

we expect the API break here since we're removing methods, but we should get type checking to pass.

…ehouse

github-actions · 2026-07-29T15:24:49Z

🚨 API breaking changes detected! 🚨

View workflow run

Griffe output

$ griffe check "openfe" -s src --no-inspection --no-color --verbose -a origin/main
src/openfe/tests/storage/test_resultclient.py:0: <module>:
Public object was removed

src/openfe/tests/storage/test_metadatastore.py:0: <module>:
Public object was removed

src/openfe/tests/storage/test_resultserver.py:0: <module>:
Public object was removed

src/openfe/storage/metadatastore.py:0: <module>:
Public object was removed

src/openfe/storage/resultserver.py:0: <module>:
Public object was removed

src/openfe/storage/resultclient.py:0: <module>:
Public object was removed


$ griffe check "openfecli" -s src --no-inspection --no-color --verbose -a origin/main

…ehouse

* refactor: remove result_server references in favor of gufe ExternalStorage (#1632) * feat: remove result_server references in favor of gufe external_storage Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * fix: remove extra external_store * test: change from result_server to external_storage * test: remove external_store reference * test: change extension to item per class changes * refactor: remove result server import * chore: remove metadata store from resultclient Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * refactor: remove result server * refactor: remove metadata store --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * refactor: rename to WarehouseBaseClass (#1660) * refactor(warehouse): rename to warehouse * refactor: rename _ResultContainerTest to _ContainerTest * refactor: rename _Container instances to _DataContainer * feat/refactor: add setup store (#1671) * refactor: remove extra implementations We may end up adding these back later but for the time being, we are going to remove them to keep life easy. * refactor: remove load_* and store_* functions Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * refactor: keys are being stored really strangely * refactor(warehouse): huge refactor to simplify working with Warehouse * refactor(warehouse): remove dead code * fix: delete failed due to incorrect object * test: add a test for the FileSystemWarehouse * refactor: remove dead code * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs: update warehouse docstrings * feat(warehouse): leverage keyed_chain for object dedup * Update openfe/storage/warehouse.py * import Literal * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs(warehouse): change from return, to raises * refactor(warehouse): move _key_exists to exists * chore(warehouse): remove extra todo * chore(warehouse): remove _load_stream * fix(warehouse): deduplicate objects on the filesystem * chore(warehouse): add typehint to exists --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io> * Temporarily build pooch from main w/ hotfix (#1806) * build with pooch@main to see if hotfix works * add link * Add support for result tokenizables to warehouse (#1763) * feat(warehouse): add result tokenizable store * docs(warehouse): add docs on the result store * chore: add property for result store * fix: use the correct function signature for handling setup --------- Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io> * test: cleanup warehouse test to be more modular (#1809) * fix dict type * add news item * run mypy CI on PRs into all branches (not just main) (#2112) * ignore a couple type check violations for now --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Ethan Holz <ethan.holz@omsf.io>

ethanholz and others added 12 commits November 3, 2025 07:56

refactor: rename to WarehouseBaseClass (#1660)

c27bfa5

* refactor(warehouse): rename to warehouse * refactor: rename _ResultContainerTest to _ContainerTest * refactor: rename _Container instances to _DataContainer

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

8d9995c

…ehouse

Temporarily build pooch from main w/ hotfix (#1806)

f6b8499

* build with pooch@main to see if hotfix works * add link

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

0e186a0

…ehouse

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

eddba56

…ehouse

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

c86aaea

…ehouse

test: cleanup warehouse test to be more modular (#1809)

3acdd18

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

fd24777

…ehouse

Merge branch 'main' into feat/warehouse

70c62d0

atravitz changed the title ~~epic: warehouse & exorcist work~~ epic: warehouse Mar 5, 2026

dotsdl self-requested a review March 16, 2026 18:03

atravitz self-assigned this Mar 20, 2026

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

7ec596b

…ehouse

atravitz force-pushed the feat/warehouse branch from b0108f2 to 7ec596b Compare June 5, 2026 20:04

atravitz mentioned this pull request Jun 8, 2026

Exorcist & Warehouse Integration Plan #1990

Open

9 tasks

atravitz mentioned this pull request Jul 13, 2026

update exorcist to use new gufe storage (breaking change) #2073

Open

atravitz added 3 commits July 13, 2026 11:34

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

c7ef32e

…ehouse

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

11d44fc

…ehouse

fix dict type

2fa2e76

atravitz and others added 2 commits July 21, 2026 10:27

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

54e876b

…ehouse

Merge branch 'main' into feat/warehouse

86caac0

atravitz changed the base branch from main to epic/execution_improvements July 30, 2026 18:25

atravitz added 2 commits July 30, 2026 11:31

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

e2edac1

…ehouse

add news item

ee2ccd1

atravitz marked this pull request as ready for review July 30, 2026 18:34

atravitz and others added 2 commits July 30, 2026 11:42

run mypy CI on PRs into all branches (not just main) (#2112)

1692562

Merge branch 'main' of github.com:OpenFreeEnergy/openfe into feat/war…

801cafe

…ehouse

atravitz changed the title ~~epic: warehouse~~ epic: add warehouse Jul 30, 2026

ignore a couple type check violations for now

755cfd1

atravitz force-pushed the feat/warehouse branch from 4c3e98b to 755cfd1 Compare July 30, 2026 21:52

atravitz merged commit ef0a60f into epic/execution_improvements Jul 30, 2026
6 of 8 checks passed

atravitz deleted the feat/warehouse branch July 30, 2026 21:58

atravitz restored the feat/warehouse branch July 30, 2026 22:00

Navigation Menu

Uh oh!

epic: add warehouse - #1864

epic: add warehouse#1864
atravitz merged 23 commits into
epic/execution_improvementsfrom
feat/warehouse

atravitz commented Mar 5, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

atravitz commented Jul 14, 2026

Uh oh!

github-actions Bot commented Jul 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

atravitz commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Developers certificate of origin

Uh oh!

codecov Bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

atravitz commented Jul 14, 2026

Uh oh!

github-actions Bot commented Jul 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

atravitz commented Mar 5, 2026 •

edited

Loading

codecov Bot commented Mar 5, 2026 •

edited

Loading