-
Notifications
You must be signed in to change notification settings - Fork 3
WIP: docs(guides): Add new admin guides #32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
trispera
wants to merge
9
commits into
main
Choose a base branch
from
updates
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
d3cef1c
docs(guides): Add new admin guides
trispera 64ffc15
Fix links. Add Sourcery recommendations
trispera dc76358
Update docs/guides/guide-admin/index.md
trispera 445025d
Update docs/guides/guide-user/index.md
trispera 8bb141b
Update docs/guides/guide-admin/tesk.md
trispera 273a098
Update docs/guides/guide-admin/prowes.md
trispera ea236d4
Update docs/guides/guide-admin/protes.md
trispera 9964f7c
Update docs/guides/guide-admin/funnel.md
trispera 9949d36
Fix typo
trispera File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,3 @@ | ||
| site/ | ||
| _site/ | ||
| .venv |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| ## Synopsis | ||
|
|
||
| GA4GH Service Registry API implementation for the ELIXIR Cloud. | ||
|
|
||
| Service entries comply with the [external service schema](https://github.com/ga4gh-discovery/ga4gh-service-registry/blob/8c45be52940db92c2fa1cd821519c271c22b1c4c/service-registry.yaml#L158) defined in the [GA4GH Service Registry API][ga4gh-service-registry] | ||
|
|
||
| Developers can find the API documentation [here](https://cloud-registry.readthedocs.io/en/latest/) | ||
|
|
||
| ## Installation | ||
|
|
||
| You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/cloud-registry/tree/dev/deployment) of Cloud-registry | ||
|
|
||
| Follow these instructions | ||
|
|
||
| - Install [Helm][helm-install] | ||
| - Clone the [Cloud-registry repository](https://github.com/elixir-cloud-aai/cloud-registry/) | ||
|
|
||
| ```sh | ||
| git clone https://github.com/elixir-cloud-aai/cloud-registry.git | ||
| ``` | ||
|
|
||
| - Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files | ||
|
|
||
| ## Usage | ||
|
|
||
| First you must create a namespace in Kubernetes in which to deploy Cloud-registry. The | ||
| commands below assume that everything is created in the context of this | ||
| namespace. How the namespace is created depends on the cluster, so we won't | ||
| document it here. | ||
|
|
||
| You need to edit the `values.yaml` file | ||
|
|
||
| After this you can deploy Cloud-registry using `helm`: | ||
|
|
||
| ```bash | ||
| helm install cloud-registry . -f values.yaml | ||
| ``` | ||
|
|
||
| ### Updates | ||
|
|
||
| If you want to edit any of the Deployments, you can update them with | ||
| `helm` and the `values.yaml` file. Once edited, you can run this command: | ||
|
|
||
| ```bash | ||
| helm upgrade cloud-registry . -f values.yaml | ||
| ``` | ||
|
|
||
| ## Technical details | ||
|
|
||
| ### MongoDB | ||
|
|
||
| The MongoDB database is deployed using: | ||
|
|
||
| - `templates/mongo-deploy.yaml` | ||
|
|
||
| ### Cloud-registry | ||
|
|
||
| TRS-Filer is deployed using: | ||
|
|
||
| - `templates/cloud-registry-deploy.yaml` | ||
|
|
||
| ## Destroy | ||
|
|
||
| Simply run: | ||
|
|
||
| ```bash | ||
| helm uninstall cloud-registry | ||
| ``` | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| ## Synopsis | ||
|
|
||
| Microservice implementing the [Global Alliance for Genomics and | ||
| Health][ga4gh] (GA4GH) [Workflow Execution Service][ga4gh-wes] (WES) | ||
| API specification for the execution of workflows written in the [Common | ||
| Workflow Language](https://www.commonwl.org/) (CWL). | ||
|
|
||
| cwl-WES is a core service of the [ELIXIR Cloud & AAI | ||
| project][elixir-cloud-aai-github]. | ||
|
|
||
| ## Description | ||
|
|
||
| cwl-WES (formerly: WES-ELIXIR) is a Flask/Gunicorn | ||
| application that makes use of [Connexion](https://github.com/ga4gh/workflow-execution-service-schemas) to implement the | ||
| [GA4GH WES OpenAPI specification][ga4gh-wes]. It enables clients/users | ||
| to execute [CWL](https://www.commonwl.org) workflows in the cloud via a [GA4GH Task Execution | ||
| Service][ga4gh-tes] (TES)-compatible execution backend (e.g., | ||
| [TESK][tesk] or [Funnel][funnel]). Workflows can be sent for execution, | ||
| previous runs can be listed, and the status and run information of individual | ||
| runs can be queried. The service leverages [cwl-tes][res-cwl-tes] to | ||
| interpret [CWL](https://www.commonwl.org) workflows, break them down into individual tasks and | ||
| emit [GA4GH TES][ga4gh-tes]-compatible HTTP requests to a configured | ||
| [TES][ga4gh-tes] instance. Access to endpoints can be configured to require | ||
| JSON Web Token-based access tokens, such as those issued by | ||
| [ELIXIR AAI](https://elixir-europe.org/platforms/compute/aai). Run information is stored in a | ||
| MongoDB database. | ||
|
|
||
| Note that development is currently in beta stage. | ||
| Further test deployments can be found at the [ELIXIR Cloud & AAI's resource | ||
| listings](https://github.com/elixir-cloud-aai/elixir-cloud-aai/blob/dev/resources/resources.md). | ||
|
|
||
| cwl-WES is developed and maintained by the [ELIXIR Cloud & AAI | ||
| project][elixir-cloud], a multinational effort aimed at establishing and | ||
| implementing [FAIR][fair] research in the Life Sciences. | ||
|
|
||
| ## Installation | ||
|
|
||
| You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/cwl-wes/tree/dev/deployment) of CWL-WES | ||
|
|
||
| Follow these instructions | ||
|
|
||
| - Install [Helm][helm-install] | ||
| - Clone the [CWL-WES repository](https://github.com/elixir-cloud-aai/cwl-wes/) | ||
|
|
||
| ```sh | ||
| git clone https://github.com/elixir-cloud-aai/cwl-wes.git | ||
| ``` | ||
|
|
||
| - Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files | ||
|
|
||
| ## Usage | ||
|
|
||
| First you must create a namespace in Kubernetes in which to deploy CWL-WES. The | ||
| commands below assume that everything is created in the context of this | ||
| namespace. How the namespace is created depends on the cluster, so we won't | ||
| document it here. | ||
|
|
||
| You need to edit the `values.yaml` file | ||
|
|
||
| After this you can deploy CWL-WES using `helm`: | ||
|
|
||
| ```bash | ||
| helm install CWL-WES . -f values.yaml | ||
| ``` | ||
|
|
||
| ### Updates | ||
|
|
||
| If you want to edit any of the Deployments, you can update them with | ||
| `helm` and the `values.yaml` file. Once edited, you can run this command: | ||
|
|
||
| ```bash | ||
| helm upgrade CWL-WES . -f values.yaml | ||
| ``` | ||
|
|
||
| ## Technical details | ||
|
|
||
| ### MongoDB | ||
|
|
||
| The MongoDB database is deployed using: | ||
|
|
||
| - `templates/mongodb-deployment.yaml` | ||
|
|
||
| ### RabbitMQ | ||
|
|
||
| The message broker RabbitMQ that allows the app to communicate with the | ||
| worker is deployed using: | ||
|
|
||
| - `templates/rabbitmq/rabbitmq-deployment.yaml` | ||
|
|
||
| ### CWL-WES | ||
|
|
||
| CWL-WES consists of a Flask server and a Celery worker. | ||
| There are deployed using: | ||
|
|
||
| - `templates/wes-deployment.yaml` | ||
| - `templates/celery-deployment.yaml` | ||
|
|
||
| ## Destroy | ||
|
|
||
| Simply run: | ||
|
|
||
| ```bash | ||
| helm uninstall cwl-wes | ||
| ``` | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| ## Synopsis | ||
|
|
||
| Microservice implementing the [Global Alliance for Genomics and | ||
| Health (GA4GH)][ga4gh] [Data Repository Service][ga4gh-drs] (DRS) | ||
| API specification. | ||
|
|
||
| ## Installation | ||
|
|
||
| You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/drs-filer/tree/dev/deployment) of DRS-Filer | ||
|
|
||
| Follow these instructions | ||
|
|
||
| - Install [Helm][helm-install] | ||
| - Clone the [DRS-Filer repository](https://github.com/elixir-cloud-aai/drs-filer/) | ||
|
|
||
| ```sh | ||
| git clone https://github.com/elixir-cloud-aai/drs-filer.git | ||
| ``` | ||
|
|
||
| - Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files | ||
|
|
||
| ## Usage | ||
|
|
||
| First you must create a namespace in Kubernetes in which to deploy DRS-Filer. The | ||
| commands below assume that everything is created in the context of this | ||
| namespace. How the namespace is created depends on the cluster, so we won't | ||
| document it here. | ||
|
|
||
| You need to edit the `values.yaml` file | ||
|
|
||
| After this you can deploy DRS-Filer using `helm`: | ||
|
|
||
| ```bash | ||
| helm install drs-filer . -f values.yaml | ||
| ``` | ||
|
|
||
| ### Updates | ||
|
|
||
| If you want to edit any of the Deployments, you can update them with | ||
| `helm` and the `values.yaml` file. Once edited, you can run this command: | ||
|
|
||
| ```bash | ||
| helm upgrade drs-filer . -f values.yaml | ||
| ``` | ||
|
|
||
| ## Technical details | ||
|
|
||
| ### MongoDB | ||
|
|
||
| The MongoDB database is deployed using: | ||
|
|
||
| - `templates/mongo-deploy.yaml` | ||
|
|
||
| ### DRS-Filer | ||
|
|
||
| DRS-Filer is deployed using: | ||
|
|
||
| - `templates/drs-filer-deploy.yaml` | ||
|
|
||
| ## Destroy | ||
|
|
||
| Simply run: | ||
|
|
||
| ```bash | ||
| helm uninstall drs-filer | ||
| ``` | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| Follow these instructions if you wish to deploy a TES endpoint in front of your | ||
| HPC/HTC cluster (currently tested with [Slurm][slurm] and [OpenPBS][openpbs]. | ||
sourcery-ai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| - Make sure the build dependencies `make` and [Go 1.21+][go-install] are | ||
| installed, `GOPATH` is set and `GOPATH/bin` is added to `PATH`. | ||
|
|
||
| For example, in Ubuntu this can be achieved via: | ||
|
|
||
| ```sh | ||
| sudo apt update | ||
| sudo apt install make golang-go | ||
| export GOPATH=/your/desired/path | ||
| export PATH=$GOPATH/bin:$PATH | ||
| go version | ||
| ``` | ||
|
|
||
| - Clone the repository: | ||
|
|
||
| ```sh | ||
| git clone https://github.com/ohsu-comp-bio/funnel.git | ||
| ``` | ||
|
|
||
| - Build Funnel: | ||
|
|
||
| ```sh | ||
| cd funnel | ||
| make | ||
| ``` | ||
|
|
||
| - Test the installation by starting the Funnel server with: | ||
|
|
||
| ```sh | ||
| funnel server run | ||
| ``` | ||
|
|
||
| If all works, Funnel should be ready for deployment on your HPC/HTC. | ||
|
|
||
| Alternatively, you can install Funnel via Homebrew: | ||
|
|
||
| ```sh | ||
| brew tap ohsu-comp-bio/formula | ||
| brew install [email protected] | ||
| ``` | ||
|
|
||
| Source: [Funnel website](https://ohsu-comp-bio.github.io/funnel/) | ||
|
|
||
| ### Slurm | ||
|
|
||
| For the use of Funnel with Slurm, make sure the following conditions are met: | ||
|
|
||
| 1. The `funnel` binary must be placed in a server with access to Slurm. | ||
| 2. A config file must be created and placed on the same server. [This | ||
| file][funnel-config-slurm] can be used as a starting point. | ||
| 3. If we would like to deploy Funnel as a Systemd service, | ||
| [this file][funnel-config-slurm-service] can be used as a template. Set the | ||
| correct paths to the `funnel` binary and config file. | ||
|
|
||
| If successful, Funnel should be listening on port `8080`. | ||
|
|
||
| ### OpenPBS | ||
|
|
||
| !!! warning "Under construction" | ||
| More info coming soon... | ||
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.