Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
site/
_site/
.venv
10 changes: 8 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,17 @@ i: install
.PHONY: venv
venv:
@echo "\nCreating a virtual environment ++++++++++++++++++++++++++++++++++++++++++++++++\n"
@python -m venv .venv
@if command -v python3 >/dev/null 2>&1; then \
python3 -m venv .venv; \
elif command -v python >/dev/null 2>&1; then \
python -m venv .venv; \
else \
echo "Python interpreter not found" >&2; exit 1; \
fi
@echo "\nSummary +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n"
@echo "Virtual environment created successfully."
@echo "To activate the environment for this shell session, run:"
@echo "source .venv/bin/activate"

.PHONY: v
v: venv
v: venv
69 changes: 69 additions & 0 deletions docs/guides/guide-admin/cloudregistry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
## Synopsis

GA4GH Service Registry API implementation for the ELIXIR Cloud.

Service entries comply with the [external service schema](https://github.com/ga4gh-discovery/ga4gh-service-registry/blob/8c45be52940db92c2fa1cd821519c271c22b1c4c/service-registry.yaml#L158) defined in the [GA4GH Service Registry API][ga4gh-service-registry]

Developers can find the API documentation [here](https://cloud-registry.readthedocs.io/en/latest/)

## Installation

You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/cloud-registry/tree/dev/deployment) of Cloud-registry

Follow these instructions

- Install [Helm][helm-install]
- Clone the [Cloud-registry repository](https://github.com/elixir-cloud-aai/cloud-registry/)

```sh
git clone https://github.com/elixir-cloud-aai/cloud-registry.git
```

- Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files

## Usage

First you must create a namespace in Kubernetes in which to deploy Cloud-registry. The
commands below assume that everything is created in the context of this
namespace. How the namespace is created depends on the cluster, so we won't
document it here.

You need to edit the `values.yaml` file

After this you can deploy Cloud-registry using `helm`:

```bash
helm install cloud-registry . -f values.yaml
```

### Updates

If you want to edit any of the Deployments, you can update them with
`helm` and the `values.yaml` file. Once edited, you can run this command:

```bash
helm upgrade cloud-registry . -f values.yaml
```

## Technical details

### MongoDB

The MongoDB database is deployed using:

- `templates/mongo-deploy.yaml`

### Cloud-registry

TRS-Filer is deployed using:

- `templates/cloud-registry-deploy.yaml`

## Destroy

Simply run:

```bash
helm uninstall cloud-registry
```

105 changes: 105 additions & 0 deletions docs/guides/guide-admin/cwlwes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
## Synopsis

Microservice implementing the [Global Alliance for Genomics and
Health][ga4gh] (GA4GH) [Workflow Execution Service][ga4gh-wes] (WES)
API specification for the execution of workflows written in the [Common
Workflow Language](https://www.commonwl.org/) (CWL).

cwl-WES is a core service of the [ELIXIR Cloud & AAI
project][elixir-cloud-aai-github].

## Description

cwl-WES (formerly: WES-ELIXIR) is a Flask/Gunicorn
application that makes use of [Connexion](https://github.com/ga4gh/workflow-execution-service-schemas) to implement the
[GA4GH WES OpenAPI specification][ga4gh-wes]. It enables clients/users
to execute [CWL](https://www.commonwl.org) workflows in the cloud via a [GA4GH Task Execution
Service][ga4gh-tes] (TES)-compatible execution backend (e.g.,
[TESK][tesk] or [Funnel][funnel]). Workflows can be sent for execution,
previous runs can be listed, and the status and run information of individual
runs can be queried. The service leverages [cwl-tes][res-cwl-tes] to
interpret [CWL](https://www.commonwl.org) workflows, break them down into individual tasks and
emit [GA4GH TES][ga4gh-tes]-compatible HTTP requests to a configured
[TES][ga4gh-tes] instance. Access to endpoints can be configured to require
JSON Web Token-based access tokens, such as those issued by
[ELIXIR AAI](https://elixir-europe.org/platforms/compute/aai). Run information is stored in a
MongoDB database.

Note that development is currently in beta stage.
Further test deployments can be found at the [ELIXIR Cloud & AAI's resource
listings](https://github.com/elixir-cloud-aai/elixir-cloud-aai/blob/dev/resources/resources.md).

cwl-WES is developed and maintained by the [ELIXIR Cloud & AAI
project][elixir-cloud], a multinational effort aimed at establishing and
implementing [FAIR][fair] research in the Life Sciences.

## Installation

You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/cwl-wes/tree/dev/deployment) of CWL-WES

Follow these instructions

- Install [Helm][helm-install]
- Clone the [CWL-WES repository](https://github.com/elixir-cloud-aai/cwl-wes/)

```sh
git clone https://github.com/elixir-cloud-aai/cwl-wes.git
```

- Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files

## Usage

First you must create a namespace in Kubernetes in which to deploy CWL-WES. The
commands below assume that everything is created in the context of this
namespace. How the namespace is created depends on the cluster, so we won't
document it here.

You need to edit the `values.yaml` file

After this you can deploy CWL-WES using `helm`:

```bash
helm install CWL-WES . -f values.yaml
```

### Updates

If you want to edit any of the Deployments, you can update them with
`helm` and the `values.yaml` file. Once edited, you can run this command:

```bash
helm upgrade CWL-WES . -f values.yaml
```

## Technical details

### MongoDB

The MongoDB database is deployed using:

- `templates/mongodb-deployment.yaml`

### RabbitMQ

The message broker RabbitMQ that allows the app to communicate with the
worker is deployed using:

- `templates/rabbitmq/rabbitmq-deployment.yaml`

### CWL-WES

CWL-WES consists of a Flask server and a Celery worker.
There are deployed using:

- `templates/wes-deployment.yaml`
- `templates/celery-deployment.yaml`

## Destroy

Simply run:

```bash
helm uninstall cwl-wes
```

67 changes: 67 additions & 0 deletions docs/guides/guide-admin/drsfiler.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
## Synopsis

Microservice implementing the [Global Alliance for Genomics and
Health (GA4GH)][ga4gh] [Data Repository Service][ga4gh-drs] (DRS)
API specification.

## Installation

You can find a Helm chart in the [GitHub repository](https://github.com/elixir-cloud-aai/drs-filer/tree/dev/deployment) of DRS-Filer

Follow these instructions

- Install [Helm][helm-install]
- Clone the [DRS-Filer repository](https://github.com/elixir-cloud-aai/drs-filer/)

```sh
git clone https://github.com/elixir-cloud-aai/drs-filer.git
```

- Browse to `deployment` to find the `Chart.yaml` and the `values.yaml` files

## Usage

First you must create a namespace in Kubernetes in which to deploy DRS-Filer. The
commands below assume that everything is created in the context of this
namespace. How the namespace is created depends on the cluster, so we won't
document it here.

You need to edit the `values.yaml` file

After this you can deploy DRS-Filer using `helm`:

```bash
helm install drs-filer . -f values.yaml
```

### Updates

If you want to edit any of the Deployments, you can update them with
`helm` and the `values.yaml` file. Once edited, you can run this command:

```bash
helm upgrade drs-filer . -f values.yaml
```

## Technical details

### MongoDB

The MongoDB database is deployed using:

- `templates/mongo-deploy.yaml`

### DRS-Filer

DRS-Filer is deployed using:

- `templates/drs-filer-deploy.yaml`

## Destroy

Simply run:

```bash
helm uninstall drs-filer
```

63 changes: 63 additions & 0 deletions docs/guides/guide-admin/funnel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
Follow these instructions if you wish to deploy a TES endpoint in front of your
HPC/HTC cluster (currently tested with [Slurm][slurm] and [OpenPBS][openpbs].

- Make sure the build dependencies `make` and [Go 1.21+][go-install] are
installed, `GOPATH` is set and `GOPATH/bin` is added to `PATH`.

For example, in Ubuntu this can be achieved via:

```sh
sudo apt update
sudo apt install make golang-go
export GOPATH=/your/desired/path
export PATH=$GOPATH/bin:$PATH
go version
```

- Clone the repository:

```sh
git clone https://github.com/ohsu-comp-bio/funnel.git
```

- Build Funnel:

```sh
cd funnel
make
```

- Test the installation by starting the Funnel server with:

```sh
funnel server run
```

If all works, Funnel should be ready for deployment on your HPC/HTC.

Alternatively, you can install Funnel via Homebrew:

```sh
brew tap ohsu-comp-bio/formula
brew install [email protected]
```

Source: [Funnel website](https://ohsu-comp-bio.github.io/funnel/)

### Slurm

For the use of Funnel with Slurm, make sure the following conditions are met:

1. The `funnel` binary must be placed in a server with access to Slurm.
2. A config file must be created and placed on the same server. [This
file][funnel-config-slurm] can be used as a starting point.
3. If we would like to deploy Funnel as a Systemd service,
[this file][funnel-config-slurm-service] can be used as a template. Set the
correct paths to the `funnel` binary and config file.

If successful, Funnel should be listening on port `8080`.

### OpenPBS

!!! warning "Under construction"
More info coming soon...
1 change: 1 addition & 0 deletions docs/guides/guide-admin/images/overview_protes.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading