Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@
* [Permission model](server-mode/permission-model.md)
* [Add documents from the CLI](server-mode/add-documents-from-the-cli.md)
* [Add entities from the CLI](server-mode/add-entities-from-the-cli.md)
* [Manage users from the CLI](server-mode/manage-users-from-the-cli.md)
* [Manage projects from the CLI](server-mode/manage-projects-from-the-cli.md)
* [User management](server-mode/user-management/README.md)
* [OAuth2](server-mode/user-management/oauth2.md)
* [HTML form](server-mode/user-management/form-auth.md)
Expand Down
41 changes: 41 additions & 0 deletions concepts/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ Running `datashare` with no arguments is equivalent to `datashare app start` —
| `api-key get` | Get the current user's API key |
| `api-key create` | Create an API key |
| `api-key delete` | Delete an API key |
| `user create` | Create a Datashare user |
| `user delete` | Delete a Datashare user and all their owned data |
| `project create` | Create a Datashare project |
| `project delete` | Delete a Datashare project (DB, ES index, queues, report map, artifacts) |
| `project grant` | Grant a role to a user on a project (replaces any existing role) |
| `project revoke` | Revoke all project roles for a user |

## `app start`

Expand Down Expand Up @@ -95,6 +101,41 @@ datashare extension install datashare-extension-nlp-opennlp
datashare extension delete datashare-extension-nlp-opennlp
```

## `user`

Create and delete Datashare users. See [Manage users from the CLI](../server-mode/manage-users-from-the-cli.md) for details.

```bash
# Create a user
datashare user create alice --email alice@example.org

# Create a user with a password and project access
datashare user create alice --email alice@example.org --password $PW --groups p1,p2

# Delete a user and all their owned data
datashare user delete alice --yes
```

## `project`

Create and delete projects, and manage project permissions. See [Manage projects from the CLI](../server-mode/manage-projects-from-the-cli.md) for details.

```bash
# Create a project
datashare project create my-project --label 'My Project'

# Delete a project (DB, ES index, queues, report map, artifacts)
datashare project delete my-project --yes

# Grant a role to a user (admin, editor, member or visitor)
datashare project grant my-project alice admin

# Revoke all project roles for a user
datashare project revoke my-project alice
```

All `user` and `project` subcommands accept `--json` (machine-readable output), `--no-input` (disable prompts) and idempotency flags (`--if-not-exists` / `--if-exists`) for scripting.

## Global flags

These flags can be placed before any subcommand and apply to all subcommands.
Expand Down
4 changes: 2 additions & 2 deletions concepts/running-modes.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ Datashare is shipped as a single executable, with all modes available. As previo

```sh
datashare app start --mode SERVER \
# Dummy session filter to creates ephemeral users
--authFilter org.icij.datashare.session.YesCookieAuthFilter \
# Dummy session authentication that creates ephemeral users
--auth yesCookie \
# Name of the default project for every user
--defaultProject local-datashare \
# URI of Elasticsearch
Expand Down
4 changes: 1 addition & 3 deletions server-mode/add-documents-from-the-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,7 @@

**This document assumes that you have installed Datashare** [**in server mode within Docker**](install-with-docker.md)**.**

In server [mode](../concepts/running-modes.md), it's important to understand that Datashare does not provide an interface to add documents. As there is no build-in roles and permission in Datashare's data model, we have no way to differentiate users to offer admin additional tools.

This is likely to be changed in the near future, but in the meantime, you can still add documents to Datashare using the command-line interface.
In server [mode](../concepts/running-modes.md), it's important to understand that Datashare does not provide a web interface to add documents. Documents are added using the command-line interface.

Here is a simple command to scan a directory and index its files:

Expand Down
4 changes: 1 addition & 3 deletions server-mode/add-entities-from-the-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,7 @@

**This document assumes that you have installed Datashare** [**in server mode within Docker**](install-with-docker.md) **and already** [**added documents to Datashare**](add-documents-from-the-cli.md)**.**

In server [mode](../concepts/running-modes.md), it's important to understand that Datashare does not provide an interface to add documents. As there is no build-in roles and permission in Datashare's data model, we have no way to differentiate users to offer admin additional tools.

This is likely to be changed in the near future, but in the meantime, you can extract named entities using the command-line interface.
In server [mode](../concepts/running-modes.md), it's important to understand that Datashare does not provide a web interface to add documents or extract entities. Named entities are extracted using the command-line interface.

Datashare has the ability to detect email addresses, name of people, organizations and locations. This process use a Natural Language Processing (NLP) pipeline called CORENLP. Once your documents have been indexed in Datashare, you can perform the named entities extraction in the same fashion as the previous CLI's [stages](../concepts/cli-stages.md):

Expand Down
124 changes: 124 additions & 0 deletions server-mode/manage-projects-from-the-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
---
description: >-
Create and delete projects, and grant or revoke project roles, with the
datashare project subcommands.
---

# Manage projects from the CLI

The `datashare project` subcommands let you manage projects and project permissions directly from the command line:

| Subcommand | Description |
| ---------- | ----------- |
| `project create` | Create a Datashare project |
| `project delete` | Delete a Datashare project (database, Elasticsearch index, queues, report map, artifacts) |
| `project grant` | Grant a role to a user on a project (replaces any existing role) |
| `project revoke` | Revoke all project roles for a user |

Run these commands with the same data-store settings as your server (`--dataSourceUrl`, `--elasticsearchAddress`, and `--redisAddress` if your deployment uses Redis) so they operate on the same projects.

## Create a project

```bash
datashare project create my-project
```

The project name can be passed as a positional argument (as above) or with `--name`. Useful options:

| Option | Description |
| ------ | ----------- |
| `--label` | Display label (default: name) |
| `--description` | Free-form description |
| `--source-path` | Filesystem source path (default: `/vault/<name>`) |
| `--source-url` | URL of the data origin |
| `--logo-url` | URL to the project logo |
| `--maintainer-name` | Maintainer display name |
| `--publisher-name` | Publisher display name |
| `--allow-from-mask` | IP mask for download access (default: `*.*.*.*`) |
| `--creator` | Grant `PROJECT_ADMIN` on the new project to this user (default: `defaultUserName` in LOCAL/EMBEDDED mode) |
| `--creation-date` | Creation timestamp (ISO-8601, e.g. `2026-05-15T10:00:00Z`). Defaults to now |
| `--no-index` | Skip Elasticsearch index creation |
| `--if-not-exists` | Idempotent: exit 0 if the project already exists |
| `--json` | Emit a JSON result on stdout |
| `--no-input` | Disable interactive prompts |

For example, to create a fully described project:

```bash
datashare project create my-project \
--label 'My Project' \
--description 'leak archive' \
--source-path /data/my-project \
--allow-from-mask 10.0.0.0
```

## Delete a project

```bash
datashare project delete my-project --yes
```

{% hint style="danger" %}
Deleting a project removes its database records, Elasticsearch index, queues, report map and artifacts. This cannot be undone. Without `--yes`, the command asks you to type the project name to confirm.
{% endhint %}

| Option | Description |
| ------ | ----------- |
| `--yes`, `-y` | Skip the typed-name confirmation |
| `--keep-index` | Do not drop the Elasticsearch index |
| `--if-exists` | Idempotent: exit 0 if the project doesn't exist |
| `--json` | Emit a JSON result on stdout |
| `--no-input` | Disable interactive prompts (implies `--yes`) |

## Grant a role

`project grant` assigns a role to a user on a project. It **replaces** any role the user already has on that project:

```bash
datashare project grant my-project alice admin
```

The arguments can be passed positionally (project, user, role) or with `--project`, `--user` and `--role`. The available roles map to the [permission model](permission-model.md) roles:

| CLI role | Permission model role |
| -------- | --------------------- |
| `admin` | `PROJECT_ADMIN` |
| `editor` | `PROJECT_EDITOR` |
| `member` | `PROJECT_MEMBER` |
| `visitor` | `PROJECT_VISITOR` |

Use `--if-not-exists` to exit 0 if the user already has the role, and `--json` / `--no-input` for scripting:

```bash
datashare project grant my-project alice editor --if-not-exists
datashare project grant my-project alice visitor --json --no-input
```

## Revoke roles

`project revoke` removes **all** roles a user has on a project (membership-level revoke):

```bash
datashare project revoke my-project alice
```

| Option | Description |
| ------ | ----------- |
| `--yes`, `-y` | Skip the y/N confirmation |
| `--if-exists` | Idempotent: exit 0 if the user has no role on the project |
| `--json` | Emit a JSON result on stdout |
| `--no-input` | Disable interactive prompts (implies `--yes`) |

## Scripting

All `project` subcommands are script-friendly: combine `--json` (machine-readable output), `--no-input` (no prompts) and the idempotency flags (`--if-not-exists` / `--if-exists`) to write provisioning scripts that can safely be re-run:

```bash
datashare project create my-project --no-index --if-not-exists --no-input --json
```

## Related pages

* [Permission model](permission-model.md) for what each role allows.
* [Manage users from the CLI](manage-users-from-the-cli.md) for creating the users you grant roles to.
* [Add documents from the CLI](add-documents-from-the-cli.md) for indexing documents into a project.
91 changes: 91 additions & 0 deletions server-mode/manage-users-from-the-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
description: Create and delete Datashare users with the datashare user subcommands.
---

# Manage users from the CLI

The `datashare user` subcommands let you provision and remove users directly from the command line, without writing SQL or Redis commands by hand:

| Subcommand | Description |
| ---------- | ----------- |
| `user create` | Create a Datashare user |
| `user delete` | Delete a Datashare user and all their owned data |

Run these commands with the same data-store settings as your server (`--dataSourceUrl`, and `--redisAddress` if your deployment uses Redis) so they operate on the same user inventory.

## Create a user

```bash
datashare user create alice --email alice@example.org
```

The login can be passed as a positional argument (as above) or with `--login`. Useful options:

| Option | Description |
| ------ | ----------- |
| `--email` | RFC 5322 email address |
| `--name` | Display name (default: login) |
| `--password` | Password, for the `local` provider |
| `--provider` | `local`, `oauth` or `external` (default: `local`) |
| `--groups` | Comma-separated project names the user can access |
| `--if-not-exists` | Idempotent: exit 0 if the user already exists |
| `--json` | Emit a JSON result on stdout |
| `--no-input` | Disable interactive prompts |

For example, to create a local user with a password and access to two projects:

```bash
datashare user create alice \
--email alice@example.org \
--password "$PW" \
--groups project1,project2
```

Or to provision a user authenticated by an external identity provider (see [OAuth2](user-management/oauth2.md)):

```bash
datashare user create alice --email alice@example.org --provider oauth --no-input
```

{% hint style="info" %}
Prefer passing the password through an environment variable (as in the example above) rather than typing it literally, so it doesn't end up in your shell history.
{% endhint %}

With Docker Compose, prefix the command with the web container's entrypoint:

```bash
docker compose exec datashare_web /entrypoint.sh \
user create alice \
--email alice@example.org \
--dataSourceUrl 'jdbc:postgresql://postgres/datashare?user=<username>&password=<password>'
```

## Delete a user

```bash
datashare user delete alice --yes
```

{% hint style="danger" %}
Deleting a user removes the user **and all their owned data**. This cannot be undone.
{% endhint %}

| Option | Description |
| ------ | ----------- |
| `--yes`, `-y` | Skip the confirmation prompt |
| `--if-exists` | Idempotent: exit 0 if the user doesn't exist |
| `--json` | Emit a JSON result on stdout |
| `--no-input` | Disable interactive prompts (forces `--yes`) |

## Scripting

All `user` subcommands are script-friendly: combine `--json` (machine-readable output), `--no-input` (no prompts) and the idempotency flags (`--if-not-exists` / `--if-exists`) to write provisioning scripts that can safely be re-run:

```bash
datashare user create alice --email alice@example.org --if-not-exists --no-input --json
```

## Related pages

* [User management](user-management/README.md) for choosing an authentication method and the user record format.
* [Manage projects from the CLI](manage-projects-from-the-cli.md) for granting users a role on a project.
24 changes: 21 additions & 3 deletions server-mode/permission-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,17 @@ enforce("bob", "default", "citrus-confidential" "PROJECT_MEMBER")
* `PROJECT_ADMIN`
* `PROJECT_EDITOR`
* `PROJECT_MEMBER`
* `PROJECT_VISITOR`

Hierarchy:

```
instance_admin > domain_admin > project_admin > project_editor > project_member
instance_admin > domain_admin > project_admin > project_editor > project_member > project_visitor
```

### Mapping Datashare features to permissions

* `PROJECT_VISITOR`: the most limited project role, below `PROJECT_MEMBER`
* `PROJECT_MEMBER`: read-only access inside a project
* `PROJECT_EDITOR`: project\_member + can change user-generated data (tags, recommendations, …)
* `PROJECT_ADMIN`: project\_editor + can manage the project (membership, settings)
Expand All @@ -62,7 +64,9 @@ Where policies live in production ? `CasbinRule` table&#x20;

How policies are updated:

* CLI : only to grant one INSTANCE\_ADMIN with grantAdminTask&#x20;
* CLI :
* project-level roles with `datashare project grant` and `datashare project revoke` (see [Manage projects from the CLI](manage-projects-from-the-cli.md))
* one INSTANCE\_ADMIN with grantAdminTask (see below)
* API endpoints :
* CRUD on all policies `/api/policies` (see the openapi schema).
* Current user policies `/api/me/permissions`
Expand All @@ -79,6 +83,20 @@ To secure policies changes we added guards so a user cannot grant or act on high

### How to enroll users in the permission model ?

#### Granting project roles from the CLI

The `datashare project grant` and `datashare project revoke` subcommands assign or remove project-level roles:

```bash
# Grant a role (admin, editor, member or visitor) — replaces any existing role
datashare project grant my-project alice admin

# Revoke all project roles for a user
datashare project revoke my-project alice
```

See [Manage projects from the CLI](manage-projects-from-the-cli.md) for all options.

#### Migration

To ensures a smooth and incremental enrollment with the new system, each auth filters has an auto-enrolling system happening after a user logs in. On post login, it adds the user as `PROJECT_MEMBER` to each project the user is already allowed to access, if they are not enrolled yet.&#x20;
Expand All @@ -92,7 +110,7 @@ datashare \
--grantAdmin foo \
--defaultProject local-datashare \
--redisAddress redis://redis:6379 \
--authUsersProvider org.icij.datashare.session.UsersInRedis \
--authUsersProvider redis \
--dataSourceUrl "jdbc:postgresql://postgres/datashare?user=admin&password=admin"
```

Expand Down
Loading