Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 96 additions & 32 deletions docs/HyperIndex/Advanced/effect-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,10 @@ The first argument is an options object that describes the effect:

- `name` (required) - the name of the effect used for debugging and logging
- `input` (required) - the input type of the effect
- `output` (required) - the output type of the effect
- `output` (required) - the output type of the effect (not required for `unorderedAfterCommit` and `orderedAfterCommit` modes which return `void`)
- `rateLimit` (required) - the maximum calls allowed per timeframe, or `false` to disable
- `cache` (optional) - save effect results in the database to prevent duplicate calls (Starting from `envio@2.26.0`)
- `mode` (optional) - the execution intent of the effect. Defaults to `speculative`. See [Execution Modes](#execution-modes).

The second argument is a function that will be called with the effect's input.

Expand All @@ -64,10 +65,15 @@ After defining an effect, you can use `context.effect` to call it from your hand
The `context.effect` function accepts an effect as the first argument and the effect's input as the second argument:

```typescript
ERC20.Transfer.handler(async ({ event, context }) => {
const metadata = await context.effect(getMetadata, event.params.from);
// Process the event with the metadata
});
import { indexer } from "envio";

indexer.onEvent(
{ contract: "ERC20", event: "Transfer" },
async ({ event, context }) => {
const metadata = await context.effect(getMetadata, event.params.from);
// Process the event with the metadata
},
);
```

### Reading On-Chain State (eth_call)
Expand Down Expand Up @@ -213,6 +219,68 @@ export const getBalance = createEffect(
Watch the following video to learn more about createEffect and other updates introduced in [v2.32.0](https://github.com/enviodev/hyperindex/releases/tag/v2.32.0).
<Video id="yvUVzV1ifig" title="Envio v2.32.0" />

### Execution Modes

The `mode` option controls **when** the effect runs, **how** it's ordered relative to other effect calls in the batch, and what `context.effect` returns. One constructor, one call verb — behavior shifts on `mode`.

| Mode | Input | Order | Timing | Returns | Use case |
| --- | --- | --- | --- | --- | --- |
| `speculative` *(default)* | maybe-wrong | unordered | preload (parallel) | value | RPC reads, fetch, IPFS — the default behavior |
| `unordered` | correct | unordered | inline (parallel within batch) | value | parallel reads/writes with verified input where order doesn't matter |
| `ordered` | correct | in-order | inline (sequential pass) | value | low-latency writes where order matters (e.g. depends on prior entity writes in the same batch) |
| `unorderedAfterCommit` | correct | unordered | after DB commit | `void` | high-throughput sends, fire-and-forget webhooks, partitioned Kafka |
| `orderedAfterCommit` | correct | in-order | after DB commit | `void` | sends where order matters (Telegram, ordered streams) |

The two `*AfterCommit` modes are the right choice for **outbound messaging** — Redis, Kafka, RabbitMQ, SNS/SQS, Telegram, Discord, Slack, webhooks. They fire only after the batch's database transaction commits, so a downstream consumer never sees a message for a state that didn't actually persist. They also return `void`, so you don't need to `await` the call from your handler — the runtime takes care of dispatching after commit.

```typescript
import { createEffect, indexer, S } from "envio";

const notifyLargeSwap = createEffect(
{
name: "notifyLargeSwap",
input: { usd: S.bigint, blockNumber: S.number },
mode: "orderedAfterCommit",
},
async ({ input }) => {
const text = `Large swap: $${input.usd.toLocaleString()} in block ${input.blockNumber}`;
await fetch(`https://api.telegram.org/bot${process.env.ENVIO_TG_BOT_TOKEN}/sendMessage`, {
method: "POST",
body: JSON.stringify({
chat_id: process.env.ENVIO_TG_CHAT_ID,
text,
}),
});
}
);

indexer.onEvent(
{ contract: "Pool", event: "Swap" },
async ({ event, context }) => {
const usd = event.params.amount;
if (usd > 1_000_000n) {
context.effect(notifyLargeSwap, {
usd,
blockNumber: event.block.number,
});
// no await — orderedAfterCommit returns void; runtime fires it after the batch's DB commit
}
},
);
```

#### When to pick which mode

- **`speculative`** — your call is idempotent, the input may not be final (preloading runs handlers ahead of confirmed state), and you only need a value back. RPC `eth_call`s, `fetch` of IPFS metadata, and any read where retries are cheap belong here.
- **`unordered`** — you've already verified the input (e.g. derived it from a committed entity) and the work is parallel-safe. Use this for batched reads/writes that don't depend on each other.
- **`ordered`** — the effect's correctness depends on something written earlier in the same batch. The runtime will run these calls sequentially in handler order.
- **`unorderedAfterCommit`** — fire-and-forget sends where ordering doesn't matter: partitioned Kafka topics, Redis Streams keyed by tx hash, generic webhooks. Highest throughput, never sends for state that wasn't committed.
- **`orderedAfterCommit`** — sends to a single ordered destination: a Telegram chat, a Slack channel, a single Kafka partition. The runtime preserves handler order across the batch.

For outbound messaging, `*AfterCommit` is the safe default. If you've measured commit latency as the bottleneck and your downstream consumer is idempotent, you can also use the inline `unordered` / `ordered` modes for **lower latency** sends — same parallel/sequential semantics, but they fire before the DB commit and return a value. The tradeoff is weaker delivery: a failed batch can still produce a delivered message, so retries on the next run are duplicates rather than first-time sends.

This is what makes the Effect API a good fit for **streams and chat bots** — you describe what to send and where, the runtime handles batching, ordering, and delivery semantics. See the [Streams](/docs/HyperIndex/streams) and [Chat Bots](/docs/HyperIndex/chatbots) guides for end-to-end examples per provider.

### Sending Notifications (Webhooks)

You can use the Effect API to send push notifications or webhook calls when specific events occur. This is useful for alerting systems, Discord/Slack bots, or triggering downstream workflows.
Expand All @@ -227,48 +295,44 @@ export const sendWebhook = createEffect(
event: S.string,
data: S.string,
},
output: S.boolean,
rateLimit: {
calls: 10,
per: "second",
},
// Don't cache webhook calls - we want them to fire every time
cache: false,
mode: "unorderedAfterCommit",
},
async ({ input, context }) => {
try {
await fetch("https://your-webhook-url.com/notify", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ event: input.event, data: input.data }),
});
return true;
} catch (error) {
context.log.error(`Webhook failed: ${error}`);
return false;
}
await fetch("https://your-webhook-url.com/notify", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ event: input.event, data: input.data }),
});
}
);
```

Then call it from your handler:

```typescript
MyContract.LargeTransfer.handler(async ({ event, context }) => {
await context.effect(sendWebhook, {
event: "large_transfer",
data: JSON.stringify({
from: event.params.from,
to: event.params.to,
amount: event.params.value.toString(),
}),
});
});
import { indexer } from "envio";

indexer.onEvent(
{ contract: "MyContract", event: "LargeTransfer" },
async ({ event, context }) => {
context.effect(sendWebhook, {
event: "large_transfer",
data: JSON.stringify({
from: event.params.from,
to: event.params.to,
amount: event.params.value.toString(),
}),
});
// no await — `unorderedAfterCommit` returns void and dispatches after the DB commit
},
);
```

:::warning
Webhook effects will fire on every indexer re-run unless you set `cache: true`. If you cache them, the webhook will only fire once per unique input. Consider which behavior is appropriate for your use case.
:::
Because the effect runs **after** the batch's DB commit, the webhook will only fire for state that actually persisted — a reorg or a failed batch never produces a phantom notification. For send-only effects you typically don't need `cache: true`; the mode itself prevents duplicate sends within a successful run.

### Migrate from Experimental

Expand Down
116 changes: 116 additions & 0 deletions docs/HyperIndex/Chatbots/discord.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
---
id: chatbots-discord
title: Discord
sidebar_label: Discord
slug: /chatbots/discord
description: Send Discord messages from HyperIndex handlers using the Effect API.
---

Post messages to a Discord channel using either an [incoming webhook](https://support.discord.com/hc/en-us/articles/228383668) (simplest) or a bot token. Both work with raw `fetch`.

### Channel setup

1. Channel **Settings → Integrations → Webhooks → New Webhook**.
2. Copy the webhook URL.
3. Set `ENVIO_DISCORD_WEBHOOK_URL` in your `.env`.

### Define the effect

Webhook URL is baked into the effect. The embed is built inside the effect body — `input` is just raw event data.

```typescript title="src/effects/discord.ts"
import { createEffect, S } from "envio";

const formatUnits = (value: bigint, decimals = 18) => {
const base = 10n ** BigInt(decimals);
const whole = value / base;
const frac = value % base;
if (frac === 0n) return whole.toString();
return `${whole}.${frac.toString().padStart(decimals, "0").replace(/0+$/, "")}`;
};

export const notifyTransfer = createEffect(
{
name: "notifyTransfer",
input: {
from: S.string,
to: S.string,
value: S.bigint,
contract: S.string,
txHash: S.string,
},
rateLimit: { calls: 5, per: "second" }, // Discord webhook limit
mode: "orderedAfterCommit",
},
async ({ input, context }) => {
const res = await fetch(process.env.ENVIO_DISCORD_WEBHOOK_URL!, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
content: "**New RETH Transfer Event**",
embeds: [
{
title: `${formatUnits(input.value)} RETH transferred`,
url: `https://etherscan.io/tx/${input.txHash}`,
fields: [
{ name: "From", value: input.from, inline: true },
{ name: "To", value: input.to, inline: true },
{ name: "Contract", value: input.contract },
],
},
],
allowed_mentions: { parse: [] },
}),
});
if (!res.ok) {
context.log.error(`Discord failed: ${res.status} ${await res.text()}`);
throw new Error(`Discord ${res.status}`);
}
}
);
```

### Call it from a handler

The rindexer template…

```yaml
chat:
discord:
- messages:
- event_name: Transfer
filter_expression: "value >= 10 && value <= 2000000000000000000"
template_inline: |
*New RETH Transfer Event*
from: {{from}}
to: {{to}}
amount: {{format_value(value, 18)}}
contract: {{transaction_information.address}}
[etherscan](https://etherscan.io/tx/{{transaction_information.transaction_hash}})
```

…becomes:

```typescript title="src/handlers/RocketPoolETH.ts"
import { indexer } from "envio";
import { notifyTransfer } from "../effects/discord";

indexer.onEvent(
{ contract: "RocketPoolETH", event: "Transfer" },
async ({ event, context }) => {
const { from, to, value } = event.params;

if (value < 10n || value > 2_000_000_000_000_000_000n) return;

context.effect(notifyTransfer, {
from,
to,
value,
contract: event.srcAddress,
txHash: event.transaction.hash,
});
},
);
```

Discord embeds give you richer formatting than plain text — fields, colors, thumbnails — without any extra dependency. If you'd rather keep things simple, replace `embeds` with a markdown `content` string. For lower latency, switch the effect to `mode: "ordered"` (or `"unordered"` if order doesn't matter) — Discord is rate-limited so the gain is small, but it skips the wait for the DB commit.
46 changes: 46 additions & 0 deletions docs/HyperIndex/Chatbots/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
id: chatbots
title: Chat Bots
sidebar_label: Overview
slug: /chatbots
description: Send on-chain notifications from HyperIndex to Telegram, Discord, Slack, Twilio, PagerDuty, or OpsGenie.
---

Chat bots in HyperIndex are just [Effects](/docs/HyperIndex/effect-api) running in `orderedAfterCommit` mode that call a chat platform's API. There is no separate templating language and no separate config file — you build your message string with template literals and decide when to send it with normal `if` statements.

### Picking a mode

Chat platforms display messages in arrival order to humans, so you almost always want **ordered** delivery. The runtime preserves the order of `context.effect(...)` calls across the entire batch, then dispatches them after the DB commit.

| Mode | When to use |
| --- | --- |
| `orderedAfterCommit` *(default choice for chat bots)* | Single ordered destination (Telegram chat, Slack channel). Won't message users about state that was rolled back. |
| `unorderedAfterCommit` | Independent alerts where order doesn't matter (one alert per swap, fan-out to multiple channels). Higher throughput. |
| `ordered` *(lower latency)* | Need the message out as fast as possible and your operators tolerate the rare duplicate on a failed batch. Inline, sequential, returns a value. |
| `unordered` *(lower latency)* | Same speed/safety tradeoff as `ordered`, but parallel dispatch. Use when alerts are fully independent. |

Use the after-commit modes by default. Switch to `ordered` / `unordered` only when you measure commit latency as the bottleneck — chat platforms are rate-limited anyway, so the gain is usually small.

### Comparison with rindexer

| rindexer | HyperIndex |
| --- | --- |
| `chat:` block in YAML, one entry per provider | A `createEffect({ mode: "orderedAfterCommit" })` per destination |
| `filter_expression: "value >= 10 && from = '0x…'"` | A regular TypeScript `if` |
| `template_inline` Mustache template | Template literal (full TypeScript expressions inside `${...}`) |
| 10-block max range hard-coded | Use the rate-limit option to tune throughput |

### Supported platforms

- [Telegram](/docs/HyperIndex/chatbots/telegram)
- [Discord](/docs/HyperIndex/chatbots/discord)
- [Slack](/docs/HyperIndex/chatbots/slack)
- [Twilio (SMS)](/docs/HyperIndex/chatbots/twilio)
- [PagerDuty](/docs/HyperIndex/chatbots/pagerduty)
- [OpsGenie](/docs/HyperIndex/chatbots/opsgenie)

### Pattern

Each effect bakes in its own static config (token, channel, rate limit, message template) and accepts only the per-event values that vary in `input`. That keeps call sites tiny, makes deduplication effective, and means switching destinations later is a one-line change.

The example pages below all use a small `formatUnits` helper inside the effect body to render bigints — equivalent to rindexer's `{{format_value(value, 18)}}`.
Loading