diff --git a/sips/sip-019/sip-019-token-metadata-update-notifications.md b/sips/sip-019/sip-019-token-metadata-update-notifications.md new file mode 100644 index 000000000..f76cf1047 --- /dev/null +++ b/sips/sip-019/sip-019-token-metadata-update-notifications.md @@ -0,0 +1,381 @@ +# Preamble + +SIP Number: 019 + +Title: Notifications for Token Metadata Updates + +Author: Rafael Cárdenas (rafael@hiro.so), Matthew Little (matt@hiro.so) + +Consideration: Technical + +Type: Standard + +Status: Ratified + +Created: 17 May 2022 + +License: GPL-3.0 + +Sign-off: Jude Nelson (jude@stacks.org), Aaron Blankstein (aaron@hiro.so), Marvin Janssen (http://github.com/MarvinJanssen) + +Layer: Traits + +# Abstract + +As the use of tokens (fungible and non-fungible) has grown in popularity, Stacks developers have +found novel ways to define and use metadata to describe them. This rich data is commonly cached and +indexed for future use in applications such as marketplaces, statistics aggregators, and developer +tools like the [Stacks Blockchain API](https://github.com/hirosystems/stacks-blockchain-api). + +Occasionally, however, this metadata needs to change for a number of reasons: artwork reveals, media +storage migrations, branding updates, etc. As of today, these changes do not have a standardized way +of being propagated through the network for indexers to refresh their cache, so the display of stale +metadata is a very common problem. + +This SIP aims to define a simple mechanism for developers to notify the Stacks network when metadata +for a token has changed, so interested parties can refresh their cache and display up-to-date +information in their applications. + +# Introduction + +Smart contracts that declare NFTs, FTs and SFTs conform to a standard set of traits used to describe +each token (see [SIP-009](../sip-009/sip-009-nft-standard.md), +[SIP-010](../sip-010/sip-010-fungible-token-standard.md) and +[SIP-013](https://github.com/stacksgov/sips/blob/main/sips/sip-013/sip-013-semi-fungible-token-standard.md)). +One of these traits is `get-token-uri`, which should return a URI string that resolves to a token's +metadata usually in the form of a JSON file. There is currently no defined structure for this data, +and it is not considered to be immutable. + +To illustrate a common use of `get-token-uri`, we'll look at the +[`SPSCWDV3RKV5ZRN1FQD84YE1NQFEDJ9R1F4DYQ11.newyorkcitycoin-token-v2`](https://explorer.stacks.co/txid/0x969192220b1c478ef9d18d1cd413d7c79fe02937a9b33af63d441bd5519d1715?chain=mainnet) +contract which declares the NewYorkCityCoin fungible token. + +At the time of writing, the value returned by this contract for `get-token-uri` is the string: +``` +"https://cdn.citycoins.co/metadata/newyorkcitycoin.json" +``` +When this URI is resolved, it returns a JSON file with the following metadata: +```json +{ + "name": "NewYorkCityCoin", + "description": "A CityCoin for New York City, ticker is NYC, Stack it to earn Stacks (STX)", + "image": "https://cdn.citycoins.co/logos/newyorkcitycoin.png" +} +``` +Even though the URI string is fixed, this file lives off-chain so it is conceivable that its +contents could change at any point in the future. Additionally, this contract includes a way for its +owners to change this URI via a `var-set` function call: + +```clarity +(define-data-var tokenUri (optional (string-utf8 256)) (some u"https://cdn.citycoins.co/metadata/newyorkcitycoin.json")) + +;; set token URI to new value, only accessible by Auth +(define-public (set-token-uri (newUri (optional (string-utf8 256)))) + (begin + (asserts! (is-authorized-auth) ERR_UNAUTHORIZED) + (ok (var-set tokenUri newUri)) + ) +) +``` + +This setup is very flexible for administrators, but it creates a complex problem for metadata +indexers which now need to figure out if (and when) they should re-index token contracts to avoid +displaying stale metadata in their applications. + + +## Metadata staleness + +Within the Stacks ecosystem, there are a number of applications that need to index token metadata +and struggle with specific challenges caused by changed metadata. For example: + +* An NFT marketplace, which needs to display a token's artwork for users to view. + * Presenting a token's icon correctly is difficult given that the `get-token-uri` on-chain + variable could change, the off-chain JSON file could change, and/or the image served by the URL + could change. +* A [blockchain API](https://github.com/hirosystems/stacks-blockchain-api), which needs to serve FT +metadata to return account balances correctly. + * Wallets require the on-chain decimals value in order to correctly send and receive tokens. + Critical balance draining is possible when this property is zero at contract launch but updated + later. + +For indexing, developers usually run and maintain a background process that listens for new token +contracts deployed to the blockchain so they can immediately call on their metadata to save the +results. This works for new contracts, but it is insufficient for old ones that may change their +metadata after it has been processed. + +To avoid staleness, some indexers resort to a cron-like periodic refresh of all tracked contracts, +but while this may work for individual applications, it does not provide a consistent experience for +Stacks users that may interact with different metadata-aware systems with different refresh periods. +This workaround also adds unnecessary network traffic and creates extra strain on public Stacks +nodes due to aggressively polling contract-read RPC endpoints. + +## Metadata update notifications + +To solve this problem reliably, contract administrators need a way to notify the network when they +have made changes to the metadata so any indexers may then perform a refresh just for that contract. + +The proposed mechanism for these notifications leverages the [`print` Clarity +language function](https://docs.stacks.co/write-smart-contracts/language-functions#print). When +used, its output is bundled inside an event of type `contract_event`: + +```json +{ + "type": "contract_event", + "contract_event": { + "contract_identifier": "", + "topic": "print", + "value": "" + } +} +``` + +This event is then attached to a transaction object and broadcasted when the same transaction is +included in a block or microblock. + +This SIP proposes a standard message structure (similar to a notification payload) that would be +used through `print`. Existing metadata indexers would receive this event through the [Stacks node +event-emitter +interface](https://github.com/stacks-network/stacks-blockchain/blob/master/docs/event-dispatcher.md#post-new_block), +parse and validate its contents, and refresh any contracts that were updated. `print` was also +selected for the following reasons: + +1. There is precedent for the use of `print` notifications in the Stacks ecosystem: the BNS +contract, for example, uses it to notify the network when a change to a name or its zonefile has +occurred. The PoX-2 contract for Stacks 2.1 will make heavy use of it to record stacking state +changes across addresses. This SIP aims to continue this trend. +1. For chain indexers, consuming it is practically free if they already process transactions. This +would enable, for example, a notification to be clearly displayed in the Stacks Explorer alongside +its transaction. +1. Adding a `print` notification to a function's Clarity code also serves as self-explanatory + documentation. +1. If there is a new notification use case in the future, a newer SIP can propose an additional + `print` structure and indexers would be quick to adopt these if they need to. See [Notification + structure reusability](#notification-structure-reusability). + +# Specification + +Notification messages for each token class are specified below. Token metadata update notifications +must be made via a contract call transaction to the [deployed reference +contract](https://explorer.stacks.co/txid/0xe92af2ea5c11e2e6fde4d31fd394de888070efff23bffad04465c549543abfa2?chain=mainnet) +or from a call to `print` within any other contract, including the token contract itself. + +## Fungible Tokens + +When a contract needs to notify the network that metadata has changed for a **Fungible Token**, it +shall call `print` with a tuple with the following structure: + +```clarity +{ notification: "token-metadata-update", payload: { token-class: "ft", contract-id: }} +``` + +| Key | Value | +|-----------------------|------------------------------------------------------------------------| +| `notification` | The string `"token-metadata-update"` | +| `payload.token-class` | The string `"ft"` | +| `payload.contract-id` | The contract id (principal) of the contract that declared the token | +| `payload.update-mode` | _[optional]_ Metadata update mode (see section below) | +| `payload.ttl` | _[optional]_ Time-to-live for `payload.update-mode: dynamic` | + +## Non-Fungible Tokens + +When a contract needs to notify the network that metadata has changed for a **Non-Fungible Token**, +it shall call `print` with a tuple with the following structure: + +```clarity +{ notification: "token-metadata-update", payload: { token-class: "nft", token-ids: (list u100, u101), contract-id: }} +``` + +| Key | Value | +|-----------------------|------------------------------------------------------------------------| +| `notification` | The string `"token-metadata-update"` | +| `payload.token-class` | The string `"nft"` | +| `payload.contract-id` | The contract id (principal) of the contract that declared the tokens | +| `payload.token-ids` | _[optional]_ A list with the uint token ids that need to be refreshed | +| `payload.update-mode` | _[optional]_ Metadata update mode (see section below) | +| `payload.ttl` | _[optional]_ Time-to-live for `payload.update-mode: dynamic` | + +If a notification does not contain a value for `payload.token-ids`, it means it is requesting an +update for all tokens. + +## Semi-Fungible Tokens + +When a contract needs to notify the network that metadata has changed for a **Semi-Fungible Token**, +it shall call `print` with a tuple with the following structure: + +```clarity +{ notification: "token-metadata-update", payload: { token-class: "sft", token-ids: (list u100, u101), contract-id: }} +``` + +| Key | Value | +|-----------------------|------------------------------------------------------------------------| +| `notification` | The string `"token-metadata-update"` | +| `payload.token-class` | The string `"sft"` | +| `payload.contract-id` | The contract id (principal) of the contract that declared the tokens | +| `payload.token-ids` | A list with the uint token ids that need to be refreshed | +| `payload.update-mode` | _[optional]_ Metadata update mode (see section below) | +| `payload.ttl` | _[optional]_ Time-to-live for `payload.update-mode: dynamic` | + +Notifications for SFTs must include a value for `payload.token-ids`. + +## Metadata update modes + +Applications may use tokens for very different purposes. Some of these could require none or very +few metadata updates ever (e.g. digital artwork that never changes except for reveals), while others +could need to alter it several times a day (e.g. NFTs for in-game items that are traded and modded +continuously). + +This use-case variety also affects how developers decide to host their metadata JSON files. For +example, they could choose to use IPFS for low-frequency updates and finality, versus Amazon S3 for +high-frequency off-chain updates. + +In order to allow creators and app developers to specify how token metadata should be treated by +indexers, notifications support an optional `payload.update-mode` key that may contain one of the +following values: + +* `standard`: The new metadata will be valid until the next notification comes. + + This is the default mode if none is specified. +* `frozen`: This token's metadata will never change again, ever. + + Indexers should ignore new notifications for this token, even if valid. +* `dynamic`: The new metadata is expected to change very quickly and many times in the future (even + off-chain). + + Indexers should not expect to receive explicit notifications for each of these changes and +should consider refreshing this token's metadata frequently. Token developers may suggest a +reasonable amount of time between refreshes by adding an estimated value (defined in seconds) to the +`payload.ttl` notification property. + +## Considerations for metadata indexers + +For a token metadata update notification to be considered valid by metadata indexers, it must meet +the following requirements: + +1. Its payload structure should be correct whether it is updating a [FT](#fungible-tokens), an + [NFT](#non-fungible-tokens) or an [SFT](#semi-fungible-tokens). +1. Either the `contract_identifier` field of the contract event must be equal to the + `payload.contract-id` (i.e., the event was produced by the contract that owns the metadata) or + the transaction's `tx-sender` principal should match the principal contained in the + notification's `payload.contract-id` (i.e., the STX address that sent the transaction which emits + the notification should match the owner of the token contract being updated). + +Notifications that do not meet these requirements must be ignored. + +### Other implications + +* Notifications can come at any point in time and are persistent in the Stacks blockchain. + * When performing a local sync to the chain tip, old notifications for old metadata updates could + not necessarily have a distinct effect in metadata responses when processed in the present. +* Multiple notifications for the same tokens will not necessarily correspond to multiple metadata + updates. + * Refreshing a token's metadata should be an idempotent operation. Repeated refreshes should not + create distinct records in the internal metadata database. + * To prevent slow performance and guard against any Denial of Service attack attempts, contract + call rate limiting should be implemented locally. +* Notifications can be delayed and out of order. + * A notification transaction's timestamp should not be considered to be the time when the token + metadata was actually updated. + +Given these constraints the notifications this SIP proposes should be taken as _hints_ to metadata +indexers. Metadata indexers are not obliged to follow them. + +## Notification structure reusability + +Even though establishing a generalized smart contract notification standard is out of scope for this +SIP, the proposed `print` message structure was designed for reusability by future SIPs that wish to +standardize other events. + +For example, developers could vary the `notification` and `payload` values to notify the network +when an NFT collection has been fully minted or another important milestone is reached. + +# Related work + +An alternative considered for token metadata update notifications is for them to be transmitted via +an off-chain notification service that indexer developers may subscribe to, such as: + +* An official mailing list +* A forum post +* An authoritative API service + +While these channels would have several advantages like being simpler to update, faster to +propagate, and easier to moderate, they have key disadvantages that make them inadequate for this +SIP's intended use: + +1. They introduce a third party dependency + * An off-chain notification service would most likely be maintained by centralized entities + unrelated to the Stacks ecosystem. As such, they could modify the channel, its reach, or its + rules at any time while affecting the entire network. + * Accepting third party solutions would invite developers to use many different hinting service + APIs and implementations, defeating the standardization purpose of this SIP. Moving + notifications to the blockchain establishes a canonical way to store and access them. + * Even if a decentralized off-chain third-party solution is found, it could still add a layer + of friction for developer adoption. +1. They are not future proof + * If the selected off-chain service changes at any point, a migration to another notification + channel will be much more difficult once the Stacks ecosystem has more token applications and + metadata indexers. + +# Backwards compatibility + +Developers who need to emit metadata update notifications for tokens declared in older contracts +(that were deployed before this notification standard was established) could do so by either calling +the contract described in [Reference Implementations](#reference-implementations) or by first +deploying a new separate contract containing a public function that prints this notification and +then calling it to have it emitted. + +# Activation + +This SIP will be activated when the following conditions are met: + +1. At least 10 unique contracts have had metadata updates triggered via contract-call transactions + that print the proposed notification payload. +1. At least 3 metadata indexers (like the Stacks Blockchain API or an NFT marketplace) start + listening for and reacting to the emitted notifications. + +If the Stacks blockchain reaches block height 170000 and the above has not happened, this SIP will +be considered rejected. + +# Reference implementations + +A [reference contract](./token-metadata-update-notify.clar) has been deployed to mainnet as +[`SP1H6HY2ZPSFPZF6HBNADAYKQ2FJN75GHVV95YZQ.token-metadata-update-notify`](https://explorer.stacks.co/txid/0xe92af2ea5c11e2e6fde4d31fd394de888070efff23bffad04465c549543abfa2?chain=mainnet). +It demonstrates how to send notifications for each token class and it is available for developers to +use for refreshing any existing or future token contract. If the SIP evolves to require a change to +this contract pre-activation, a new one will be deployed and noted here. + +```clarity +;; token-metadata-update-notify +;; +;; Use this contract to notify token metadata indexers that an NFT or FT needs its metadata +;; refreshed. + +(use-trait nft-trait 'SP2PABAF9FTAJYNFZH93XENAJ8FVY99RRM50D2JG9.nft-trait.nft-trait) +(use-trait ft-trait 'SP3FBR2AGK5H9QBDH3EEN6DF8EK8JY7RX8QJ5SVTE.sip-010-trait-ft-standard.sip-010-trait) + +;; Refresh the metadata for one or more NFTs from a specific contract. +(define-public (nft-metadata-update-notify (contract ) (token-ids (list 100 uint))) + (ok (print + { + notification: "token-metadata-update", + payload: { + contract-id: contract, + token-class: "nft", + token-ids: token-ids + } + }))) + +;; Refresh the metadata for a FT from a specific contract +(define-public (ft-metadata-update-notify (contract )) + (ok (print + { + notification: "token-metadata-update", + payload: { + contract-id: contract, + token-class: "ft" + } + }))) +``` + +The [Stacks Blockchain API](https://github.com/hirosystems/stacks-blockchain-api) will also add +compatibility for this standard while this SIP is being considered to demonstrate how indexers can +listen for and react to these notifications. diff --git a/sips/sip-019/token-metadata-update-notify.clar b/sips/sip-019/token-metadata-update-notify.clar new file mode 100644 index 000000000..5b2a0df23 --- /dev/null +++ b/sips/sip-019/token-metadata-update-notify.clar @@ -0,0 +1,30 @@ +;; token-metadata-update-notify +;; +;; Use this contract to notify token metadata indexers that an NFT or FT needs its metadata +;; refreshed. + +(use-trait nft-trait 'SP2PABAF9FTAJYNFZH93XENAJ8FVY99RRM50D2JG9.nft-trait.nft-trait) +(use-trait ft-trait 'SP3FBR2AGK5H9QBDH3EEN6DF8EK8JY7RX8QJ5SVTE.sip-010-trait-ft-standard.sip-010-trait) + +;; Refresh the metadata for one or more NFTs from a specific contract. +(define-public (nft-metadata-update-notify (contract ) (token-ids (list 100 uint))) + (ok (print + { + notification: "token-metadata-update", + payload: { + contract-id: contract, + token-class: "nft", + token-ids: token-ids + } + }))) + +;; Refresh the metadata for a FT from a specific contract +(define-public (ft-metadata-update-notify (contract )) + (ok (print + { + notification: "token-metadata-update", + payload: { + contract-id: contract, + token-class: "ft" + } + })))