Subgraph Best Practices 5 and 6 (#825)

* timeseries included * updated subgraph best practices ... and added links to bottom of each best practice * best practices link changes * formatting * Update website/pages/en/cookbook/avoid-eth-calls.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Fixing links * Update website/pages/en/cookbook/derivedfrom.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/derivedfrom.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/timeseries.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/timeseries.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Update website/pages/en/cookbook/grafting-hotfix.mdx Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> * Ida's feedback * Update website/pages/en/cookbook/grafting-hotfix.mdx * pnpm check * Fix build by reproducing `en/cookbook` file structure in other languages --------- Co-authored-by: Michael Macaulay <56690114+MichaelMacaulay@users.noreply.github.com> Co-authored-by: Idalith <126833353+idalithb@users.noreply.github.com> Co-authored-by: benface <benoit.rouleau@icloud.com>
graphprotocol · Dec 12, 2024 · e674dea · e674dea
1 parent 1420c3c
commit e674dea
Show file tree

Hide file tree

Showing 100 changed files with 9,230 additions and 6,261 deletions.
diff --git a/website/pages/ar/cookbook/base-testnet.mdx b/website/pages/ar/cookbook/base-testnet.mdx
diff --git a/website/pages/ar/cookbook/grafting-hotfix.mdx b/website/pages/ar/cookbook/grafting-hotfix.mdx
@@ -0,0 +1,186 @@
+---
+Subgraph Best Practice 6 - Use Grafting for Quick Hotfix Deployment
+---
+
+## TLDR
+
+Grafting is a powerful feature in subgraph development that allows you to build and deploy new subgraphs while reusing the indexed data from existing ones.
+
+### Overview
+
+This feature enables quick deployment of hotfixes for critical issues, eliminating the need to re-index the entire subgraph from scratch. By preserving historical data, grafting minimizes downtime and ensures continuity in data services.
+
+## Benefits of Grafting for Hotfixes
+
+1. **Rapid Deployment**
+
+   - **Minimize Downtime**: When a subgraph encounters a critical error and stops indexing, grafting enables you to deploy a fix immediately without waiting for re-indexing.
+   - **Immediate Recovery**: The new subgraph continues from the last indexed block, ensuring that data services remain uninterrupted.
+
+2. **Data Preservation**
+
+   - **Reuse Historical Data**: Grafting copies the existing data from the base subgraph, so you don’t lose valuable historical records.
+   - **Consistency**: Maintains data continuity, which is crucial for applications relying on consistent historical data.
+
+3. **Efficiency**
+   - **Save Time and Resources**: Avoids the computational overhead of re-indexing large datasets.
+   - **Focus on Fixes**: Allows developers to concentrate on resolving issues rather than managing data recovery.
+
+## Best Practices When Using Grafting for Hotfixes
+
+1. **Initial Deployment Without Grafting**
+
+   - **Start Clean**: Always deploy your initial subgraph without grafting to ensure that it’s stable and functions as expected.
+   - **Test Thoroughly**: Validate the subgraph’s performance to minimize the need for future hotfixes.
+
+2. **Implementing the Hotfix with Grafting**
+
+   - **Identify the Issue**: When a critical error occurs, determine the block number of the last successfully indexed event.
+   - **Create a New Subgraph**: Develop a new subgraph that includes the hotfix.
+   - **Configure Grafting**: Use grafting to copy data up to the identified block number from the failed subgraph.
+   - **Deploy Quickly**: Publish the grafted subgraph to restore service as soon as possible.
+
+3. **Post-Hotfix Actions**
+
+   - **Monitor Performance**: Ensure the grafted subgraph is indexing correctly and the hotfix resolves the issue.
+   - **Republish Without Grafting**: Once stable, deploy a new version of the subgraph without grafting for long-term maintenance.
+     > Note: Relying on grafting indefinitely is not recommended as it can complicate future updates and maintenance.
+   - **Update References**: Redirect any services or applications to use the new, non-grafted subgraph.
+
+4. **Important Considerations**
+   - **Careful Block Selection**: Choose the graft block number carefully to prevent data loss.
+   - **Tip**: Use the block number of the last correctly processed event.
+   - **Use Deployment ID**: Ensure you reference the Deployment ID of the base subgraph, not the Subgraph ID.
+   - **Note**: The Deployment ID is the unique identifier for a specific subgraph deployment.
+   - **Feature Declaration**: Remember to declare grafting in the subgraph manifest under features.
+
+## Example: Deploying a Hotfix with Grafting
+
+Suppose you have a subgraph tracking a smart contract that has stopped indexing due to a critical error. Here’s how you can use grafting to deploy a hotfix.
+
+1. **Failed Subgraph Manifest (subgraph.yaml)**
+
+   ```yaml
+   specVersion: 1.0.0
+   schema:
+     file: ./schema.graphql
+   dataSources:
+     - kind: ethereum/contract
+       name: OldSmartContract
+       network: sepolia
+       source:
+         address: '0xOldContractAddress'
+         abi: Lock
+         startBlock: 5000000
+       mapping:
+         kind: ethereum/events
+         apiVersion: 0.0.7
+         language: wasm/assemblyscript
+         entities:
+           - Withdrawal
+         abis:
+           - name: Lock
+             file: ./abis/OldLock.json
+         eventHandlers:
+           - event: Withdrawal(uint256,uint256)
+             handler: handleOldWithdrawal
+         file: ./src/old-lock.ts
+   ```
+
+2. **New Grafted Subgraph Manifest (subgraph.yaml)**
+   ```yaml
+   specVersion: 1.0.0
+   schema:
+     file: ./schema.graphql
+   dataSources:
+     - kind: ethereum/contract
+       name: NewSmartContract
+       network: sepolia
+       source:
+         address: '0xNewContractAddress'
+         abi: Lock
+         startBlock: 6000001 # Block after the last indexed block
+       mapping:
+         kind: ethereum/events
+         apiVersion: 0.0.7
+         language: wasm/assemblyscript
+         entities:
+           - Withdrawal
+         abis:
+           - name: Lock
+             file: ./abis/Lock.json
+         eventHandlers:
+           - event: Withdrawal(uint256,uint256)
+             handler: handleWithdrawal
+         file: ./src/lock.ts
+   features:
+     - grafting
+   graft:
+     base: QmBaseDeploymentID # Deployment ID of the failed subgraph
+     block: 6000000 # Last successfully indexed block
+   ```
+
+**Explanation:**
+
+- **Data Source Update**: The new subgraph points to 0xNewContractAddress, which may be a fixed version of the smart contract.
+- **Start Block**: Set to one block after the last successfully indexed block to avoid reprocessing the error.
+- **Grafting Configuration**:
+  - **base**: Deployment ID of the failed subgraph.
+  - **block**: Block number where grafting should begin.
+
+3. **Deployment Steps**
+
+   - **Update the Code**: Implement the hotfix in your mapping scripts (e.g., handleWithdrawal).
+   - **Adjust the Manifest**: As shown above, update the `subgraph.yaml` with grafting configurations.
+   - **Deploy the Subgraph**:
+     - Authenticate with the Graph CLI.
+     - Deploy the new subgraph using `graph deploy`.
+
+4. **Post-Deployment**
+   - **Verify Indexing**: Check that the subgraph is indexing correctly from the graft point.
+   - **Monitor Data**: Ensure that new data is being captured and the hotfix is effective.
+   - **Plan for Republish**: Schedule the deployment of a non-grafted version for long-term stability.
+
+## Warnings and Cautions
+
+While grafting is a powerful tool for deploying hotfixes quickly, there are specific scenarios where it should be avoided to maintain data integrity and ensure optimal performance.
+
+- **Incompatible Schema Changes**: If your hotfix requires altering the type of existing fields or removing fields from your schema, grafting is not suitable. Grafting expects the new subgraph’s schema to be compatible with the base subgraph’s schema. Incompatible changes can lead to data inconsistencies and errors because the existing data won’t align with the new schema.
+- **Significant Mapping Logic Overhauls**: When the hotfix involves substantial modifications to your mapping logic—such as changing how events are processed or altering handler functions—grafting may not function correctly. The new logic might not be compatible with the data processed under the old logic, leading to incorrect data or failed indexing.
+- **Deployments to The Graph Network**: Grafting is not recommended for subgraphs intended for The Graph’s decentralized network (mainnet). It can complicate indexing and may not be fully supported by all Indexers, potentially causing unexpected behavior or increased costs. For mainnet deployments, it’s safer to re-index the subgraph from scratch to ensure full compatibility and reliability.
+
+### Risk Management
+
+- **Data Integrity**: Incorrect block numbers can lead to data loss or duplication.
+- **Testing**: Always test grafting in a development environment before deploying to production.
+
+## Conclusion
+
+Grafting is an effective strategy for deploying hotfixes in subgraph development, enabling you to:
+
+- **Quickly Recover** from critical errors without re-indexing.
+- **Preserve Historical Data**, maintaining continuity for applications and users.
+- **Ensure Service Availability** by minimizing downtime during critical fixes.
+
+However, it’s important to use grafting judiciously and follow best practices to mitigate risks. After stabilizing your subgraph with the hotfix, plan to deploy a non-grafted version to ensure long-term maintainability.
+
+## Additional Resources
+
+- **[Grafting Documentation](/cookbook/grafting/)**: Replace a Contract and Keep its History With Grafting
+- **[Understanding Deployment IDs](/querying/querying-by-subgraph-id-vs-deployment-id/)**: Learn the difference between Deployment ID and Subgraph ID.
+
+By incorporating grafting into your subgraph development workflow, you can enhance your ability to respond to issues swiftly, ensuring that your data services remain robust and reliable.
+
+## Subgraph Best Practices 1-6
+
+1.  [Improve Query Speed with Subgraph Pruning](/cookbook/pruning/)
+
+2.  [Improve Indexing and Query Responsiveness by Using @derivedFrom](/cookbook/derivedfrom/)
+
+3.  [Improve Indexing and Query Performance by Using Immutable Entities and Bytes as IDs](/cookbook/immutable-entities-bytes-as-ids/)
+
+4.  [Improve Indexing Speed by Avoiding `eth_calls`](/cookbook/avoid-eth-calls/)
+
+5.  [Simplify and Optimize with Timeseries and Aggregations](/cookbook/timeseries/)
+
+6.  [Use Grafting for Quick Hotfix Deployment](/cookbook/grafting-hotfix/)