feat: add data cache for UTXO and RGB++ asset cells

[ahonn]

Purpose

Implement a data cache for UTXO and RGB++ asset cells. API requests directly return cached data, which is updated using an asynchronous update mechanism (Job queue).

Changes

• Cache the RGB++ Cells for the BTC Address and update it asynchronously
  • /rgbpp/v1/address/:address/assets with same parameters
    • no cache: 18s
    • with cache: 8s
  • also optimize /rgbpp/v1/assets routes by using batch request, speed up response time
• Cache the UTXO data for the BTC Address and update it asynchronously
  • /bitcoin/v1/address/:address/upspent with same parameters
    • no cache: 8s
    • with cache: ~= 50ms
• Trigger the update of the RGB++ Cells cache when the UTXO is updated
• UTXO update interval increases exponentially
• Update UTXO data immediately when getting a new request
• Add UTXO sync job expiration policy (336 hours by default) and data cache expiration time
• Add new env vars for enable/disable data cache: UTXO_SYNC_DATA_CACHE_ENABLE, RGBPP_COLLECT_DATA_CACHE_ENABLE
• Add error handle and sentry capture
• Add unit test cases for new services

New Envs

/**
   * UTXO sync data cache enable flag, used to cache the UTXO sync data
   * enable by default
   */
  UTXO_SYNC_DATA_CACHE_ENABLE: z
    .enum(['true', 'false'])
    .default('true')
    .transform((value) => value === 'true'),
  /**
   * UTXO sync repeat base duration, used to set the UTXO sync repeat interval
   * repeat job start interval is 10 seconds by default
   */
  UTXO_SYNC_REPEAT_BASE_DURATION: z.coerce.number().default(10 * 1000),
  /**
   * UTXO sync repeat max duration, used to maximum the UTXO sync repeat interval
   * 1 hour by default
   */
  UTXO_SYNC_REPEAT_MAX_DURATION: z.coerce.number().default(60 * 60 * 1000),
  /**
   * UTXO sync repeat expired duration, used to remove the expired UTXO sync job
   * 336 hours by default
   */
  UTXO_SYNC_REPEAT_EXPRIED_DURATION: z.coerce.number().default(336 * 60 * 60 * 1000),
  /**
   * UTXO sync data cache expire duration, used to cache the UTXO sync data
   * 30 minutes by default
   */
  UTXO_SYNC_DATA_CACHE_EXPIRE: z.coerce.number().default(30 * 60 * 1000),

  /**
   * RGB++ collect data cache enable flag, used to cache the RGB++ collect data
   * enable by default
   */
  RGBPP_COLLECT_DATA_CACHE_ENABLE: z
    .enum(['true', 'false'])
    .default('true')
    .transform((value) => value === 'true'),
  /**
   * RGB++ collect data cache expire duration, used to cache the RGB++ collect data
   */
  RGBPP_COLLECT_DATA_CACHE_EXPIRE: z.coerce.number().default(30 * 60 * 1000),

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
btc-assets-api	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 16, 2024 4:46am

[Dawn-githup]

[Flouse]

• link feat(service): add no_cache params to btc/rgbpp service api rgbpp-sdk#181