Purging old records from explorer database

[SunnyAjin]

What would you like to be added?

Now explorer database is saving each transaction that is committed on to the ledger making replica copies of ledger. When number of transactions increases there is a chance of outage of postgres database. In order to address this we would keep only the recent data in database and provide a query back mechanism to read back from the blockchain for any of the purged data.

Implementation

• Based on BLOCKCOUNT
• Based on TIME

We can set the purgeMode attribute in config.json file and according to the purge mode, it enables purging from database.
If it is based on blockcount (mention the number of blocks as blockCount value in config.json) ,it will retain those many blocks in the database and the rest of the records would be purged.
If it is based on time (mention the number of days as daysToPurge value in config.json) then it will keep those many days of records in the database according to the daystoPurge provided in the config file.
If the purgeMode attribute is set to "NONE", then purging won't happen and it will retain the complete data as of ledger.

All the audit records of deletion is saved to explorer_audit_table which is further used for the sync process.

Why is this needed?

Explorer creates duplicate copy of complete ledger data in postgres which can be of larger number and can lead to database storage outage.