How do I migrate docs after changing schema or enabling localization?

[jacobsfletch]

• How do I resave all posts programmatically?
• How do I migrate docs after a schema change?
• How do I migrate data after enabling or disabling localization?

These are common questions and fortunately pretty easy to solve using a migration script. All we have to do is spin up Payload, find all records within a given collection, and then immediately update each. If needed, we can also modify the data before it gets updated.

1. First, add a scripts/resaveCollection.js file to your codebase:

   const payload = require('payload');
   
   // In Payload v3, use these imports instead:
   // import config from '@payload-config'
   // import { getPayload } from 'payload'
   
   require('dotenv').config();
   
   const resaveCollection = async () => {
     // In Payload v3, call `getPayload` here:
     // const payload = await getPayload({ config })
   
     await payload.init({
       secret: process.env.PAYLOAD_SECRET,
       mongoURL: process.env.MONGO_URL,
       local: true,
     });
   
     const args = process.argv.slice(2); // nodejs command line args are an array that begin at the third item
     
     const [
       collectionSlug,
       overrides
     ] = args || [];
   
     const results = await payload.find({
       collection: collectionSlug,
       depth: 0,
       limit: 700,
     });
   
     try {
       await Promise.all(results.docs.map(async (result) => {
         const { id } = result;
   
         if (collectionSlug) {
           try {
             await payload.update({
               collection: collectionSlug,
               id,
               data: {
                 ...overrides || {}
                 // Adjust the data here based on your specific schema
               },
             })
   
             console.log(`Document in '${collectionSlug}' with id '${id}' updated successfully`);
           } catch (e) {
             payload.logger.error(`Document in '${collectionSlug}' with id '${id}' failed to update`);
             payload.logger.error(e);
           }
         } else {
           console.log(`No document found in '${collectionSlug}' with id '${id}'`);
         }
       }));
     } catch (e) {
       payload.logger.error('Something went wrong.');
       payload.logger.error(e);
     }
   
     console.log('Complete');
     process.exit(0);
   };
   
   resaveCollection();

2. Modify the above function based on your specific schema change.

3. To run it, send the slug of the collection along with optional overrides for node to execute:

   node scripts/resaveCollection pages

^ Your specific file path may different. Update this based on your specifications. Also note that Payload may need to know the location of your config, see this page for more detail.

[jacobsfletch]

If you're here needing to regenerate media after changing image sizes, check out this discussion: #1834

[revnelson]

To clear things out in my head, and in case it helps anyone else, could somebody confirm the modified code would be the correct way to migrate a nested field objectId to objId as an example? For clarity in this example the collection field config would be:

Initial:

{ name: 'import', type: 'group', fields: [{name: 'objectId', type: 'string'}] }

Interim:

{ name: 'import', type: 'group', fields: [{name: 'objectId', type: 'string'}, {name: 'objId', type: 'string'}] }

Final:

{ name: 'import', type: 'group', fields: [{name: 'objId', type: 'string'}] }

During the interim stage of the schema, run the following script:

const payload = require('payload');

require('dotenv').config();

const resaveCollection = async () => {
  await payload.init({
    secret: process.env.PAYLOAD_SECRET,
    mongoURL: process.env.MONGO_URL,
    local: true,
  });

  const args = process.argv.slice(2); // nodejs command line args are an array that begin at the third item
  const [
    collectionSlug,
    overrides
  ] = args || [];

  const results = await payload.find({
    collection: collectionSlug,
    depth: 0,
    limit: 700,
  });

  try {
    await Promise.all(results.docs.map(async (result) => {
      const { id } = result;

      if (collectionSlug) {
        try {
          await payload.update({
            collection: collectionSlug,
            id,
            data: {
							import: {
								...result.import,
								objId: result.import?.objectId
							},
              ...overrides || {}
            },
          })

          console.log(`Document in '${collectionSlug}' with id '${id}' updated successfully`);
        } catch (e) {
          payload.logger.error(`Document in '${collectionSlug}' with id '${id}' failed to update`);
          payload.logger.error(e);
        }
      } else {
        console.log(`No document found in '${collectionSlug}' with id '${id}'`);
      }
    }));
  } catch (e) {
    payload.logger.error('Something went wrong.');
    payload.logger.error(e);
  }

  console.log('Complete');
  process.exit(0);
};

resaveCollection();

(The important parts are around line 33 where I added import: { ...result.import, objId: result?.import?.objectId } to the payload.update function)

Then we can remove the 'objectId' field in the 'import' group.

Is this correct? If so, how can we go about removing the original/stale field data from the database? I tried adding objectId: undefined but that didn't work.