Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add primary key to audit enqueued table #456

Merged
merged 1 commit into from
Mar 10, 2024
Merged

Conversation

hlascelles
Copy link
Owner

@hlascelles hlascelles commented Mar 10, 2024

In some circumstances it is useful or even necessary for tables to have a primary key, eg Blue / Green deployment.

The que_scheduler_audit_enqueued does not have a primary key, so this PR adds one. This will require a migration so a new major version of que-scheduler is required. Note this may be a DB intensive operation that can take some time if you have been running que-scheduler for a while. Unless you are running the "cleanup job" (QueSchedulerAuditClearDownJob), you will have 1 row per job scheduled enqueued since the audit started.

Use a migration like so:

class CreateQueSchedulerSchema < ActiveRecord::Migration[6.0]
  def change
    Que::Scheduler::Migrations.migrate!(version: 8)
  end
end

The scheduler will pause until the migration to version 8 is completed, as the table will be locked. It may also mean that one que worker will be paused waiting to access that table if it takes so long that the "scheduler run" happens ("on the minute"). This may be fine if you have only tens or hundreds of thousands of rows. If you have millions, and don't wish to have any scheduler pause, it may be best to run a clear down with QueSchedulerAuditClearDownJob first (and maybe on an ongoing basis). See the README.md for more details here.

@hlascelles hlascelles changed the title Add primary key to audit table Add primary key to audit enqueued table Mar 10, 2024
In some circumstances it is useful or even necessary for tables to have a primary key, eg [Blue / Green deployment](https://docs.aws.amazon.com/whitepapers/latest/overview-deployment-options/bluegreen-deployments.html).

The `que_scheduler_audit_enqueued` does not have a primary key, so this PR adds one. This will require a migration so a new major version of que-scheduler is required. Note this may be a DB intensive operation that can take some time if you have been running que-scheduler for a while. Unless you are running the "cleanup job" (`QueSchedulerAuditClearDownJob`), you will have about 1 row per minute since the audit started.

The scheduler will pause until the migration to version 8 is completed. This may be fine if you have only tens or hundreds of thousands of rows. If you have millions, and don't wish to have any scheduler pause, it may be best to run a clear down with `QueSchedulerAuditClearDownJob` first (and maybe on an ongoing basis). See the README.md for more details.
@hlascelles hlascelles merged commit 9240a1a into master Mar 10, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant