GA release v3.0.3

Changes since last release: v3.0.2...v3.0.3

See detailed announcement: orchestrator 3.0.3: auto provisioning raft nodes, native Consul support and more

Noteworthy:

• orchestrator/raft auto provisioning nodes: With 3.0.3 the failed node can go down for as long as it wants. Once it comes back, it attempts to join the raft cluster. A node keeps its own snapshots and its raft log outside the relational backend DB. If it has recent-enough data, it just needs to catch up with raft replication log, which is acquires from one of the active nodes. If it went down for a substantial amount of time, or is completely wiped out, the node gets, and bootstraps from, a valid snapshot from one of the active nodes.
• Key-Value support for master discovery: orchestrator will write to KV stores (Consul supported at this time) clusters' master identities. It will update KV upon failover.
• Web UI improvements: graceful promotion, mater takeover, silent mode
• More.

sha256sum:

0748e3e499447b6168dd67de24e562388477bf5b9a51d3783d88f81831cbb259  orchestrator-3.0.3-linux-amd64.tar.gz
162981cdad50f3f9f163ae1b4cee3e4974b12d62541affcdb506f827a38f3b62  orchestrator-cli_3.0.3_amd64.deb
329f71888743a95a12ee7c086408bc8516ebd0a5d6583a12072c5e3e80c4ec56  orchestrator_3.0.3_amd64.deb
4d34757572e784f8042aae9f8e11e8d437f0b9569afdcca4771635e464ebd851  orchestrator-cli-3.0.3-1.x86_64.rpm
63307f4de051c774ef23074d81d34d8d0ab4ea0684f7c55d8e1166d9f40157ad  orchestrator-client_3.0.3_amd64.deb
6c3e9ca89188c50132186aaec5a1c4afaf69719c0093439193caf1799bfe691b  orchestrator-client-3.0.3-1.x86_64.rpm
c6489fd6e4f4382f26a435a4b2b90c1a67f5ab4955eb12815009bb913a20fa7d  orchestrator-3.0.3-1.x86_64.rpm


5fe7bf4a920643c2bb3309f54945bd99b385b07b8e8955477741592567f02858  orchestrator-centos6-3.0.3-1.x86_64.rpm
80d7222444d6b40b3e9147e059dd67d49bcc67225bcc137b1867669c06c4ed85  orchestrator-client-centos6-3.0.3-1.x86_64.rpm
e0da1fb38341cba55e6f02bed5817d11d0ce7572b4f125256dec65e89f259794  orchestrator-cli-centos6-3.0.3-1.x86_64.rpm

GA release v3.0.2

GA release of orchestrator 3.0, please note v3.0-pre-release and v3.0.1.pre-release

Re-listing major changes in 3.0 version, followed by specific changes since v3.0.1

raft consensus

• orchestrator/raft setup: consensus based leader election & quorum. orchestrator/raft setup achieves high availability without the need for a shared-backend high availability solution such as Galera/InnoDB Cluster.

  Backend DB nodes are independent and do not communicate with each other. All communication goes by the orchestrator nodes gossiping via raft.

SQLite

• SQLite support: in a orchestrator/raft setup each orchestrator node has a dedicated backend DB. This backend DB can be a standalone MySQL server or an embedded SQLite database. orchestrator now comes with SQLite embedded within its binary.

  SQLite is also supported on single-node setups, useful for local dev machines, and on testing or CI environments.

orchestrator-client

• orchestrator-client: a utility script which removes the need for an orchestrator binary on remote boxes.

---

changelog

Changes since last release: v3.0.1.pre-release...v3.0.2

Noteworthy:

• Packaging for orchestrator-client - the standalone client shell script
• Support for --ignore-raft-setup command line client invocation, overriding raft config; power to the engineers.
• Failure detection takes notice of downtimed replicas
• Web/UI fixes, adjustments
• Web/UI: Better visualization of downtimed problems
• Fix and reprioritization of promoted servers on DeadIntermediateMaster
• Misc. bug fixes
• more...

$ md5sum *
ea5391f69e1ada71d3c58f7646f8a048  orchestrator-3.0.2-1.x86_64.rpm
0030f8379fa870a2e761f178824c9241  orchestrator-3.0.2-linux-amd64.tar.gz
6ce932f9356baaa2ce8a51f88404cb01  orchestrator-cli-3.0.2-1.x86_64.rpm
51573a791f3642e75537c36add0b5b08  orchestrator-cli_3.0.2_amd64.deb
657185cb863363a6291d30591d439c70  orchestrator-client-3.0.2-1.x86_64.rpm
ced92c8648d28514dd6b99a7a3f87842  orchestrator-client_3.0.2_amd64.deb
e16e960f3252409e50ff6b9443f0b642  orchestrator_3.0.2_amd64.deb

$ sha1sum *
157c6b4d9d3d6c658afa56ec7e6259934c4a48fd  orchestrator-3.0.2-1.x86_64.rpm
10e66cd09097c2cf1be8e36a5b1f351113a3fe7c  orchestrator-3.0.2-linux-amd64.tar.gz
40a0d83d764d61eeec364299754a8098258d5206  orchestrator-cli-3.0.2-1.x86_64.rpm
e2c4e4773d425b021f0031deafcb98e66517b7d9  orchestrator-cli_3.0.2_amd64.deb
7a92a93a9ea2922e055771b37d0c08b88a3c84f0  orchestrator-client-3.0.2-1.x86_64.rpm
a58e4b6db3a140ef9e3f32b7eceaa40a6b332c66  orchestrator-client_3.0.2_amd64.deb
378ca20b5eaf980e83283f36ac2c71ec8bd32bd2  orchestrator_3.0.2_amd64.deb

$ sha256sum *
c5ca99a39727d22edebeb0b0f2908a085497d031d6d7b2bf06d41c92282fcb7a  orchestrator-3.0.2-1.x86_64.rpm
7b43ae94517ead11fbc01c55499d9e19d88adf2be5c40e9c346af202fef78451  orchestrator-3.0.2-linux-amd64.tar.gz
f7708562abc757ee471f9258c2640cef552254db4b3496f40fb12f073ddbde9d  orchestrator-cli-3.0.2-1.x86_64.rpm
1c2508d9376f3cdf16b4f8d46b114da1176ab096b3844bd91546a6d464623694  orchestrator-cli_3.0.2_amd64.deb
fe0665dd1a5d6093fb26c1b7f33b83a9ed42612639d2daf34dd0d40a9337f3c2  orchestrator-client-3.0.2-1.x86_64.rpm
6991b293435e2482c6e915b725c1a48e6a6128c6c5a08c8b7145d2874be9ee59  orchestrator-client_3.0.2_amd64.deb
d365f2bf614d659450f3dbad0920175914b02d62a736e465793b9dfdb1e98561  orchestrator_3.0.2_amd64.deb

$ md5sum *RHEL6*
ffa077686a54b0e826d2f47253db3349  orchestrator-RHEL6-3.0.2-1.x86_64.rpm
68d222e0531531ae078fe6616d64dc42  orchestrator-RHEL6-cli-3.0.2-1.x86_64.rpm
65e03a04c1691cd21d9a98600bb2934d  orchestrator-RHEL6-client-3.0.2-1.x86_64.rpm

PRE RELEASE v3.0.1

Pre release 3.0.1:

Changes since last release: v3.0.pre-release...v3.0.1.pre-release

• orchestrator/raft: configuration supports fqdn (hostnames): #254
• Graceful period before running detection/recovery: #251
• Support for prefer_not promotion rule: #247
• Updated docuentation
• Fixed pem tests: #255 thanks, @maurosr!
• Fixed handling of ClusterNameToAlias map

$ md5sum *
a62bdd734282ad20d2da7313eb416d7a  orchestrator-3.0.1-1.x86_64.rpm
f9aeb7e74a86400af80958336052a759  orchestrator-3.0.1-linux-amd64.tar.gz
1a20484abec704480874ca1e342537a8  orchestrator-cli-3.0.1-1.x86_64.rpm
22ec4b9532e1415608c4e4f6f7cacdd3  orchestrator-cli_3.0.1_amd64.deb
3e974e6ffaa70076e52f13f1d3490624  orchestrator_3.0.1_amd64.deb

$ sha1sum *
4b2c4983c4b7238904fdc45b71dac5dc7806c54a  orchestrator-3.0.1-1.x86_64.rpm
94e4588cfe016c9d2248679477f75ac796c8f923  orchestrator-3.0.1-linux-amd64.tar.gz
3456a283b7dc49c2a22a402ce77b533dc325cd8a  orchestrator-cli-3.0.1-1.x86_64.rpm
87a05f4ab88bfcb47abc503926bf3219bfe4020f  orchestrator-cli_3.0.1_amd64.deb
6c733d4223bf7b273eb1d98fb2b4fb3f1fc46692  orchestrator_3.0.1_amd64.deb

$ sha256sum *
28e64b718d08eac5e299d99d14c922e1b305b242096da8c7672f5dfc2ca0c8ee  orchestrator-3.0.1-1.x86_64.rpm
a1975798664f008a7a869676e9a00d6c626bb127c915cf87b87395aba3c186f0  orchestrator-3.0.1-linux-amd64.tar.gz
3d7fcc491a0e105954b76fab7b9c47cac5521f370d6252dfaac6a84ccba83a2e  orchestrator-cli-3.0.1-1.x86_64.rpm
008ca64abec27620a33c5bd08889796744d40dc8882bcf3d1f5d5b07680c7108  orchestrator-cli_3.0.1_amd64.deb
7f9784a3462b8c4c5ed2bd6185dfae4c28117000d423c8ea103751f274758238  orchestrator_3.0.1_amd64.deb

PRE RELEASE v3.0

Pre release 3.0 offers:

• orchestrator/raft setup: consensus based leader election & quorum. orchestrator/raft setup achieves high availability without the need for a shared-backend high availability solution such as Galera/InnoDB Cluster.

  Backend DB nodes are independent and do not communicate with each other. All communication goes by the orchestrator nodes gossiping via raft.

• SQLite support: in a orchestrator/raft setup each orchestrator node has a dedicated backend DB. This backend DB can be a standalone MySQL server or an embedded SQLite database. orchestrator now comes with SQLite embedded within its binary.

  SQLite is also supported on single-node setups, useful for local dev machines, and on testing or CI environments.

• orchestrator-client: a utility script which removes the need for an orchestrator binary on remote boxes.

• Various other changes, see v2.1.5...v3.0.pre-release ; special thanks to @sjmudd, @maurosr and @dveeden for continuous contributions.

See announcement on http://code.openark.org/blog/mysql/orchestratorraft-pre-release-3-0

md5 checksums:

c357aef3e6794b7c473042b10b0b49d2  orchestrator-3.0-1.x86_64.rpm
3d79877ec33d220975c710b148996c49  orchestrator-3.0-linux-amd64.tar.gz
7ca1d6120162bdda3b576bc6c3eac203  orchestrator-cli-3.0-1.x86_64.rpm
6620eba6a5c73345928c3c908e1d15c0  orchestrator-cli_3.0_amd64.deb
9d80bb78e86a01eb54a2872508e9df8b  orchestrator_3.0_amd64.deb

sha1 checksums:

67f29ac832f48ea983a3f0d66640ed522ff3f5fb  orchestrator-3.0-1.x86_64.rpm
5b283e763436326493c655f0a24c3826bd2ea9be  orchestrator-3.0-linux-amd64.tar.gz
7ec395b5a1c9f0fdebaa95b7133e569590db1a96  orchestrator-cli-3.0-1.x86_64.rpm
4da53ffebcf0edb5aa84cd5ea1a8211cd7af8b26  orchestrator-cli_3.0_amd64.deb
939bdebfa5dc15979542e0969710438f0c306a46  orchestrator_3.0_amd64.deb

sha256 checksums:

3229aaf90d0ba8cd4408afb619aa9f926060ec84079ea9607faf16b3941527be  orchestrator-3.0-1.x86_64.rpm
06d87b5de0c2a5145c3a5a74113a61bb1804e971ef68c4dc3fd79f757d595d03  orchestrator-3.0-linux-amd64.tar.gz
91aec077f417bdec90a172e76e0d30710c5768351d4ca237641c266f4988f011  orchestrator-cli-3.0-1.x86_64.rpm
90b21f5ed181c54c9e3f2863afe9516888d7b9b3083d547dd77e7b273be78323  orchestrator-cli_3.0_amd64.deb
b46c987fee44e05a8bce21c918269357ed1ce2d4bc5b7eac571e41507960909d  orchestrator_3.0_amd64.deb

GA release v2.1.5

Changes since last release: v2.1.4...v2.1.5

Notable:

• ReplicationLagSeconds value was inconsistently broken following #182, fixed by #219
• Fixed XSS flaw with search box, #218, thanks @Oneiroi
• Capping of idle backend connections by @sjmudd , #210
• /web/status better visualization via #213 and #216
• PowerAuthGroups config via #215: authentication based on unix groups, by @sjmudd
• Other small changes

md5 checksums:

ea75098db392fd1aa58348503cffd9b3  orchestrator-2.1.5-1.x86_64.rpm
088aed4caac9430b00455ac8d0c23acf  orchestrator-2.1.5-linux-amd64.tar.gz
6725899bd5ac21b603ee0ddb4126b709  orchestrator-cli-2.1.5-1.x86_64.rpm
f58f27b24851cad956fba70013ca4cf4  orchestrator-cli_2.1.5_amd64.deb
eef8f39d43c3ad4c874563e4139c5a34  orchestrator_2.1.5_amd64.deb

sha1 checksums:

59464521a1b454ed499ff32088fc0bf39432de92  orchestrator-2.1.5-1.x86_64.rpm
84f101f16fbb8427909cdb8c48b7544c37652285  orchestrator-2.1.5-linux-amd64.tar.gz
b1e3328d69738cab52333d34c9f54f295c871905  orchestrator-cli-2.1.5-1.x86_64.rpm
eba21da0445991d063e8c97f962a054c30d3b1e8  orchestrator-cli_2.1.5_amd64.deb
919f02d413f7c7fbc2bef1c97d492ce75dc2e7b9  orchestrator_2.1.5_amd64.deb

GA release v2.1.4

Changes since last release: v2.1.2...v2.1.4

Notable:

• Important bugfix in recovery flow; some failure detections could go unhandled depending on concurrency behavior, see #196
• Added -c force-master-failover command, which just kicks in a failover for a cluster; there is no designated server to promote, orchestrator just makes believe master is dead and does whatever it does to fix the situation.
  • likewise /api/force-master-failover/:hostname/:port
• The Great Configuration Variables Exodus (deprecation) -- deprecating many variables; more work to come
• lost-in-recovery: masters who have been lost in recovery are:
  • downtimed forever
  • or until found to be healthy in a replication environment
• Concurrent probing of topology servers: reduces per-server discovery time by some 70% on high latency networks
• More visibility in failure detection process:
  • Better logging
  • Introduced "is actionable" flag for detection
  • Allows multiple detections in backend database for same instance/incident: up to one detection entry for "actionable", up to one detection entry for "not actionable". This gives some more (though not full) visibility into escalation process.
• added /api/leader-check: returns 200 when node is the leader, 404 if not.
• more... thanks @sjmudd for continued contributions

GA release v2.1.2

Changes since last release: v2.1.0...v2.1.2

Notable:

• structured, audited recovery steps:
  • each recovery has audited steps, identified by UID
  • UID presented in post failover hooks
  • underlying table: topology_recovery_steps
  • presented in web interface
• recovery speed improvements:
  • faster, concurrent execution of postponed functions
  • yet more optimization in prefering DC-local operations
  • incremental sleep following START SLAVE reduces wasted sleep time
  • more postponed functions for unurgent operations
  • recognizing hosts with discovery latencies, pushing them to postponed recovery
• graceful-master-takeover attempts setting replication credentials on demoted master (doable if master_info_repository=TABLE)
• FailMasterPromotionIfSQLThreadNotUpToDate config (bool, default false), when true and master fails while all replicas are lagging, heal topology but do not proceed with post-failover hooks, and in fact consider the failover as failed.
• Many metrics added by @sjmudd
• Fixes and improvements to GTID master recoveries
• binlog_row_image collected and made visible
• TravisCI now tests all PRs
• more...

BETA release 2.1.1

This solves:

• GTID based graceful takeover #78
• Promotion of lagging replicas #83
• many other things

GA release v2.1.0

Changes since last release: v2.0.3...v2.1.0

• Massive SQL related changes, supporting sqlite as backend (sqlite support is tested but sqllite as backend is not included in this release).
  • Backend schema: changes to index names. By running this release first time you will experience a once time rebuild of almost all backend tables
  • Queries: many queries simplified. Multi-table DMLs broken into parts. GROUP/GROUP_CONCAT simplified; some logic moved to app.
  • Otherwise operations are unchanged; many tests added
• Supporting MultiMatchBelowIndependently. This supposedly step-back from multi-server bucket optimization is made possible by binlog indexing, released a year ago, which alleviates the pain from re-scanning master binlogs. Right now, for backwards compatibility, this defaults false; but potentially in the future it will change to true. Very large installments, where number of replicas per server is within the many dozens or hundreds, will still benefit from the bucketing approach (MultiMatchBelowIndependently: false)
• Fix bug in identifying Percona Server GTID based replicaiton
• Support DiscoveryIgnoreReplicaHostnameFilters (thanks @sjmudd). With growing number of apps that pretend to be replicas (gh-ost, tungsten, others...) orchestrator falls into the trap of trying to discover a "server" where that "server" is really some app. This is an initial, simple approach to filter out such apps based on hostnames. We may iterate on that in the future.
• More failure analysis scenarios and detection
• More...
• Also thanks @samveen (working on build script init script)
• Thanks to the many collaborators who report issues and discuss potential fixes

GA release v2.0.3

Changes since last release: v2.0.2...v2.0.3

Notable changes:

• Better support for 5.6->5.7 replication with Pseudo-GTID (Booking.com contribution)
• Docket image improvmenets (@calind contribution)
• More detailed timing metrics for discoveries, backend latencies etc. (Booking.com contribution)
• Logging improvements (Booking.com contribution)
• supporting SkipOrchestratorDatabaseUpdate and PanicIfDifferentDatabaseDeploy
• relaxation of health expiry writes (Booking.com contribution)
• support for PseudoGTIDPreferIndependentMultiMatch

and many more. Special thanks to @sjmudd!