PS-9334 Release tasks ticket for PS 8.4.0-1 #5387
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove previously deprecated option --language and its supporting code. --lc-messages-dir/--lc-messages should be used instead. Change-Id: I741cce0dde478f4874ff56a11426941a8d1187d2
Post push fix: Some test files had to be rerecorded. Change-Id: I9ff77050cc4b8620ec99bf5de5230aa79331709f
This WL introduces a new privilege named FLUSH_PRIVILEGES which will allow the users to execute FLUSH PRIVILEGES command without the need of RELOAD privilege. Change-Id: I1babda8686a8a34d0f594a1c425cabfa43fb30d8
Symptoms ---------- The SELECT COUNT(*) query that uses secondary index for scan, takes more than 6X with 8.0 compared to 5.7. Performance of the query that use the clustered index for scan is at par with 5.7. Background ---------- In 8.0.14, we introduced parallel scan of subtrees of a clustered index through 'WL#11720 InnoDB: Parallel read of index'. This feature was further improved through 'WL#12978 InnoDB:Fix imbalance during parallel scan' in 8.0.17. The idea of the second worklog was to utilize the parallel read threads which were introduced through first worklog. Second worklog also made another intentional change(not documented!) that is, secondary index scan in InnoDB started using the parallel scan of clustered index. In other words, even if Optimizer hints InnoDB to use secondary index for scan, InnoDB ignores that, and uses clustered index for scan. This observation was reported in the following bugs: 'Bug#31791868 INDEX HINT DOES NOT AFFECT COUNT(*) EXECUTION' 'Bug#35981123 SELECT COUNT is slow with O_DIRECT' The second worklog enabled the clustered index (parallel) scan even for secondary index scan for the following two reasons : - Post WL#12978, performance of clustered index scan was much improved. - Cost of scanning the secondary index is much higher to support the MVCC semantics i.e. read-committed, repeatable-read isolation levels. That is because before reporting a record from a secondary index, InnoDB must ensure it is visible to the query's read-view. That might require a dive into clustered index to consult corresponding row's undo chain. This can be avoided in case InnoDB sees that a secondary index record was not modified since read-view's creation, but unfortunately this information is not available per-row, only per-page. So, if any of the records on the page was modified since read-view's creation, the heuristic will not help, and dives to clustered index will be needed for all records from this secondary index page. The second reason is main argument behind this change which can be easily reproduced by modifying a few records in the table and then executing the query in a session that has read_view created before the modification, for example by having opened transaction on isolation level of Repeatable Read before the modifications. In my experiment, I found with secondary index scan(or with this fix), query performed 30X worse than the query performed without a read view. Thus it is really a trade off between using the clustered scan that is better in worst case vs using the secondary index as Optimizer hints which is better in most of the cases. Fix ---- - Do not break the contract with Optimizer(Bug#31791868). User now (again) has an option to provide index hint to the query, to force using clustered index. - We must improve Optimizer's heuristic so that it can hints the correct index to use. Created 'Bug#36297856 Query performance dropped by 30X with secondary index scan due to MVCC checks', for Optimizer team to improve the heuristic in Optimizer. - Reverted part of the WL#12978 changes that forced clustered index (parallel) scan for secondary index scan. Change-Id: Icb081d076cb6875e7b973e68ff6cfcce8ae3f82a
…hd_protocol - use_thd_protocol is the name used in the backend for passResultToClient. - passResultToClient is used at XDevAPI, truffle and MLE while use_thd_protocol is used inside service and backend. Change-Id: I00ba4ae06bbbbeccee24a4f3975c234e1dfa0d16
Change-Id: I5c4e86111159a5143f23a0f6d36cd28c43d6fe57
Problem ======= - At udf_handler::get_and_convert_string, we get a String* result pointer, which might be the address of Item::str_value. We then call c_ptr_safe() on this, so the buffer of Item::str_value gets reallocated. - Queries which have UDFs consuming the same argument more than once, the Item_ref has the same ref_item, with the same underlying str_value. Example: SELECT UDF(col1, col1) FROM view_t1; - In such cases, when setting the m_ptr of Item::str_value directly (while processing the second argument), we free the buffer that we had allocated at c_ptr_safe(), at the time of processing the first argument. - As a result, the UDF will try to read a buffer that has been freed, causing the ASAN issue. Solution ======== - The root cause is that the udf_handler changes contents of Item::str_value directly. Instead of doing that, assign the contents of Item::str_value to udf_handler local buffers. Note that this is not a deep string copy. - Any further calls to c_ptr_safe() will not modify Item::str_value object, avoiding the issue. Change-Id: Ib3d6e1d52825dafbc1e495f5f45f8fac75bc90f1
Step 1. Log sanitizer - common class for relay/binary log recovery This step refactors Binlog_recovery class and methods to be re-usable during the relay log recovery process. Introduced changes: New Log_sanitizer class, containing utilities for: - iteration over events in existing log file/files and extracting: - the last valid log position - the last valid source log file and position - removal of log files containing only partially-written transactions from disk and the index file - truncation of the last log file to contain only fully written transactions Some of the functionalities are used only for relay log recovery. Binary log recovery will never remove binlog file, since binary log file always start after the last finished transaction. When relay log recovery is considered, we need to find a start place, which will be either: - a Query Event containing the end of the transaction: - COMMIT / ROLLBACK / XA COMMIT / XA ROLLBACK - a Query Event containing an atomic DDL - a source Rotate Event - a Xid Event Afterwards, searching for the last valid position begins. If no valid position can be found in relay log files (e.g. we cannot decrypt relay log files), relay log recovery won't perform any action. Change-Id: If76694d2dfa0c62ff7b7243cff9bea451117b376
Step 2: Relay log sanitization This step implements relay log sanitization. Sanitization process runs during applier metadata initialization, in case relay-log-recovery server option is turned off. It is implemented in a separate class, 'Relay_log_sanitizer'. Whenever a partially-written transaction is encountered at the end of the file, sanitizer performs truncation of the relay log file via execution of the existing 'truncate_update_log_file' function. After the relay log truncation and loading information stored in applier metadata on disk, relay log sanitizer updates the position of the receiver thread. In case no valid source position could be obtained from the existing relay log files, sanitizer matches the receiver thread position with the applier source position read from the applier metadata. Change-Id: I13203812b2d73f38fca711473097daed55721f00
Step3: Testing This step tests the feature introduced in WL#7353 and adjusts existing tests to account for the relay log sanitization executed during the applier metadata initialization. Change-Id: Ia3845040323925fe98edaf22bd1547bdb0e1052a
This worklog is aimed at extending the current identification of the transactions within the system. It gives the user a possibility of tagging groups of transactions - giving a group of transactions a specific name. This way, the user will be able to easily distinguish between groups of transactions by looking at transaction GTID and searching for a specific transaction tag. Server already provides functionality to assign unique or automatically generated GTID for the next transaction or transactions executed within the current session scope. Instead of using source UUID as first part of transaction identifier, we introduce Transaction Source Identifier (TSID) composed of source UUID and transaction tag (Tag). TSID will replace SID as the first component of transaction GTID. Replication already contains parts of code related to: - recovery of binlog files, which finds last valid position within the binary log file - the 'Binlog_recovery' class - function to truncate relay log file to the last valid position: truncate_relaylog_file - function to truncate log file to the last valid position: truncate_update_log_file This worklog will create relay-log sanitization process executed at start of the replica. We will: - rewrite existing Binary log recovery code to be reused for truncation of the relay log files - scan relay log files for the last relay log file containing last finished transaction - truncate last relay log file so that it contains only fully-written transactions - remove relay log files containing only partially written transactions - update relay log index file - update of receiver position to recovered source log position and source log filename in case a valid source position and filename have been obtained while reading of the relay log files - update of receiver position to applier position read from the repository in case no valid source position and filename could be obtained while reading of the relay log files Binlog file always start with a new transaction, whereas relay log file may start in the middle of a transaction. Therefore, it might be possible that there are some invalid relay-log files which contain last partially written transactions. In that case, those relay log files will be removed and relay log index file will be updated accordingly. Whenever a last valid position and corresponding filename were recovered while reading the relay log files, server will update receiver position to match recovered one. Otherwise, receiver thread position will be matched with applier position read from the repository. Note that this functionality won't check event consistency in all of the relay logs, it will only check consistency of events after relay log sanitization process has been started. The worklog consists of three steps: Step 1. Log sanitizer - common class for relay/binary log recovery Step 2. Relay log sanitization Step 3. Testing Change-Id: I9fcf0c5a2e4ebfc6eb5e1cfcc0a3bf730d091e2d
…ons [8.0] Post WL#15989: Innodb: Allow clone across patch releases within one LTS Series clone is allowed between LTS patch version. At present Group Replication blocks cloning between cross version members. However GR should allow clone from any version. And Clone plugin can decide if clone can be done or not between cross-version members. Change-Id: Ib06a54487fa0b76c3df95babd0ce2c031859f6cb
The Router now exposes its static configuration in the metadata on boostrap and updates it each time it starts. It is exposed as a JSON in the routers.attributes.Configuration field. Additionally on fist bootstrap of the given version, the Router exposes the default bootstrap options for this version in the cluster/clusterset table respectively, in the router_options.Configuration.<version>.Defaults field. The options that the given version of the Router supports changing dynamically via the metadata are exposed in router_options.Configuration.<version>.ConfigurationChangesSchema field. Change-Id: I167622519c48eb1f60ef89400150e7ec74890ee1
This reverts commit 1f61a2118335a0f91795bcd907fe69c36748eaf8. Change-Id: I17eaf6ba06de36e37d8af278de3a48243197be50
…ling on pb2 Asserts on binlog events may be evaluated before receiving them. Add wait conditions to confirm events were received before doing assert. Also increase timeout on rpl_sync due use of big write sets. Change-Id: I86a434c05d2e8fe4f7c826516f36518632e3fecf
Removed the --ssl and --admin-ssl options. Moved the client utility header files into the client/include directory. Removed all use of --ssl/--admin-ssl in the tests. Changed the default for --require-secure-transport to on. Removed have_ssl and have_openssl Change-Id: I3e7a4b1ee3725310bccab3cb731b72d7448ad144
This step removes the keywords and rules pertaining the commands that were deprecated and will now be removed in this patch. It also does minor test cleanup dealing with reserved keywords that will now disappear. Minor code renames are also done to modernize the parser code in terms of terminology. Change-Id: Id188dfce2ae74d912c3d982f775095c0c023bb38
This patch does minor changes to mysql client to modernize the list of words it uses for completion. For mysqladmin we replace the usage of RESET MASTER with the new command RESET BINARY LOGS AND GTIDS. The tool will however use the old command if the version of the server it is contacting does not support it. We also do minor renames for command enum values in the server. Finally, we also remove the deprecated status variables that contain the master/slave terminology. Change-Id: If609924fc99a61e7e373f1ed01a3e758627cf66f
As a result of the renaming done in the past steps, several files in the server need to be updated to use the new command names, class names, enumeration values, etc. Change-Id: I6f969d68a56ae292f99fb1162c0ed3d8d1183758
The usage of the removed commands was mostly dealt with in previous worklogs. Still there are tests that still used old terminology and also tests that tested that both new and old terminology was accepted, and these were now corrected or removed. Change-Id: Ie1e7717f59c84107e376940584811c91ff4f2b84
the server > Merge branch 'mysql-trunk-wl15831' into mysql-trunk This worklog removes the all the replication commands, clauses and status variables that contain offensive terminology and that were deprecated in previous versions. The removed commands are: - START/STOP SLAVE: replaced by START/STOP REPLICA - SHOW SLAVE STATUS: replaced by SHOW REPLICA STATUS - SHOW SLAVE HOSTS: replaced by SHOW REPLICAS - RESET SLAVE: replaced by RESET REPLICA - CHANGE MASTER: replaced by CHANGE REPLICATION SOURCE - RESET MASTER: replaced by RESET BINARY LOGS AND GTIDS - SHOW MASTER STATUS: replaced by SHOW BINARY LOG STATUS - PURGE MASTER LOGS: replaced by PURGE BINARY LOGS - SHOW MASTER LOGS: replaced by SHOW BINARY LOGS We also remove all the deprecated clauses for the following commands: - For CHANGE REPLICATION SOURCE we remove the following deprecated clauses: + MASTER_AUTO_POSITION: replaced by SOURCE_AUTO_POSITION + MASTER_HOST: replaced by SOURCE_HOST + MASTER_BIND: replaced by SOURCE_BIND + MASTER_USER: replaced by SOURCE_USER + MASTER_PASSWORD: replaced by SOURCE_PASSWORD + MASTER_PORT: replaced by SOURCE_PORT + MASTER_CONNECT_RETRY: replaced by SOURCE_CONNECT_RETRY + MASTER_RETRY_COUNT: replaced by SOURCE_RETRY_COUNT + MASTER_DELAY: replaced by SOURCE_DELAY + MASTER_SSL: replaced by SOURCE_SSL + MASTER_SSL_CA: replaced by SOURCE_SSL_CA + MASTER_SSL_CAPATH: replaced by SOURCE_SSL_CAPATH + MASTER_SSL_CIPHER: replaced by SOURCE_SSL_CIPHER + MASTER_SSL_CRL: replaced by SOURCE_SSL_CRL + MASTER_SSL_CRLPATH: replaced by SOURCE_SSL_CRLPATH + MASTER_SSL_KEY: replaced by SOURCE_SSL_KEY + MASTER_SSL_VERIFY_SERVER_CERT: replaced by SOURCE_SSL_VERIFY_SERVER_CERT + MASTER_TLS_VERSION: replaced by SOURCE_TLS_VERSION + MASTER_TLS_CIPHERSUITES: replaced by SOURCE_TLS_CIPHERSUITES + MASTER_SSL_CERT: replaced by SOURCE_SSL_CERT + MASTER_PUBLIC_KEY_PATH: replaced by SOURCE_PUBLIC_KEY_PATH + GET_MASTER_PUBLIC_KEY: replaced by GET_SOURCE_PUBLIC_KEY + MASTER_HEARTBEAT_PERIOD: replaced by SOURCE_HEARTBEAT_PERIOD + MASTER_COMPRESSION_ALGORITHM: replaced by SOURCE_COMPRESSION_ALGORITHM + MASTER_ZSTD_COMPRESSION_LEVEL: replaced by SOURCE_ZSTD_COMPRESSION_LEVEL + MASTER_LOG_FILE: replaced by SOURCE_LOG_FILE + MASTER_LOG_POS: replaced by SOURCE_LOG_POS - For START REPLICA we remove the clauses: + MASTER_LOG_POS: replaced by SOURCE_LOG_POS + MASTER_LOG_FILE: replaced by SOURCE_LOG_FILE We also remove the status variables: - Com_slave_start: replaced by Com_replica_start - Com_slave_stop: replaced by Com_replica_stop - Com_show_slave_status: replaced by Com_show_replica_status - Com_show_slave_hosts: replaced by Com_show_replicas - Com_show_master_status: replaced by Com_show_binary_log_status - Com_change_master: replaced by Com_change_replication_source Reviewed-by: Sven Sandberg <sven.sandberg@oracle.com> Reviewed-by: Neha Kumari <neha.n.kumari@oracle.com>
Post-push change: Fix for routertest_harness_loader_lifecycle failing on ASAN build. The test calls loader.run() without properly initializing the loader, which with new API introduced by the WL is causing a dangling pointer to be used inside expose_initial_config_all(). This patch adds a call that inits a necessary data structure. Change-Id: Ib1e6fe85e6676291cb102f1a19b5eb502aadba88
Removed the SET_USER_ID privilege. Fixed the tests to reflect its absence. Change-Id: I1cceb6bb778324660c269fb837c7ecc376863451
Disable MLE for RPM on 8.4. Change-Id: Ie550fd5faea35283e2456f0f207b67d3d518abe8
After WL#16095 remove server side --ssl option was pushed, the integration tests of the router started to fail. The router tests expected that non-encrypted connections to the server can succeed, but WL#16095 changed the default for --require_secure_transport from OFF to ON. Change ====== - set --require_secure_transport=OFF for the mysqld that's started by the integration tests. Change-Id: I3728693bc5e0ff9a358b99c0547353cc074dc979
Server-side connections that are sharable are currently placed into the connection-pool which is limited by "max_idle_server_connections". If "max_idle_server_connections" is 0, connection-sharing is disabled. Moving a connection into the connection-pool, means that moving it out of the pool needs to restore the earlier session-state of system-vars, trx-state, ... (even if the connection wasn't used by another connection). This WL changes how sharable connections are maintained: 1. the max_idle_server_connection only controls server-side connections where the client side is closed 2. sharable connections don't need to be restored if they weren't used by another connection. Result is that connection-sharing and rw-splitting have less overhead. Change ====== - new REST API element: stashedServerConnections - store sharable connections on "the stash", seperate from the normal connection pool for dangling server-connections. - simplified the connection-class that's stored in the connection-pool into the routing_connections library - max_idle_server_connections=0 is allowed now - moved headers that are shared between connection-pool and routing into mysqlrouter/ - moved TlsSwitchable to mysqlrouter/connection_base.h Change-Id: I2c7b280ba06c66daccd731e551d0655a49691fea
Follow up fix for broken build of ndb_top due to moved include files Change-Id: I1262be73289e0089c6902c8045e5df7045b6d9ee
Post-push fix: Adjusting the copyright notice in the new files to the new format. Change-Id: I0d974933f80f1e0f374db5f541c4a30c5874f06a
Split router/cmake/configure.cmake to allow reuse. Change-Id: I4f4ca13729767e14ff75d0d915b124596ed1e132
Issue: In bug#35183686 fix, we started logging the fields whose column order changed. But while calculating size needed in redo to log index information these columns were missing from calculation. Fix Make sure these colums are also considered while calculating size needed to log index entry. Change-Id: Ic8752c72a8f5beddfc5739688068b9c32b02a700 (cherry picked from commit e6248f5)
- Post push fix for memory leak Change-Id: I034bb20c71dfe3ae467e762b2dd7c7f95fb0679b (cherry picked from commit a3561b3)
PS-9222 Include reordered fields when calculating mlog buffer size
https://perconadev.atlassian.net/browse/PS-9233 UUID lib is from dev branch of upcoming version 1.86 The boost::uuid lib is from develop branch, last coomit hash 02c82ce
https://perconadev.atlassian.net/browse/PS-9233 This squashed commit contains also fixes done or suggested by Yura Sorokin
https://perconadev.atlassian.net/browse/PS-9233 date_time/c_time.hpp header and dependencies is from version 1.84.
…ng=WRITESET https://perconadev.atlassian.net/browse/PS-9302 Problem: Comparing to 8.0 where the default value of binlog_transaction_dependency_tracking was COMMIT_ORDER, 8.4 introduces a visible write performance drop. Cause: 8.4 uses WRITESET dependency tracking. For this tracking we maintain a map of binlog_transaction_dependency_history_size for row_id to newest transaction sequence_number which modified a given row. Every new transaction needs to check its dependency by examining the map (find), which is done with logarithmic complexity. Solution: Change std::map to std::unordered_map. This allows us to find a row dependency in constant on average complexity.
https://perconadev.atlassian.net/browse/PS-9218 - removed enable/disable_reconnect - fixed path to included files ('extra/' -> 'common/') - removed 'rocksdb.upgrade_parts_from_prev_ver' test, it checked upgrades from previous (non-native) partitioning format in 5.7 - updated rocksdb_rpl.rpl_rocksdb_stm_mixed_auto_increment and rocksdb_rpl.rpl_rocksdb_row_auto_increment tests to use rpl_auto_increment_innodb.test internally as rpl_auto_increment.test is now deleted - updated and renamed rocksdb_rpl.rpl_rocksdb_extra_col_slave and rocksdb_rpl.rpl_rocksdb_extra_col_master tests - deprecated LOW_PRIORITY WRITE changed to WRITE in rocksdb.lock - removed ENGINE=xxx from DROP TABLESPACE statement in rocksdb.create_table - re-record innodb.log_encrypt_kill test - updated percona.utility_user test to mask thread ID in ER_KILL_DENIED_ERROR error string - in rocksdb.autoinc_vars removed checks for DOUBLE AUTO_INCREMENT as support for it was removed in this release - in rocksdb_rpl.rpl_rocksdb_row_img_idx_full, rocksdb_rpl.rpl_rocksdb_row_img_idx_min and rocksdb_rpl.rpl_rocksdb_row_img_idx_noblob binlog_transaction_dependency_tracking=COMMIT_ORDER changed to binlog_transaction_dependency_history_size=1 as COMMIT_ORDER was removed by upstream.
…connection_configuration [Post fix] Fixing test rpl_log_info_repository_persistence_require_row that was not accounting for the group replication recovery channel that is now shown when configured. Change-Id: I514e8f03536d928f884505cca7be130fe8285b4d
PS-9302: Improve performance for binlog_transaction_dependency_tracki…
https://perconadev.atlassian.net/browse/PS-9218 Added more warning suppressions for RocksDB submodule files that appeared in GCC 12.3 in RelWithDebInfo mode.
https://perconadev.atlassian.net/browse/PS-9218 Reworked "table_list->partition_names = &alter_info->partition_names;" part from the WL#15517 "[5] HeatWave Support for InnoDB Partitions - Support for Loading Partitions" patch introduced in mysql/mysql-server@951ffba. It turned out that changing this 'partition_names' affects "ALTER TABLE ... ALGORITHM=INPLACE, DROP PARTITION ..." logic under '--rocksdb-allow-unsafe-alter=ON'. In particular, in the following call stack 1. 'mysql_inplace_alter_table()' 2. 'open_table()' from the /* Tell the SE that the changed table in the data-dictionary. For engines which don't support atomic DDL this needs to be done before trying to rename the table. */ 3. 'table->part_info->set_partition_bitmaps()' from the /* Position for each partition in the bitmap is read from the Handler_share instance of the table. In MYSQL_OPEN_NO_NEW_TABLE_IN_SE mode, table is not opened in the SE and Handler_share instance for it is not created. Hence skipping partitions bitmap setting in the MYSQL_OPEN_NO_NEW_TABLE_IN_SE mode. */ 4. 'set_read_partitions()' 5. 'add_named_partition()' if the "table_list->partition_names = &alter_info->partition_names;" assignment is made, we will get directly into 'my_error(ER_UNKNOWN_PARTITION, MYF(0), part_name, table->alias);' as 'partition_names' will include the list of partitions mentioned in the "DROP PARTITION ..." clause and 'find_or_nullptr()' will not be able to find already dropped partition in the 'partition_name_hash'. Fixed by saving the old value of 'partition_names' before calling 'secondary_engine_unload_table()' and restoring it afterwards. Stabilized 'rocksdb.partition' MTR test case by adding '--sorted_result' to 'SHOW TABLES".
…RFC 9562 (packaging) (#5360) https://perconadev.atlassian.net/browse/PS-9233 Updated both DEB and RPM packaging scripts to include new 'component_uuid_vx_udf.so' shared library.
RM-1407 PS-8.4.0-1
RM-1407 PS-8.4.0-1
PS-9334 Release tasks ticket for PS 8.4.0-1
fix nss bundled libraries
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.