Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PS-9422 Release tasks ticket for PS 8.0.39 #5446

Merged
merged 262 commits into from
Oct 22, 2024
Merged

PS-9422 Release tasks ticket for PS 8.0.39 #5446

merged 262 commits into from
Oct 22, 2024

Conversation

adivinho
Copy link
Contributor

@adivinho adivinho commented Oct 8, 2024

No description provided.

Andrzej Religa and others added 30 commits March 12, 2024 15:39
…rade

When the Router is bootstrapped to the fresh directory it tries to
create a  new account in the `routers` metadata table. This table has
a unique constraint UNIQUE KEY `address` (`address`,`router_name`).
If there already is a record with that address and router_name pair
the insert fails due to the contraint mentioned before. If no --force
parameter is given the Router fails to bootstrap with an error
as expected.
However if the --force parameter is provided and there already is
a record matching (`address`,`router_name`) pair, the Router queries for
the router_id to re-use it. The problem is that the query that does it
is invalid, it only has "WHERE router_name=?" clause, ignoring the
address part. This means that it can return multiple rows for any
router with a given name for various hosts/addresses. Then the id of the
random (first from the list of the rows) is taken. This can lead to
multiple Routers on various host using the same account, updating
the attibutes etc.

This patch fixes the query retrieving the router_id so that it matches
the unique constrain (address, router_name).

Change-Id: I8f8f7227e9597cbd5a917d8ef068c7b6214c5288
The router does not allow the router_id higher than 999999 in order
to be able to create an user name containing mysql_routerXX_YY where
XX is the router_id and YY is random 12-character suffix.

This patch changes that so that the highest possible router_id is
supported (2^32-1). To allow that the YY part is reduced to
7 characters.

Change-Id: I7db486d8849b01a790e07f265941c9fd89d27309
Post push fix. Also handle rmdir in PosixAsyncFile::rmrfReq.

Change-Id: I56ff2274b0ccd34e0e538ab499834d09288e9e84
Followup patch:
The upgrade to the latest protobuf library has made some changes that
are incompatible with the MSVC PGO compiler and linker options on
Windows.

These incompatibilities are addressed by excluding the protoc
executable and Abseil/protobuf DLLs from PGO.

Change-Id: I587e2c8729bd07afb6508c748ddf73eca881d2a7
Followup patch:
Suppress linker warnings generated as a consequence of combining
CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS with __declspec(dllexport)

Change-Id: I7c6b63e6db5c327b582eb8994ee6ef3a64356abc
… error

Loading of some of the libraries may fail:
ELF load command alignment not page-aligned

Linux aarch64 platform seems to be two flavours: 4k and 64k page size.

4k systems can load binaries with 4k and 64k page size, while 64k
systems can load 64k binaries only.

Hence, hard code argument for patchelf to --page-size=65536 to produce
most portable binary.

Change-Id: Ifbde485f61e0ce3814393b21f027b8be2f48d547
Problem: mysqldump not sanitizing the version string obtained from
server which may lead to injecting malicious commands to the output.

Fix: added function sanitizing the version string by cutting off
illegal part and issuing warning.

Test: check the server version in the output with and without injected
payload.

Change-Id: I1f19e1c90bdb8d444285e427092face3bb16da01
Problem: mysqldump_bugs.test failing on release builds
Fix: replace --source include/have_debug.inc by mysql_have_debug.inc
Change-Id: I151b69dc0da1c5d36714b453ba51349e5057564b
…e failing

Symptom:
Test failing on Linux when run with --mem and --no-skip.

Cause:
Warning in error log due to lack  of O_DIRECT support in tmpfs;
--no-skip causes have_odirect.inc to have no effect.

Solution:
have_odirect.inc added to excludenoskip.list will cause the test
(and all other tests including it) to be skipped when O_DIRECT is
not supported, regardless of --no-skip.

Change-Id: I2ef0e9939567f03db93dabd264c4590ed7c6c759
X Plugin documentation
https://dev.mysql.com/doc/dev/mysql-server/latest/mysqlx_protocol_notices.html
mentions "SLAVE" in a few places.

Change
======

- s/slave/replica/
- s/master/replication source/

Change-Id: I672d2ecda2556daac0b3a61b903ab484ae9a8c50
Fixed to use strtoul() instead of std::stoul.

Change-Id: Ic108b8e559c20cb88d4db3963cf3ca147c17189e
Problem: SET offline_mode=on could hang and make system unresponsive
Root cause: With many connections SET offline_mode could end up writing
so much to the TP notify descriptor that send() would block. If this
happens when the TP waiting_thread is waiting for a mutex held by SET
offline_mode a deadlock would result.

Solution: Add a flag which is set when the notify descriptor is written
to, and cleared by the waiting thread when it drains (reads all
available bytes) the notify descriptor. As long as the flag is set new
kill notifcations do not actually write to the notify descriptor. That
way the capacity of the notify descriptor is not exceeded.
Additional writes before the notify descriptor has been drained are
redundant since the waiting_thread iterates over
all connections in the thread group to identify those that are killed.

Increase the number of connections allowed by mysqltest, to be able to
create the required number of connections.

Change-Id: I8c84c0b9c9e5a4e2312fd6758e07eebc18e7c7a4
Problem: SET offline_mode=on could hang and make system unresponsive
Root cause: With many connections SET offline_mode could end up writing
so much to the TP notify descriptor that send() would block. If this
happens when the TP waiting_thread is waiting for a mutex held by SET
offline_mode a deadlock would result.

Solution: Add a flag which is set when the notify descriptor is written
to, and cleared by the waiting thread when it drains (reads all
available bytes) the notify descriptor. As long as the flag is set new
kill notifcations do not actually write to the notify descriptor. That
way the capacity of the notify descriptor is not exceeded.
Additional writes before the notify descriptor has been drained are
redundant since the waiting_thread iterates over
all connections in the thread group to identify those that are killed.

Increase the number of connections allowed by mysqltest, to be able to
create the required number of connections.

Change-Id: I8c84c0b9c9e5a4e2312fd6758e07eebc18e7c7a4
Change-Id: I733cc835b1617920e8700d7ce0fc3fb979069963
Change-Id: I0baedae7b84b8dd4a5800c14f3e715052071e65e
Change-Id: Ica35d9293a67a51c803f67381a933124d7470564
Post push fix. Remove compiler warnings when building for MySQL 5.7,
MySQL Cluster 7.5 and 7.6.

Change-Id: I5e8decbf776f8a1f4171dd3beae257235418756f
Change-Id: Iab626a487ddf444979d812f50a6ff9e8405c2882
Change-Id: I3adc25f4e53201b02dac7b68f7740472f6f7119b
Post push fix. Remove compiler warnings when building for MySQL 5.7,
MySQL Cluster 7.5 and 7.6.

Change-Id: I5e8decbf776f8a1f4171dd3beae257235418756f
…n_cols thread 1406284440

Issue:
An assert in storage/innobase/include/dict0dict.ic
triggers, indicating an invalid column index, while the column
index is in fact correct.

Background:
The column index provided to dict_table_get_index_on_first_col
is an index among all columns, while the n_cols field of
dict_table_t only countains the number of non-virtual columns.
Thus, the assert could be triggered with correct arguments.

Fix:
Changing the assert to compare against the n_t_cols field, the
total number of columns.

Change-Id: I90cc8c07d428c9117ca7873e9f53752d50f800e6
Approved-by: Balasubramanian Kandasamy <balasubramanian.kandasamy@oracle.com>
Approved-by: Balasubramanian Kandasamy <balasubramanian.kandasamy@oracle.com>
Approved-by: Balasubramanian Kandasamy <balasubramanian.kandasamy@oracle.com>
Approved-by: Balasubramanian Kandasamy <balasubramanian.kandasamy@oracle.com>
Approved-by: Balasubramanian Kandasamy <balasubramanian.kandasamy@oracle.com>
Change-Id: I2dfd8d9b84724dc820a098fcbd65a4d9f2d49a1c
Change-Id: I948602390fd4848b45dc99c528eab87b912db706
lukin-oleksiy and others added 24 commits September 3, 2024 14:27
https://perconadev.atlassian.net/browse/PS-9293

***
New release just added SQL echo of prepared statement. Test results
re-recorded
Merge fixes for PS-9144 and PS-9214 into 8.0 release tree.
After Upstream's fix for bug#35952353 "SELECT COUNT(*) degraded performance on
8.0 compared to 5.7" (mysql/mysql-server@ffad6cf)
MySQL stopped using parallel read for COUNT(*) evaluation when secondary index
scan is used.

As result innodb.parallel_read_kill test started to fail, as it employed
COUNT(*) that used secondary index scan to test InnoDB parallel read
killability. The parallel read code is simply no longer invoked causing
DEBUG_SYNC time out.

This patch fixes the failure by forcing the test to use COUNT(*) that
uses primary key scan, which is still processed using parallel read,
instead.
PS-9293, PS-9328: Fix innodb.parallel_read_kill test failures (8.0 version).
https://perconadev.atlassian.net/browse/PS-9286

extra/libkmip head is at baf6832 with activate operation implementation.
keyring_kmip/backend/backend.cc updateed to execute op_activate after key
registration.
…lect Sys_var_tz::session_value_ptr)

Reverting Percona Server's version of the fix which is no longer necessary.
Upstream fixed this issue in a different, more complex, but also probably more
clean way in their fix for bug#31168097 "SIG 6 ASSERTION `0' IN
TIME_ZONE_UTC::GET_NAME AT TZTIME.CC:1182".

See: mysql/mysql-server@1f8c02e

The test case for this issue is kept around.
PS-9293, PS-9328: Revert of fix for PS-7221: Bug #100402 (crash on se… (8.0 version).
…il.cc:13097:fil_system->shard_by_id(id)->mutex_owned().

Reverted Percona Server's fix for PS-7666 as it became redundant
after Upstream's has fixed the problem in a different way.
See the Upstream's fix for bug#32819101 "ASSERTION FAILURE: FIL0FIL.CC:
12089:FIL_SYSTEM->SHARD_BY_ID(ID)->MUTEX_OWNED()" for details:

4a2cf86

Removed Percona's test case as well, as Upstream fix comes with one.
PS-9293, PS-9328: Revert of fix for PS-7666: Assertion failure: fil0f… (8.0 version)
https://perconadev.atlassian.net/browse/PS-9378

This patch was originally created for the 8.4 branch where the only
available binlog transaction dependency tracking mode is 'WRITESET'.
However, it also makes sense to backport this to 8.0 as the same
code paths are used when user explicitly set the
'binlog_transaction_dependency_tracking' system variable to 'WRITESET'
(by default in 8.0 series, this variable is set to 'COMMIT_ORDER').

Cherry-picked PS-9302
"Improve performance for binlog_transaction_dependency_tracking=WRITESET"
(https://perconadev.atlassian.net/browse/PS-9302) from 8.4.

Problem:
Comparing to 8.0 where the default value of
binlog_transaction_dependency_tracking was COMMIT_ORDER, 8.4 introduces
a visible write performance drop.

Cause:
8.4 uses WRITESET dependency tracking. For this tracking we maintain
a map of binlog_transaction_dependency_history_size for row_id to newest
transaction sequence_number which modified a given row. Every new
transaction needs to check its dependency by examining the map (find),
which is done with logarithmic complexity.

Solution:
Change std::map to std::unordered_map. This allows us to find a row
dependency in constant on average complexity.
…et_perf_backport

PS-9378: Backport PS-9302 (WRITESET perf improvements) to 8.0 (8.0)
…n rocksdb_stress.drop_cf_stress tests.

Disabled two tests which create too much noise in Jenkins and which we
could not/do not plan to fix immediately.

binlog.binlog_mysqlbinlog_4g_start_position test requires too much memory/
space so often fails in Jenkins even when run alone.

rocksdb_stress.drop_cf_stress test executes concurrently mix of DDL and
often fails due to race.
…nent (#5407)

https://perconadev.atlassian.net/browse/PS-9379

Fix of problem with -latomic in uuid_vx component of some platofrms.
CMakeLists.txtx for the component now check code sample compilation
and adds "-latomic" if required. Note! On RHEL platforms the package
"gcc-toolset-12-libatomic-devel" must be installed as compile dependency.
PS-9293, PS-9328: Disabled binlog.binlog_mysqlbinlog_4g_start_position (8.0 version)
… has 10K tables or more

https://perconadev.atlassian.net/browse/PS-9306

Upstream fixed the problem in 8.0.39 release but no testcase is attached
to the bug fix.

This commit introduces a mtr testcase. It creates 8192 tables,
restarts the server and it should come up
… and leaves orphan truncate log files

https://perconadev.atlassian.net/browse/PS-9322

Backported Oracle fix for Bug #35784192
"tablespace is missing for table innodb_undo_002"
(commit mysql/mysql-server@bda098a and mysql/mysql-server@97f4285)
from 8.4 to 8.0.

Bug#35784192 tablespace is missing for table innodb_undo_002

This bug reports following two problems:

Querying the information schema reports undo tablespace is missing -
  while an undo file is present.
Purge threads don't remove the undo_space_number_trunc.log file.

While the first problem is already fixed through Bug#32104924, Bug#32654667

This bug fix aims to address the second problem.

The purge thread attempts to lock dictionary tables, while truncating
the undo tablespace. This lock can't be acquired in
a read-only MySQL instance, except when the thread has some special
flags associated with it. In this case, we use the skip_readonly_check
member of the thread descriptor struct, to signal that the readonly
check should not be applied to the purge thread. This is done
via the set_skip_readonly_check() method.

Change-Id: Iea6255ff6654752db8658bdd422cc3e58e3a8a97

Bug#35784192 tablespace is missing for table innodb_undo_002 [postfix]
Remove MySQL versions number from MTR test.

Change-Id: I3973435edbbc8bf442846c617ade4b5b824bb8c2
…runcation_super_read_only

PS-9322 fix: With super_read_only=1, undo truncation cannot update DD and leaves orphan truncate log files (8.0)
PS-9306: MySQL 8.0.38, 8.4.1 and 9.0.0 crashes on restart if database…
@adivinho
Copy link
Contributor Author

adivinho commented Oct 8, 2024

@dlenev Could you check the conflict?

@dlenev
Copy link
Contributor

dlenev commented Oct 8, 2024

@adivinho Resolved.

@adivinho adivinho merged commit 66b2a91 into 8.0 Oct 22, 2024
10 of 56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.