Split router/cmake/configure.cmake to allow reuse.

Change-Id: I4f4ca13729767e14ff75d0d915b124596ed1e132

               index scan

Description:
- Indexes are ordered based on their keys. Loose
  index scan effectively jumps from one unique value
  (or set of values) to the next based on the index’s prefix
  keys.

- To “jump” values in an index, we use the handler
  call: ha_index_read_map().

- the first range read sets an end-of-range
  value to indicate the end of the first range.

- The next range read does not clear the previous end-of-range value
  and applies it to the current range.

- Since the end-of-range value has already been crossed in the
  previous range read, this causes the reads to stop. So the
  iteration is finished with the current range without moving
  onto the next range(unique set of values)resulting in an
  incorrect query result.

Fix:
- In order to find the next unique value, the old end-of-range
  value is cleared.

Change-Id: I84290fb794db13ec6f0795dd14a92cf85b9dad09

Problem: mysqldump not sanitizing the version string obtained from
server which may lead to injecting malicious commands to the output.

Fix: added function sanitizing the version string by cutting off
illegal part and issuing warning.

Test: check the server version in the output with and without injected
payload.

Change-Id: I1f19e1c90bdb8d444285e427092face3bb16da01

In jtie_gcalls.hpp, macros are used to create lists and templates.
Many of these could stringify or concatenate up to 20 parameters,
but capacity beyond about 13 is not needed.

This patch shortens the lists. Clang will sometimes print a
"rejecting template" error for each candidate template not chosen,
so this helps to reduce the number of error messages generated
before the compiler finally gives up.

Change-Id: Iddbb25d0a6e5740815b686259c734a0bbf434d8d

An earlier version of jtie_gcalls generated two distinct templates
for gcall_mfv() and gcall_mfr() to map the const and non-const
variants of otherwise identical method calls.

This strategy was abandoned because some very old C++ compilers could
not handle the macro invocations that were used, and because
Microsoft's compilers considered the two templates to be equivalent.

Microsoft compilers still behave the same way, but Clang requires
distinct templates. This patch restores the earlier design, but
only in non-Microsoft environments.

Change-Id: I6694c7cdc096a6675e71bcd843fec99421367c70

In jtie_tconv_object_impl.hpp, add a missing specialization
of Target<J, C> for const C.

Fix an incorrect mapping in myjapi_classes.hpp for
C0::check().

Then undefine JTIE_USE_WRAPPED_VARIANT_FOR_FUNCTION, and
compile and run the tests (use "ctest -R jtie").

Change-Id: I5911bb6ef9ea149c2242378d39956a4d4afaab84

The jtie mapping for NdbOperation::getNdbErrorLine() should be marked
as const-overloaded.

The jtie mapping for Ndb_cluster_connection::set_service_uri() was
declared as void, but should actually return int.

The mappings for three const methods were defined to use the _t
variant of the object trait type, but need to use the _ct variant.

Change-Id: I4ab9e7a093c74edef72e2141d701e2e6c32b8bb6

In ndbjtie_defs.hpp, remove the "XXXXX temporary, for testing" comment
from 2011, and do not define NDBJTIE_USE_WRAPPED_VARIANT_FOR_FUNCTION
or NDBJTIE_USE_WRAPPED_VARIANT_FOR_OVERLOADED_FUNCTION.

This will enable use of the unwrapped mappings for most of the NDB API
on all platforms.

Change-Id: Id132500995cbaee62e7c1e5aa40f9272613501bb

Remove most JTie wrappers. They are no longer needed.
The only remaining method wrappers are a small set for
overloaded methods that have both const and non-const
versions.

Use "unifdef -UNDBJTIE_USE_WRAPPED_VARIANT_FOR_FUNCTION" and
"unifdef -UNDBJTIE_USE_WRAPPED_VARIANT_FOR_OVERLOADED_FUNCTION"
to remove wrapped variants from the ndbapi and mysql_utils jtie
header files.

Change-Id: Ie08c2242aed2bfda02defe76fef40bd0bad6e9e7

In MySQL Cluster 8.0.31 several issues were introduced in
ndb_redo_log_reader. Below fixes have been done:

- For backward compatibility the legacy options with single minus for
  example -lap are now allowed after file argument.

- One can now provide the mandatory numeric value for --mbyte/-mbyte,
  --page/-page, --pageindex/-pageindex options.

- Option --mbyte now allows up to 1023 instead of 15 as value since we
  support fragment redo log file sizes up to 1GB.

- When using --mbyte option it starts reading the file from given MB,
  previously it started from 4 times the given value.

- Now it reads data only once, before it stepped back 3 quarter of a MB
  for every MB read and read it again.

- Remove cast-qual warning in convert_legacy_options in reader.cpp.

Added new test ndb.ndb_redo_log_reader.

Change-Id: Iadfabcab9532eddf1314aeb618c72c31ab7b4a6f

…mented.

ndb_redo_log_reader had several issues for example it failed on valid
files with:

  Record type = 0 not implemented.
  Error in redoLogReader(). Exiting!

And also warned about wrong checksums.

The following changes were done:

- Ignore unused (zero filled) pages.
- Allow partial (header only) pages.
- Treat checksum value 37 as no checksum disabled. Which is the case for
  release build. Checksum is only done in debug build.
- Show mbyte up to 1023 rather than up to 15 in help, still no check.
  We support fragment log file size up to 1GB.
- Spell FfragmentId as FragmentId.
- Fix cast-qual warning when using PrepareOperationRecord::m_keyInfo.

Change-Id: I40304555488fcfb1b3f0949096c61fdfff6171c0

…mented.

ndb_redo_log_reader had several issues for example it failed on valid
files with:

  Record type = 0 not implemented.
  Error in redoLogReader(). Exiting!

And also warned about wrong checksums.

The following changes were done:

- Ignore unused (zero filled) pages.
- Allow partial (header only) pages.
- Treat checksum value 37 as no checksum disabled. Which is the case for
  release build. Checksum is only done in debug build.
- Show mbyte up to 1023 rather than up to 15 in help, still no check.
  We support fragment log file size up to 1GB.
- Spell FfragmentId as FragmentId.
- Fix cast-qual warning when using PrepareOperationRecord::m_keyInfo.

A new test ndb.ndb_redo_log_reader was added.

Change-Id: I40304555488fcfb1b3f0949096c61fdfff6171c0

Change-Id: Icc7257a29ebcf899b6809238ca1bcb5ef8fc61fc

Change-Id: I246b76d3bb974734edd426223649e6f25eef5ffc

Support for encryption was added to ndb_redo_log_reader tool.

Change-Id: If28dc314c2fab732474447db68e1a338248695e7

…nk Windows

Problem: Slow Windows machines cause unexpected GR errors during tests:

"There was an error when connecting to the donor server.."
"For details please check.."
Tests lacked these error checks, leading to false positives.

Solution:

Added suppressions for error messages in the test cases.

Change-Id: I9d54f2a8190cdd61ecf56209ac567ceb765133ba

This commit backports the fix to 8.0

This patch will solve the following duplicates of this bug:
  Bug #112425: trx_t might be Use-After-Free in innobase_commit_by_xid
  Bug #99643:  innobase_commit_by_xid/innobase_rollback_by_xid is not
thread safe
  Bug #105036: trx would be used after free in `innobase_commit_by_xid`
and rollback

Background:
  TrxInInnoDB is a RAII wrapper for trx_t object used to track if the
transaction's thread is currently executing within InnoDB code. It is
acquired on all entry points, and as Innodb can be entered "recursively",
the trx->in_depth is used to track the balance of enters and exits.
  On the outermost enter, the thread additionally checks if
trx->in_innodb has the TRX_FORCE_ROLLBACK (0x8000 0000) flag set, which
means a high priority transaction is attempting an asynchronous rollback
of this transaction, so to avoid races, this thread should wait for the
rollback to complete.

Issue:
  TrxInInnoDB's destructor calls exit which resets in_depth and
in_innodb increased by enter. However innobase_commit_by_xid and
innobase_rollback_by_xid calls trx_free_for_background which returns the
trx back to the pool, before the destructor is called. If this trx is
being reused by another thread, it can lead to data-race and corrupted
value of in_depth and in_innodb. If in_depth gets the value of -1,
subsequent calls to enter and exit will bump in_innodb by one. This can
lead to indefinite wait if in_innodb reaches TRX_FORCE_ROLLBACK.

Fix:
  Ensure that TrxInInnoDB calls exit before returning the trx object to
the pool. Further add checks to catch corrupt values of in_depth when
freeing trx. Trx state validation before free was missed in
trx_free_prepared_or_active_recovered

Thanks to Shaohua Wang (Alibaba, Ex-Innodb) for the contribution

Change-Id: Ibf79bec85ffa0eaf65f565c169db61536bff10a2

ndb.ndbinfo_plans result

Problem:
ndb.ndbinfo_plans have failed been seen to fail as below five times since
test part for WL#11968 was introduced January 2022.
Failures have mostly been on Windows but also on linux with ASAN.

    CURRENT_TEST: ndb.ndbinfo_plans
    --- .../mysql-test/suite/ndb/r/ndbinfo_plans.result
2024-02-11 06:00:21.000000000 +0300
    +++ "...\log\ndbinfo_plans.reject"
2024-02-11 11:58:09.260695300 +0300
    @@ -297,7 +297,6 @@
     2
NDB$BLOBEVENT_REPL$mysql/ndb_schema_3
NDB$BLOB_4_3
     3
REPL$mysql/ndb_schema_result
ndb_schema_result
     4
ndb_index_stat_head_event
ndb_index_stat_head
    -5
REPL$mysql/ndb_apply_status
ndb_apply_status

     ## Query uses primary keys on both tables:
     EXPLAIN SELECT event_id, e.name, table_name FROM events e

Analysis:
Test assumes that event_id are constant but that can't be assumed, for
example when prior test has recreated tables or events.

Solution:
Rewrite to find list of event_ids by table name which are known to be
stable identifier.

Change-Id: I2ba83b0a715ea2ca2c5a75dba4e1e8ea635391c1

…t one redo file is missi

After calling rename(..) on POSIX platform one should fsync the
(destination) directory node to make sure change is persisted to disc.
Redo Log module was attempting fsync("data-dir/") instead of
fsync("data-dir/#innodb_redo/").
This could manifest as data loss and InnoDB not being able to start up,
if a power-outage (or OS crash) would happen right after rename, and
before the change to directory got persisted to disc on its own pace.
Similar problem could occur if a snapshot of the file system was taken
at that time.

The way this bug would manifest in practice is that when InnoDB needs a
new redo log file with number 280 (because 279th has become full of
data), what it does is the following (see log_files_produce_file()):

1. create "#ib_redo280_tmp" (using open(..) syscall)
2. resize "#ib_redo280_tmp"  (using fallocate(..) + fsync/fdatasync())
3. close("#ib_redo280_tmp")
4. open("#ib_redo280_tmp")
5. write the correct headers to "#ib_redo280_tmp" (using pwrite(..))
6. because we use a RAII Log_file_handle which notices the file was
   modified by writes, calls fsync()/fdatasync() before calling close()
   on the handle
7. rename "#ib_redo280_tmp"  to "#ib_redo280" (using rename() syscall)
8. fsync() or fdatasync() the parent directory which contains the
   renamed file (using a handle obtained from open(directory))
9. mark "#ib_redo279" as full (using open(..), pwrite(...) )
   (note the lack of fsync. We are ok with persisting it "later",
   but it shouldn't be "too early")
10. close("#ib_redo279")

The problem is that in step 8. we fsync wrong directory, so during
recovery the InnoDB would notice that the redo log file with the
largest number, #ib_redo279, already has LOG_HEADER_FLAG_FILE_FULL
indicating that it was full, which implies a new file with even larger
number, #ib_redo280, should have been created (which involves rename)
to accommodate new data, yet this newer file doesn't exist.
InnoDB woud fail with an error:

[ERROR] MY-013893 [InnoDB] Found existing redo log files, but at least
one is missing. It is unknown if recovery could reach physically
consistent state. Please consider restoring from backup or providing
--innodb-force-recovery > 0.

This patch fixes the bug, by calling fsync on the correct directory.

Change-Id: I87f4eb23c60b0d82876789069d1a6716734815ac

This patch revert commit c6953232a38895aae60c37563e12e4476e1d8ade

Change-Id: I2c08dbd279958333b86e938cc120b64f5c1c1c5d

Problem: mysqldump_bugs.test failing on release builds
Fix: replace --source include/have_debug.inc by mysql_have_debug.inc
Change-Id: I151b69dc0da1c5d36714b453ba51349e5057564b

…inux

When building with LTO we should use gcc-ar/gcc-ranlib.
Look for these versions regardless of LTO flags.

Change-Id: I9da5b30bfbd01d4188e8f21721c26fe027e25762
(cherry picked from commit b88d1f86cd768b5555b2da899cbd07e309a156f6)

We have dropped support for some obsolete Linux distros,
so we can bump the CMAKE_MINIMUM_REQUIRED VERSION

This allows us to use more modern cmake features.

Change-Id: I7a37a0f25eff72c00457136dc3b5ee3956b3a139

Post push fix. The mysqltest command replace_result does not work on
mysqltest command echo.

Change-Id: Iaf81824f723e0ec2cdf9c306f944ed0285b4db46

Some BuildRequires: rules have not been maintained properly.
Update version required for cmake and bison.

In the patch for 8.0, changed to:
-BuildRequires:  cmake >= 3.14.6
+BuildRequires:  cmake >= 3.11.2

Change-Id: I28041799054bfbc3653e73cfc919aacf4e2b7f7f
(cherry picked from commit 33f29fdce4fa3fb71373bbfdf7f27a51514b8009)
(cherry picked from commit b0700d0e91abab644515826e156b4070f9de899f)

…tication plugin

When the Router is bootstrapped it checks if the metadata user is not using
unsupported mysql_native_password plugin. First part of this check is query
of the users table for the host and plugin. If this fails the procedure is
skipped but the user sees an error messsage as an bootstrap output. Meanwhile
this can be no error scenario. For example the user that is used to bootstrap
can have no privileges to query users table. This will be the case if the user
created by the Shell is used.

This patch removes the confusing error message from the bootstrap output.

Change-Id: Ic9509a57f5886747f2fe401e566b9f0e2a3c6bd5

Disallow system variable updates during shutdown.

Change-Id: I6f259cd8eda8c6ac662573e68bfb17ddc386d1c3

               PERFORMANCE_SCHEMA IS OFF

Analysis:
---------
When performance_schema is set to off, MEB fails with error
"Unexpected number of rows from MySQL query" while taking a
backup.

The tables innodb_redo_log_files & keyring_component_status
are not available when performance schema is disabled on the
server. When MEB tried to query these tables, it gets empty
set as the result and hence it fails.

Fix:
----
The fix is to make these tables available even when
performance_schema set to OFF or disabled on the server.

Change-Id: I0f5fa1b3293733b517f5f19f1dcdebffc651c061

It was reported that the SQL function
`group_replication_set_as_primary()` after successfully set a new
primary remained waiting for the operation completeness.

On the occurrence and implementation analysis, after ruling out
other hypothesis, the only probable cause for the issue is a
missed/not_handled broadcast on a condition wait call.
More precisely on:
```
  mysql_mutex_lock(&group_thread_end_lock);
  while (action_running && !coordinator_terminating) {
    DBUG_PRINT("sleep",
               ("Waiting for the group action execution process to terminate"));
    mysql_cond_wait(&group_thread_end_cond, &group_thread_end_lock);
  }
  mysql_mutex_unlock(&group_thread_end_lock);
```
which can cause the coordinator of the SQL function
`group_replication_set_as_primary()` to wait uninterruptedly.

To avoid the above issue, the `mysql_cond_wait()` function was
replaced by `mysql_cond_timedwait()` which will periodic check if
the predicate to wait still stands or if the wait can end.

Additionally, a superfluous call to `mysql_cond_broadcast()` was
deleted.

Change-Id: I41fa0f1b3f3ee644e19b916814837ffe5232e265

 set after NDBAPI test run

Error insertion improvements

- Have each block log when an error insertion is set
  - Indicates previous and new value
  - Gives visual indication of when error insert is
    set in log
  - Info for debugging problems

- Add new ERROR 1 code which can be used to check that
  Error inserts are cleared, e.g. at the end of a test

- Improve setting + clearing of error insert and extra
  value together.

Change-Id: I18ee88a9a02cec7349a39a85aaec6d03f8df4002

NDBAPI test run

Problem:
Some NDBAPI test do not clear active error insert injected to
data nodes during test execution. This can have an unpredictable
impact on test cases run afterward.

Solution:
New function, runCheckNoErrorInserted(), called at the end of every
NDBAPI test case to check if there are currently error insert set in
data nodes. Function uses the special error insert code '1' that
makes data node crash if ERROR_INSERT_VALUE or ERROR_INSERT_EXTRA
is set.

Change-Id: I977d8881bae2f6d0d3d5090b0302176c69ae01c4

Problem:
Currently there is a mix between CMVMI and TRPMAN error insert handling.
Part of error insert values handled in TRPMAN were originally
handled in CMVMI but, when TRPMAN was introduced and all transporter
stuff was moved to TRPMAN, they were moved from CMVMI to TRPMAN as well.
Also, as part of the moving of transporter handling to TRPMAN, part of
the CMVMI error code range was reserved to TRPMAN [9500, 9899] causing
CMVMI to have, in fact, two different error range [9000, 9499]
and [9900, 9999].

This mix of error insert handling in both CMVMI and TRPMAN is
undesirable because in some cases signals are, unnecessarily, sent to
CMVMI and then, re-sent to TRPMAN and, in addition, it can cause
difficulties reading error insert code.

Solution:
- TRPMAN error insert handling decoupled from CMVMI.
- Error range for CMVMI and TRPMAN redefined as:
   - TRPMAN [9000, 9599]
   - CMVMI  [9600, 9999]
- All existing error insert code preserved with same behaviour as before.
- Existing error in the [9000, 9599] range, handled in CMVMI, moved
to TRPMAN.
- Handling of dump 9006 moved to TRPMAN as well.

This patch also fixes testNodeRestart -n Bug24717.
Test dumps code 9002 to data node. TRPMAN sets error 9002 in self
block, but it never hit because handling of error 9002 is done in
CMVMI instead TRPMAN. (It was probably missed in the migration of
transporter stuff from CMVMI to TRPMAN).
Fix:
execNODE_START_REP Implemented in TRPMAN block so error 9002 can be
handled as expected.
Also, TRPMAN block added to all_blocks list in NDBCNTR in order
to make NODE_START_REP to be sent to TRPMAN instances.

Change-Id: Ibf56def74818f5043b43c40e39ecdaa1e4e02974

Problem:
Some testBackup test cases injects error 10099, but it
never hit. 10099 is not handled anywhere in data node code.
This makes tests fails because Error in never cleared.

Solution:
Do not inject error 10099 since it is useless.

Change-Id: I0644c372919bbd3deeeb07fada25d41fc8ca2396

Test injects error 8064 to delay LQH_TRANSREQ during TC takeover.
Error is consumed by one DBTC instance and then cleared. Since there
can be more than 1 TC instance test can fail due to error 8064 being
left in other DBTC instances.

Fix: clears EI from test side to prevent leftovers when using more
than 1 TC instance per data node.

Change-Id: Ia95513d030b42577b19748ddb5f28cf4b7c42a15

…nning

Test injects error 10046 in Backup instances to force DIH to change the
next fragment to scan.
Error is injected to Backup Proxy and then proxy resends the error
to all workers where it is eventually cleared. But, in Backup proxy
instance error is never cleared.
Fix:
Clear error in test side (Insert Error 0) to force it to be cleared
in all Backup instances (including Backup proxy).

Change-Id: Ieafdde52bf19c6d3946d6d53dd26eab9dc329526

…unning

- testBackup -n FailMaster T1
   Test injects 10038 into backup block.
   Error hits but backup block does not clear it.
   Fix: clear 10038 from test side.

- testBackup -n NFMasterAsSlave T1
- testBackup -n FailSlave T1
   Test injects 10036 into backup block.
   Error hits but backup block does not clear it.
   Fix: clear 10038 from test side.

Change-Id: If74dbcfedd117d460d3c03e46d0a308cbd1cac8f

…unning

Test injects error 8039 to DBTC block.
Error is consumed by one DBTC instance and then cleared. Since can
be more than 1 TC instance test can fail due to error 8086 being
left in other DBTC instances.

Fix: EI cleared from test side to prevent leftovers when using more
than 1 TC instance per data node.

Change-Id: I3dc7a05f80ff8a2206be7040b176c80ef139c2c3

Error Inserted after running

Test injects error 10048 to force number of parts of a partial
LCP to 1.
Error is consumed by one Backup instance, but it is never cleared.
Test can fail due to error 10048 being left in Backup instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I3c12564cd3cab4bd2869a4490cb9166903508c91

after running

Test injects error 8048 and 8049 to make TC not choose own node for
simple/dirty read.
Error is consumed by one DBTC instance, but it is never cleared.
Test can fail due to error 8048 being left in Backup instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I6aa3a4dbc0bfa057e50f5dd752b1f574963e8447

Test cases ExhaustSegmentedSectionPk, ExhaustSegmentedSectionScan
and ExhaustSegmentedSectionIx injects error 8068 to free all
segments hoarded by errors 8065, 8066, 8067 sent earlier.
Error is consumed by one DBTC instance and then cleared. Since can
be more than 1 TC instance test can fail due to error 8068 being
left in other DBTC instances.
Fix: EI cleared from test side to prevent leftovers when using more
than 1 TC instance per data node.

Change-Id: Ie0d89636b523811a4c54ee3ece06f00cbd237192

Test cases SR1 and SR1_O, injects error 5020 to force system to
read pages form file when executing prepare operation record.
Error is consumed by one LQH instance, but it is never cleared.
Test can fail due to error 5020 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I06baa010b172c0e97f9c9dcbf4e6b882fb58f66d

Test case SR1, injects error 5020 to force system to read
pages form file when executing prepare operation record.
Error is consumed by one LQH instance, but it is never cleared.
Test can fail due to error 5020 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: Iada0b78d0651c2403a1817500ed5918608f51ce0

Test case Bug54611 injects error 5055 to force LQH to abort
a fragment scan during node restart.
Error is consumed by one LQH instance and then cleared.
Since can be more than 1 LQH instance test can fail due
to error 5055 being left in other LQH instances.
Fix: EI cleared from test side to prevent leftovers whe
using more than 1 LQH instance per data node.

Change-Id: I0caf8ee277a308e463145f3504f1830b475da9dd

Test case Bug22696 injects error 7072 to split START_FRAGREQ into
several log nodes.
Error is consumed by one DIH instance, but it is never cleared.
Test can fail due to error 7072 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: Ia8a69ffa1e9caec7ea06071d4876d352d90fa446

Test cases CreateAndDropAtRandom and CreateAndDropIndexes injects
error 4013 to make TUP verify table descriptor.
Error is consumed by one TUP instance, but it is never cleared.
Test can fail due to error 4013 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I4e9df5d8494a34b1059e36e7675450ec231cf7a7

after running

Test cases DropTableConcurrentLCP and DropTableConcurrentLCP2
injects error 5088 and 5089 to delay a drop table.
Error is consumed by one LQH instance, but it is never cleared.
Tests can fail due to error 5088 or 5089 being left in LQH
instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I7f47c674cfa1062cbb6c5c8566dda29e7fa772d4

Test cases TableAddAttrsDuringError injects error 4029 to
make an alter table fail.
Error is consumed by one TUP instance and then cleared.
Since can be more than 1 TUP instance test can fail due
to error 4029 being left in other TUP instances.
Fix: EI cleared from test side to prevent leftovers when
using more than 1 TUP instance per data node.

Change-Id: Ic707e4c81bc7c72c5269cfac181c858fa9c38639

Test case Bug54945 injects error 5057 to force a fragment
scan to fail due to many active scans.
Error is consumed by one LQH instance and then cleared.
Since can be more than 1 LQH instance test can fail due
to error 5057 being left in other LQH instances.
Fix: EI cleared from test side to prevent leftovers when
using more than 1 LQH instance per data node.

Change-Id: Ice3a6fc2dc7a5f1e69e4423e3fabf897bd67c98f

Test case TupCheckSumError injects error 4036 to simulate
a tuple corruption detection.
Error is consumed by one TUP instance and then cleared.
Since can be more than 1 TUP instance test can fail due
to error 4036 being left in other TUP instances.
Fix: EI cleared from test side to prevent leftovers when
using more than 1 TUP instance per data node.

Change-Id: Ic1f5f9c6f7092b77c6a60f8b56aac2dedce81955

Error is injected to LQH Proxy and then proxy resends the error
to all workers where it is eventually cleared. But, in LQH proxy
instance error is never cleared.
Fix:
Clear error in test side (Insert Error 0) to force it to be cleared
in all LQH instances (including LQH proxy).

Change-Id: Icb06ee1b360fa424a1c972ec799749891b2f0f22

…er running

Error is injected to TRPMAN Proxy and then proxy resends the error
to all workers where it is eventually cleared. But, in TRPMAN proxy
instance error is never cleared.
Fix:
Clear error in test side (Insert Error 0) to force it to be cleared
in all TRPMAN instances (including TRPMAN proxy).

Change-Id: I7a024dca555c8c684e589c62ea2963722b7531a7

 set after NDBAPI test run

Error insertion improvements

- Have each block log when an error insertion is set
  - Indicates previous and new value
  - Gives visual indication of when error insert is
    set in log
  - Info for debugging problems

- Add new ERROR 1 code which can be used to check that
  Error inserts are cleared, e.g. at the end of a test

- Improve setting + clearing of error insert and extra
  value together.

Change-Id: I18ee88a9a02cec7349a39a85aaec6d03f8df4002

…bled

    PROBLEM
    --------
    1. Start the mysqld server innodb_redo_log_encrypt=ON;
    2. Shutdown the server.
    3. Delete the encryption key.
    4. Start the server again with innodb_redo_log_encrypt=OFF;
    5. The server is unable to start with the error message
       that the redo log is encrypted.

    ANALYSIS
    --------

    1. This is a known limitation which is documented.
    2. During the startup the server reads the redo log file
       header to determine if the redo log file containing the
       the latest checkpoint is encrypted or not. This redo log file
       can contain both encrypted and unencrypted data.
    3. If the key ring is missing the startup fails since the header
       says encrypted data present in the file, but we cannot decrypt
       it without key.
    4. It is to be noted that setting innodb_redo_log_encrypt=OFF
       dynamically or doing the startup doesn't change the header
       information in the redo log file, only subsequent new redo
       log file header will be created without encryption.
    5. The redo logs are written to the disk in blocks of 512 bytes
    6. When redo log encryption is ON, based on the setting of
       innodb_log_write_ahead_size we may write empty blocks at the
       end which are encrypted also.
    7. When the user sets innodb_redo_log_encrypt=OFF and then
       does a slow shutdown the block containing checkpoint_lsn
       is written to disk without encryption, but the empty
       block after it may be encrypted
    8. During recovery we try to read after the logical redo block
       containing checkpoint lsn we will encounter an encrypted
       empty block which causes recovery to fail.

    FIX
    ---
    1. This patch does the following things
        (i)  We will not read the encryption info from the
             redo log file header at the start of recovery
             which will allow the server to start. We only
             read it if we encounter an encrypted block
             during scanning of redo logs.
        (ii) We will not encrypt blocks having empty headers
             which will ensure that there is no encrypted
             block after the block having the checkpoint
             lsn when doing a slow shutdown.

Change-Id: I4b37f5a1997f3a2882ac64068e18b33898e871a0

The routing plugin declare the "disabled" option as a supported
configuration option even though configuring it does not have any
effect. It was added to the supported options by mistake as
a copy/paste error.

This patch removes "disabled" from the list of the supported routing
configuration options. The Router will now error out if it is
configured.

Change-Id: Ia53e9d00769ade1156813965cfe7a86c6c8d25c8

NDBAPI test run

Problem:
Some NDBAPI test do not clear active error insert injected to
data nodes during test execution. This can have an unpredictable
impact on test cases run afterward.

Solution:
New function, runCheckNoErrorInserted(), called at the end of every
NDBAPI test case to check if there are currently error insert set in
data nodes. Function uses the special error insert code '1' that
makes data node crash if ERROR_INSERT_VALUE or ERROR_INSERT_EXTRA
is set.

Change-Id: I977d8881bae2f6d0d3d5090b0302176c69ae01c4

Problem:
Currently there is a mix between CMVMI and TRPMAN error insert handling.
Part of error insert values handled in TRPMAN were originally
handled in CMVMI but, when TRPMAN was introduced and all transporter
stuff was moved to TRPMAN, they were moved from CMVMI to TRPMAN as well.
Also, as part of the moving of transporter handling to TRPMAN, part of
the CMVMI error code range was reserved to TRPMAN [9500, 9899] causing
CMVMI to have, in fact, two different error range [9000, 9499]
and [9900, 9999].

This mix of error insert handling in both CMVMI and TRPMAN is
undesirable because in some cases signals are, unnecessarily, sent to
CMVMI and then, re-sent to TRPMAN and, in addition, it can cause
difficulties reading error insert code.

Solution:
- TRPMAN error insert handling decoupled from CMVMI.
- Error range for CMVMI and TRPMAN redefined as:
   - TRPMAN [9000, 9599]
   - CMVMI  [9600, 9999]
- All existing error insert code preserved with same behaviour as before.
- Existing error in the [9000, 9599] range, handled in CMVMI, moved
to TRPMAN.
- Handling of dump 9006 moved to TRPMAN as well.

This patch also fixes testNodeRestart -n Bug24717.
Test dumps code 9002 to data node. TRPMAN sets error 9002 in self
block, but it never hit because handling of error 9002 is done in
CMVMI instead TRPMAN. (It was probably missed in the migration of
transporter stuff from CMVMI to TRPMAN).
Fix:
execNODE_START_REP Implemented in TRPMAN block so error 9002 can be
handled as expected.
Also, TRPMAN block added to all_blocks list in NDBCNTR in order
to make NODE_START_REP to be sent to TRPMAN instances.

Change-Id: Ibf56def74818f5043b43c40e39ecdaa1e4e02974

Problem:
Some testBackup test cases injects error 10099, but it
never hit. 10099 is not handled anywhere in data node code.
This makes tests fails because Error in never cleared.

Solution:
Do not inject error 10099 since it is useless.

Change-Id: I0644c372919bbd3deeeb07fada25d41fc8ca2396

Test injects error 8064 to delay LQH_TRANSREQ during TC takeover.
Error is consumed by one DBTC instance and then cleared. Since there
can be more than 1 TC instance test can fail due to error 8064 being
left in other DBTC instances.

Fix: clears EI from test side to prevent leftovers when using more
than 1 TC instance per data node.

Change-Id: Ia95513d030b42577b19748ddb5f28cf4b7c42a15

…nning

Test injects error 10046 in Backup instances to force DIH to change the
next fragment to scan.
Error is injected to Backup Proxy and then proxy resends the error
to all workers where it is eventually cleared. But, in Backup proxy
instance error is never cleared.
Fix:
Clear error in test side (Insert Error 0) to force it to be cleared
in all Backup instances (including Backup proxy).

Change-Id: Ieafdde52bf19c6d3946d6d53dd26eab9dc329526

…unning

- testBackup -n FailMaster T1
   Test injects 10038 into backup block.
   Error hits but backup block does not clear it.
   Fix: clear 10038 from test side.

- testBackup -n NFMasterAsSlave T1
- testBackup -n FailSlave T1
   Test injects 10036 into backup block.
   Error hits but backup block does not clear it.
   Fix: clear 10038 from test side.

Change-Id: If74dbcfedd117d460d3c03e46d0a308cbd1cac8f

…unning

Test injects error 8039 to DBTC block.
Error is consumed by one DBTC instance and then cleared. Since can
be more than 1 TC instance test can fail due to error 8086 being
left in other DBTC instances.

Fix: EI cleared from test side to prevent leftovers when using more
than 1 TC instance per data node.

Change-Id: I3dc7a05f80ff8a2206be7040b176c80ef139c2c3

Error Inserted after running

Test injects error 10048 to force number of parts of a partial
LCP to 1.
Error is consumed by one Backup instance, but it is never cleared.
Test can fail due to error 10048 being left in Backup instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I3c12564cd3cab4bd2869a4490cb9166903508c91

after running

Test injects error 8048 and 8049 to make TC not choose own node for
simple/dirty read.
Error is consumed by one DBTC instance, but it is never cleared.
Test can fail due to error 8048 being left in Backup instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I6aa3a4dbc0bfa057e50f5dd752b1f574963e8447

Test cases ExhaustSegmentedSectionPk, ExhaustSegmentedSectionScan
and ExhaustSegmentedSectionIx injects error 8068 to free all
segments hoarded by errors 8065, 8066, 8067 sent earlier.
Error is consumed by one DBTC instance and then cleared. Since can
be more than 1 TC instance test can fail due to error 8068 being
left in other DBTC instances.
Fix: EI cleared from test side to prevent leftovers when using more
than 1 TC instance per data node.

Change-Id: Ie0d89636b523811a4c54ee3ece06f00cbd237192

Test cases SR1 and SR1_O, injects error 5020 to force system to
read pages form file when executing prepare operation record.
Error is consumed by one LQH instance, but it is never cleared.
Test can fail due to error 5020 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I06baa010b172c0e97f9c9dcbf4e6b882fb58f66d

Test case SR1, injects error 5020 to force system to read
pages form file when executing prepare operation record.
Error is consumed by one LQH instance, but it is never cleared.
Test can fail due to error 5020 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: Iada0b78d0651c2403a1817500ed5918608f51ce0

Test case Bug54611 injects error 5055 to force LQH to abort
a fragment scan during node restart.
Error is consumed by one LQH instance and then cleared.
Since can be more than 1 LQH instance test can fail due
to error 5055 being left in other LQH instances.
Fix: EI cleared from test side to prevent leftovers whe
using more than 1 LQH instance per data node.

Change-Id: I0caf8ee277a308e463145f3504f1830b475da9dd

Test case Bug22696 injects error 7072 to split START_FRAGREQ into
several log nodes.
Error is consumed by one DIH instance, but it is never cleared.
Test can fail due to error 7072 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: Ia8a69ffa1e9caec7ea06071d4876d352d90fa446

Test cases CreateAndDropAtRandom and CreateAndDropIndexes injects
error 4013 to make TUP verify table descriptor.
Error is consumed by one TUP instance, but it is never cleared.
Test can fail due to error 4013 being left in LQH instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I4e9df5d8494a34b1059e36e7675450ec231cf7a7

after running

Test cases DropTableConcurrentLCP and DropTableConcurrentLCP2
injects error 5088 and 5089 to delay a drop table.
Error is consumed by one LQH instance, but it is never cleared.
Tests can fail due to error 5088 or 5089 being left in LQH
instances.
Fix: clears EI from test side to prevent leftovers.

Change-Id: I7f47c674cfa1062cbb6c5c8566dda29e7fa772d4

Test cases TableAddAttrsDuringError injects error 4029 to
make an alter table fail.
Error is consumed by one TUP instance and then cleared.
Since can be more than 1 TUP instance test can fail due
to error 4029 being left in other TUP instances.
Fix: EI cleared from test side to prevent leftovers when
using more than 1 TUP instance per data node.

Change-Id: Ic707e4c81bc7c72c5269cfac181c858fa9c38639

Test case Bug54945 injects error 5057 to force a fragment
scan to fail due to many active scans.
Error is consumed by one LQH instance and then cleared.
Since can be more than 1 LQH instance test can fail due
to error 5057 being left in other LQH instances.
Fix: EI cleared from test side to prevent leftovers when
using more than 1 LQH instance per data node.

Change-Id: Ice3a6fc2dc7a5f1e69e4423e3fabf897bd67c98f

Test case TupCheckSumError injects error 4036 to simulate
a tuple corruption detection.
Error is consumed by one TUP instance and then cleared.
Since can be more than 1 TUP instance test can fail due
to error 4036 being left in other TUP instances.
Fix: EI cleared from test side to prevent leftovers when
using more than 1 TUP instance per data node.

Change-Id: Ic1f5f9c6f7092b77c6a60f8b56aac2dedce81955

Error is injected to LQH Proxy and then proxy resends the error
to all workers where it is eventually cleared. But, in LQH proxy
instance error is never cleared.
Fix:
Clear error in test side (Insert Error 0) to force it to be cleared
in all LQH instances (including LQH proxy).

Change-Id: Icb06ee1b360fa424a1c972ec799749891b2f0f22

…er running

Error is injected to TRPMAN Proxy and then proxy resends the error
to all workers where it is eventually cleared. But, in TRPMAN proxy
instance error is never cleared.
Fix:
Clear error in test side (Insert Error 0) to force it to be cleared
in all TRPMAN instances (including TRPMAN proxy).

Change-Id: I7a024dca555c8c684e589c62ea2963722b7531a7

               AFTER INSTALLATION

Description:
Server does not load component variables specified in the configuration file
when the component is installed after server start up.

Cache plugin and component variables specified in the configuration file.

Fix:
Added functionality to load component variables specified in the
configuration file during component installation.

Added functionality to cache plugin and component variables
specified in the configuration file during server start up.
The server loads the cached variable values instead of rereading
the configuration file during plugin and component installation.

Change-Id: I5c1633b70daee536d9522692bde38909df894a84

… the process to end

Symptom:
MTR tests are failing on Windows on many different tests with many different symptoms after issueing `shutdown_server` command, directly or indirectly by include files.
Most failures complain about no access to files that the killed server use, in `force-rmdir` and similar, or with port being still in use, or with files being still in use during the next server instance
startup.

Root cause:
`mysqltest` `kill_process()` is not waiting for the process to fully die. Issuing `TerminateProcess()` is not enough as it is asynchronous operation.

Fix:
After `TerminateProcess()` we call `WaitForSingleObject()` to wait for the process to fully close. A new testcase is added to try to force the above scenario.

Change-Id: Ie40e4fa7ef2567281d38ab53e1fa75479abbf1d5

…ib::fatal

The `os_innodb_umask` is global variable, it can't be modified by a thread that wants to create a file with different UNIX access mode, as it will modify it for all threads (not mentioning UndefinedBehavior).
The `os_file_set_umask` is supposed to be called only once, at InnoDB initialization.
However, after `Bug #29472125 NEED OS_FILE_GET_UMASK()` it was made possible to modify it and `WL#12009 - Redo-log Archiving` used it to modify it.
Fix:
- `os_file_get_umask()` is removed
- `os_file_set_umask()` is modified so it can be called only once
- Unix-only `os_file_create_simple_no_error_handling_with_umask()` is added to allow specifying umask param.

Change-Id: I78f169cfb99704e031ea1ff758b970cd8c73240d

…Y scenario

Symptom:
Some queries with `SELECT ... GROUP BY` can be a few times slower when executed on TempTable than ones executed on Memory temporary table engine.

Root cause:
The `AllocatorState::current_block` gets allocated and quickly deallocated in case a single `Row` instance is allocated on it and released in a loop.

Fix:
The `AllocatorState::current_block` is not released when it gets empty. We release it only when `AllocatorState` is deleted, that is when the `Table` gets deleted.

Additional fixes:
`AllocationScheme` gets a new `block_freed()` method to be able to be fully responsible for managing memory usage reporting to `MemoryMonitor`.
`AllocatorState` knows the `AllocationScheme` and can on its own report usage of memory to it.

Change-Id: I64ae2387dc23b3f8d027d4050972bf126aa5d004

Change-Id: I906440eaf54d60fb8bf7a3c23859ae7fb27b0cda

MySQL Cluster no longer uses a customized MySQL server and there is no
need for re-running the non NDB server tests. Instead the NDB specific
testing is increased.

Change-Id: Ia9f9622116ecd057afcb5d8177f44cf51ef7eae5

…nit for all plugins

This patch unifies plugin's deinit function call to pass valid plugin pointer
instead of the nullptr for all types of plugin.

Change-Id: I482497bbaff28d5cd31d74d694056a4df6693152

              is not valid for CHARACTER SET

Condition pushdown to a view fails with a collation mismatch
if the view was created with a different charset than the
charset used when querying the view.
Problem is seen if the underlying fields in the view are
literals with COLLATE statements. The string literal with
the COLLATE statement is cloned when replacing expressions
in the condition that is being pushed down with the
expressions from the view. The string literal is currently
parsed with the connection charset which in this case is
different from the one that was used when the view was
created. Therefore the COLLATE statement fails.
Creation context for a view has the connection charset and
collation information which was used when the view
was created. This is currently used for parsing the view
when it is later queried. We use the same now when cloning
expressions from a view during condition pushdown.

Change-Id: Ib040b9a67ddedd5fb9bf5de6fafafb358226e9d9

NDBAPI test run

Fix compilation error due to C++11 syntax
   storage/ndb/test/include/NDBT_Test.hpp:329: error: expected ';'
before 'override'

Change-Id: Ifa4e1c594809a0467d0efd7fce1a5e425f03e112

Add test for adding index on part of primary key using inplace alter
table.

Also backport of test case from 8.0:
(cherry picked from commit fa0e0e54d6d8bb772b80aacd04e2886ad85707f3)

Change-Id: I8010caedadba618af205726a9b84faad2ffb84d6

Add test for adding index on part of primary key using inplace alter
table.

Change-Id: Id2849c1cf429ca94317dbad89067b9d2c6e850fb

Stop ndb_76_inplace_upgrade test from running in valgrind.

This has already been done for all other MySQL Server tests, see
Bug29520374 CLEAN UP THE VALGRIND TESTING.

Change-Id: I3d4d63b3570875a248aeea97c52c10abc1aebbf5

[ 79%] ndb.server_lifecycle                     w7  [ fail ]

mysqltest: At line 8: Query 'CREATE TABLE t1 (
a INT PRIMARY KEY,
b VARCHAR(32)
) engine=ndb' failed.
ERROR 1296 (HY000): Got error 4009 'Cluster Failure' from NDBCLUSTER

Skip test on valgrind since using low connect  wait time intentionally.

Change-Id: I4ef55b379f2518121843d4ccc78ac39f7f80def0

Symptom: In some circumstances an index cannot be created;
this is configuration-dependent

Root cause: Merge sort file buffer size can in some cases be
calculated to be the size of IO_BLOCK_SIZE. The logic for the
buffer is that subsequent records of data are added to the
buffer, but when adding a row would overflow the buffer, thw
contents are written to disk in multiples of IO_BLOCK_SIZE and
space is freed up. If the buffer is only IO_BLOCK_SIZE, it is
likely that at that point the existing contents' length is
less than IO_BLOCK_SIZE, in which case nothing gets written,
no space can be freed up, and the new record cannot be added.

Fix: Since maximum allowed key length is 3072 and
IO_BLOCK_SIZE is 4096 bytes, and given other factors
contributing to buffer length, like page size, which also
affects allowed key size, 2 * IO_BLOCK_SIZE is sufficient
minimum length to ensure there is always IO_BLOCK_SIZE
bytes in the buffer when the write happens. If this ever
changes, the merge will fail gracefully.

Change-Id: I7aec373cfed9e364751372cce3746eb7ad75b3b9

…iles

Added the attributes

  CompanyName
  ProductName
  LegalCopyright
  LegalTrademarks

Change-Id: I79cb92b90aabc0ca1961559b0b62c36aa5c525ca

If one put the NDB data node filesystem on some distributed filesystems
one can experience that data node fails when it tries to remove a file
due to file is reported to not exist.  For a local filesystem that
should be impossible to happen, but for some distributed filesystems can
as part of internal failover handling retry a removal which may have
succeeded before the failover in which case the second removal may fail
since file no longer exists.

Data node was changed to allow file and directory removal to fail with
'file does not exist' error and treat that as successful removal.

Note, this apply only to files under the data node filesystem and backup
files. Removing other files will not have changed semantics. Nor will
any file removal by other NDB programs change semantics.

For testing purposed an extra file deletion call is issued roughly 1% of
the times a file deletion is requested, this is only done for debug
builds.

Change-Id: Ie8f5f587e9e675c2a0705d7e450be0e139b045a8

Symptom:
When creating an index on a table containing data, valgrind
occasionally reports reads of uninitialized memory from
ddl::Builder::bulk_add_row

Root cause:
When calculating alignment for the final write of
ddl::Key_sort_buffer::serialize, the IO write may be aligned
to write from region partly past the end of IO buffer.

Fix:
When such a condition is detected, a portion of the IO buffer
is written first to free up space in the buffer.

Change-Id: I607ab549712a077cafdc5e067dfd667db40ade4f

Note: This commit backports the fix to 8.0

Background:
-----------
  Auxiliary FTS index is updated when adding a fulltext index. This
operation uses the Btree_load::insert method to insert tuples to the
index. One such table, aux_table, and the corresponding index,
FTS_INDEX_TABLE_IND, is used to create the inverted index. This index is
a mapping between each word in the table (all rows) with a vector of all
its occurrences. Each occurrence is accounted as a pair of document ID
(row where word was seen) and position (offset in the row). This vector,
called the ilist, is stored in the tuple which is inserted to the index.
The tuple is inserted into the index using Btree_load::insert(dtuple
*tuple, size_t level);

  In the Btree_load::insert, when preparing space for the tuple, we
check if there is enough space in the redo log using log_free_check. To
ensure that no latches are held when calling this, Btree_load::release
is called prior to log_free_check followed by Btree_load::latch to
acquire neccessary latches. The call sequence is
    Btree_load::release -> log_free_check -> Btree_load::latch

  The Btree_load::release calling Page_load::release will buffer fix the
page and commit the MTR. The Btree_load::latch calling Page_load::latch
will buffer unfix the page and start the MTR. However, if m_n_recs == 0,
meaning that no records are inserted yet, then Btree_load::latch will do
nothing.

  The Page_load's MTR is first started in Page_load::init, and is
commited when Page_load::release is called. It is started again in
Page_load::latch. When an index is being rebuilt, the order of the
function calls is:
  init -> latch (m_n_recs == 0 does nothing)
    -> release -> latch -> release -> latch -> ...
    -> release -> latch
  -> finish

Issue:
------
  Btree_load::log_free_check was being called when m_n_recs == 0. This
should not happen since if no records were inserted, then there is no
need to check for free space.

  Furthermore, Btree_load::insert(dtuple *tuple, size_t level) must
increase m_n_recs if insert is successful instead of Btree_load::builder

Fix:
----
  Ensure that m_n_recs is non-zero when calling log_free_check. Ensure
that insert(dtuple, size_t) increases m_n_recs after successful insert.

Note:
-----
  It was observed that threads created for FTS::insert and
FTS::start_parse_threads were run in context of std::thread instead of
runnable. Added missing calls to runnable in FTS::insert inorder to use
DBUG_EXECUTE_IF. Added similar calls in FTS::start_parse_threads as well
  Added asserts to ensure number of records inserted are calculated
correctly both during and at the end of the insert operation.

Change-Id: I8ccd55de79b3ec5d2bef0f99a831aecb99a1ca16

…sing SIGSEGV

A stack allocated protocol instance was popped while i_s.processlist was
about to check whether the client connection was still alive. When the
protocol instance went out of scope, the call to connection_alive() accessed
an invalid pointer, causing SIGSEGV.

The fix is to cache the return value from the current protocol's
connection_alive() method when pushing, popping or getting the protocol.
This might leave results that are slightly out of sync with reality, but
a better synchronization is likely to cause performance degradation.

Change-Id: I3d512fadaa0df145af3f25d4cc03fa20143c5310
(cherry picked from commit 99c1919e0596bb6fe0292e602299e6da880fa352)

Bug#36347647  Contribution by Tencent: Resiliency issue in fts_sync_commit

Symptoms:
During various operations on tables containing FTS indexes
the state of FTS as comitted to database may become
inconsistent, affecting the following scenarios:
 - the server terminates when synchronizing the FTS cache,
 - synchronization of FTS cache occurs concurrently with
   another FTS operation
This inconsistency may lead to various negative effects
including incorrect query results.
An example operation which forces the synchronization of
FTS cach is OPTIMIZE TABLE with
innodb_optimize_fulltext_only set to ON.

Root Cause:
Function 'fts_cmp_set_sync_doc_id' and 'fts_sql_commit' use
different  trx_t objects in function 'fts_sync_commit'.
This causes a scenario where 'synced_doc_id' in the config
table is already committed, but remaining FTS data isn't yet,
leading to issues in the scenarios described above - the
server terminating between the commits, or concurrent access
getting the intermediate state.

Fix:
When 'fts_cmp_set_sync_doc_id' is called from 'fts_sync_commit'
it will use the transaction provided by the caller.

Patch based on contribution by Tencent.

Change-Id: I65fa5702db5e7b6b2004a7311a6b0aa97449034f

Bug#36347647  Contribution by Tencent: Resiliency issue in fts_sync_commit
Bug#36342792  IO write to merge file aligned past end of buffer
Bug#35237928  When innodb_disable_sort_file_cache=on, create a full-text index will fail

Post-push fix:
Add doxygen documentation to function parameter.
Disable innodb_disable_sort_file_cache test on systems with
no O_DIRECT mode support.
Fix innodb.fts_sync_commit_resiliency for 8.0 branch.

Change-Id: Ie3dbf78b4e84ae22c85d1ce873b58ee926566ff8

active transaction exist

SYMPTOM:
-------
Assertion failure in InnoDB's background
when a transaction for which it wants to acquire an MDL lock, turns out
to be no longer active.

ROOT CAUSE:
----------
When creating a list of <trx id, table id> pairs for which we will need
to acquire MDL in trx_recovery_rollback_thread later, we accidentally
include a transaction which not only will not be rolled back by InnoDB
background thread (because it is TRX_STATE_PREPARED), but worse still,
might be rolled back by  even before we get to
spawn .

SOLUTION:
--------
InnoDB should never include tables of transactions which are in
TRX_STATE_PREPARED state in the list, because InnoDB is not allowed to
roll them back by itself, and they might be rolled back by binlog logic.

Thanks to Genze Wu (Alibaba) for the contribution.

Change-Id: I9f366c1e1022464e6dd43de08bed89f0510ad786

Approved by: Erlend Dahl <erlend.dahl@oracle.com>

Approved by: Erlend Dahl <erlend.dahl@oracle.com>