Commits on Mar 19, 2024

1. PS-9107 : [ERROR] [MY-013183] [InnoDB] Assertion failure: ibuf0ibuf.c… …

   …c:3833:ib::fatal triggered thread
   
   https://perconadev.atlassian.net/browse/PS-9107
   
   Problem:
   --------
   Two threads are trying to delete the same ibuf rec. The first one succeeds with
   optimistic delete and this means the record is completely gone (added to page garbage).
   
   Now second thread, cannot do optimistic. So it tries to do pessmistic and wants
   to position the cursor on the record deleted by the first thread. It cannot.
   The record is gone. And ibuf asserts.
   
   In one of the innodb master thread active tasks (srv_master_do_active_tasks),
   innodb tries to merge/contract ibuf in background. So it does this by reading
   actual pages from disk. So it leaves the job to IO threads. On IO read,
   we apply pending change buffer entries. This thread (on certain conditions),
   can decide to a sync read vs async read. In our case, it is an async read.
   Master thread submits a request to read the page. Lets say space_id:page_no (978:34)
   
   At around same time, an insert/delete into secondary indexes wants to buffer a
   change but the ibuf has reached to its max size, so it initiates a ibuf_contract().
   See ibuf_insert()-> calling ibuf_contract()->ibuf_merge_pages. This opens a cursor
   at a random record and it can happen that it sees the same space_id:page_no as the
   IO thread is processing.
   
   And just when this tablespace records are about to be processed, the tablespace
   is dropped. So the ibuf_merge_pages() decides it is no longer necessary to do the
   actual read of secondary index pages. Hey, the tablespace is gone! whats the point
   in bringing those pages to buffer pool. Hence it decides to delete all ibuf records
   belonging to space_id (978) in our example.
   
   This leads to the case where two threads can simultaneously process the ibuf
   records of the same page. IO thread  reading a secondary index page and contract
   ibuf belonging to this space_id,page_no (this read is initiated by innodb master ibuf merge task)
   user threads doing ibuf_contract, they process entries belonging to a different tablespace.
   And when they see that tabelspace is dropped), they try to delete ibuf entries.
   
   Fix:
   ----
   ibuf restore pos is designed to handle the “dropped tablespace” already, but in 8.0,
   the way tablespaces are dropped is a bit different.
   
   fil_space_get_flags(space) === ULINT32_UNDEFINED happens when space object is freed.
   And on nullptr, ULINT32_UNDEFINED is returned by fil_space_get_flags()
   
   Technically the problem can happen in 5.7 too, but it is more obvious in 8.0 because
   the window between the actual deletion of the tablespace object and the user drop
   tablespace command is bigger. In 8.0, space object is still available, so flags is
   NOT ULINT32_UNDEFINED. At the time of crash, only stop_new_ops is set to true. If this
   fil_space_t->stop_new_ops is true, fil_space_acquire() returns nullptr.
   We will use this call to determine if tablespace is dropped or not.
   
   ibuf code saw the tablespace as deleted by using fil_space_acquire() in
   ibuf_merge_or_delete_for_page().
   
   We will use the same call in ibuf_restore_pos() to determine if the tablespace is being
   deleted.

   

   satya-bodapati committed Mar 19, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for b45c4fa
   • Browse repository at this point

   Copy the full SHA

   b45c4fa View commit details

   Browse the repository at this point in the history