[15721] Logging and Recovery #1348

db-ol · 2018-05-05T10:29:38Z

This PR implements logging and recovery for robustness to crashes.

Logging:

Create log records and write them into log buffers in TimestampOrderingTransactionManager as executing transactions. Submit the buffer when it is full and then acquire a new one for the current transaction.
Push the callback for committing transactions from the network level to the worker level. Set logging callbacks and return QUEUING in CommitTransaction and AbortTransaction.
Adopt delta logging for updates. Pass values_buf, values_size and offsets all the way down to PerformUpdate.
Use tokens to guarantee the FIFO order in the logical queue for multiple workers. The logic queue consists of a few sub-queues, each per worker. Buffers belonging to the same transaction will be put into the same sub-queue.
Support the default codegen engine only.

Recovery:

Implement two-pass recovery. The first pass filters out transactions that are not committed while the second one classifies recods by txn_id. After two passes, all the transactions are replayed.
Support Create Table, Create Database, Insert and Delete currently.

…eloton into update_codegen

…into logger-callbacks

2. removed logging for read-only TXN.

…eloton into logger-callbacks

2. Completed table schema restore

… recovery is broken)

Conflicts: src/codegen/updater.cpp src/concurrency/timestamp_ordering_transaction_manager.cpp src/storage/data_table.cpp test/brain/query_logger_test.cpp

Zeninma · 2018-05-12T02:48:00Z

src/logging/wal_recovery.cpp

+            case LogRecordType::TRANSACTION_BEGIN:{
+              if(all_txns_.find(txn_id) != all_txns_.end()){
+                LOG_ERROR("Duplicate transaction");
+                PELOTON_ASSERT(false);


In the beginning, upon log file reading failure, the StartRecovery writes a LOG_ERROR and returns. I am wondering whether it should behave the same here - return after writing LOG_ERROR instead of using PELOTON_ASSERT(false) to abort?

Zeninma · 2018-05-12T02:50:02Z

src/logging/wal_recovery.cpp

+        buf_curr += (record_len + sizeof(record_len));
+        buf_rem  -= (record_len + sizeof(record_len));
+      } else {
+        break;


Might directly log an error and abort/return instead of using break.

Zeninma · 2018-05-12T03:40:27Z

src/logging/wal_recovery.cpp

+      LOG_INFO("Replaying TXN_COMMIT");
+    }
+
+    else if(record_type==LogRecordType::TRANSACTION_ABORT) {


This check might need to be moved to ParseFromDisk and files an error there.
If a TRANSACTION_ABORT record were found at then at this time, then the previous changes in this transaction would have been made (which should not be, if I got this correctly).
On the other hand, if the transaction could reach here, then it should also have a TRANSACTION_COMMIT record. I wonder if it is possible for a transaction to have both records. If not, then such check can be redundant.

Zeninma · 2018-05-12T03:55:22Z

src/logging/wal_recovery.cpp

+  }
+
+  // Pass 2
+  log_buffer_  = new char[log_buffer_size_];


I wonder if the recovery needs to read the log file twice from the disk. Since log_buffer_ is constructed here after the first pass.
If this is the case, will it be better to populate the log_buffer_ in the first phase and save the second disk read in the second phase?

nwang57

Logging looks good. recovery seems not finished yet. Add some tests for the recovery part so that correctness can be easily verified.

nwang57 · 2018-05-12T02:32:45Z

src/network/postgres_protocol_handler.cpp

@@ -862,7 +862,9 @@ void PostgresProtocolHandler::ExecExecuteMessageGetResult(ResultType status) {
 }

 void PostgresProtocolHandler::GetResult() {
-  traffic_cop_->ExecuteStatementPlanGetResult();
+
+//  traffic_cop_->ExecuteStatementPlanGetResult();


why this is commented out

nwang57 · 2018-05-12T02:56:54Z

src/concurrency/timestamp_ordering_transaction_manager.cpp

+                      current_txn->GetEpochId(), current_txn->GetTransactionId(),
+                      current_txn->GetCommitId(), schema_oid);
+
+      record.SetOldItemPointer(location);


You already set the old item pointer once during the log record constructor why do you need to set it explicitly?

nwang57 · 2018-05-12T02:57:15Z

src/concurrency/timestamp_ordering_transaction_manager.cpp

+                      LogRecordType::TUPLE_UPDATE, location, new_location, current_txn->GetEpochId(),
+                      current_txn->GetTransactionId(), current_txn->GetCommitId(), schema_oid);
+
+      record.SetOldItemPointer(location);


You already set the old item pointer once during the log record constructor why do you need to set it explicitly?

nwang57 · 2018-05-12T03:11:23Z

src/concurrency/timestamp_ordering_transaction_manager.cpp

+                      LogRecordType::TRANSACTION_COMMIT, current_txn->GetEpochId(),
+                      current_txn->GetTransactionId(), current_txn->GetCommitId());
+
+      current_txn->GetLogBuffer()->WriteRecord(record);


Will the log_buffer exceeds its threshold after this write?

nwang57 · 2018-05-12T03:11:33Z

src/concurrency/timestamp_ordering_transaction_manager.cpp

+                      LogRecordType::TRANSACTION_ABORT, current_txn->GetEpochId(),
+                      current_txn->GetTransactionId(), current_txn->GetCommitId());
+
+      current_txn->GetLogBuffer()->WriteRecord(record);


Will the log_buffer exceeds its threshold after this write?

nwang57 · 2018-05-12T03:20:41Z

src/logging/wal_logger.cpp

+  stream->flush();
+
+  if(stream->fail()){
+    PELOTON_ASSERT(false);


why not PELOTON_ASSERT(!stream->fail())? Do we really want to crash the system if the stream fails or is there any mechanism to rewrite the log to file?

nwang57 · 2018-05-12T03:52:09Z

src/logging/wal_recovery.cpp

+  }
+
+  // Pass 2
+  log_buffer_  = new char[log_buffer_size_];


This memory may not be freed

nwang57 · 2018-05-12T03:55:44Z

src/logging/wal_recovery.cpp

+    if(it->second.first != LogRecordType::TRANSACTION_COMMIT)
+      continue;
+
+    auto offset_pair = std::make_pair(curr_txn_offset, curr_txn_offset);


Why making a pair of two identical size_t values?

gvos94 and others added 30 commits February 20, 2018 16:20

serialized and inserted log records for inserts

fc8eda8

insert logging refactoring

e318f53

merged logging branch

ba219b4

merged insert-logging branch

2c6a4ba

stored single statement txn status in transaction context

a214b57

single statement transaction - pushed commit to worker thread

943cb23

Basic starter for getting values from codegen

cacdf54

Updates getting info from codegen

fb02b98

Working updates with codegen

fa8ed5a

multi statement transaction - pushed commit to worker thread

d681bf7

added order by clause to query_logger_test

0df5f99

Inserts and updates working with codegen

7cf00b6

updates bug fix

6ed8c3b

deletes and updates working with ItemPointer

2cd4caa

Writing the size of the diff buffer into the update log record

a04e251

added support for tokenized logging

ec9aa9f

merge insert-logging and logging commits

ea33dd0

Merge branch 'rearchitect_network' of https://github.com/aaron-tian/p…

982edc4

…eloton into update_codegen

Do not assert for commit log records

2f8b695

Adding log changes to old engine to log schema changes

72320a1

plugged in callbacks

8b431e9

Merge branch 'update_codegen' of https://github.com/aaron-tian/peloton …

216a9b5

…into logger-callbacks

fixing merge mistakes for abort

48ee867

refactored CommitQuery/AbortQuery callbacks

c2a60fb

1. added support for logging BEGIN_TXN

8f53eda

2. removed logging for read-only TXN.

fixed logger thread join bug

8ceff64

Merge branch 'logger-callbacks_v2' of https://github.com/aaron-tian/p…

fe38ae0

…eloton into logger-callbacks

Merge branch 'master' into logging

27d88f3

Fix macro names

913d3f4

Fix conflicts left

a09bec0

gvos94 and others added 17 commits May 4, 2018 20:04

1. Fix database recovery bug - registered db with catalog

8ac72f7

2. Completed table schema restore

completed index recovery

629218c

recovery: recovers a tuple inserts

46ceb0e

recovery: recovers tuple-deletes from non-catalog tables. (drop table…

e00732c

… recovery is broken)

PL_* to PELOTON_*

544c6f9

Few bug fixes and code cleanup

07f2e22

code refactoring

5e9b17b

moved enable_logging flag to settings

6ec3e76

merged anirudh's fixes

aa2a4d8

Adapt to new APIs

d539040

Specify whether to turn on recovery mode at initilization

52b46dd

Add the recovery switch into settings

91f7920

Adjust the log level

2b752da

Separate the insert log buffer test

1ba0f23

Separate the update log buffer test

7e38591

Separate the delete log buffer test

3f63eb5

Add the codebase for recovery tests

4627b6a

db-ol added class-project do not merge labels May 5, 2018

db-ol force-pushed the logging branch from b5e38aa to 4627b6a Compare May 5, 2018 20:57

Aaron Tian added 2 commits May 5, 2018 22:03

Change catalog singletons in recovery to instances

c6564d9

Change logging & header files

5a73bcf

db-ol force-pushed the logging branch from 0ef86bd to 5a73bcf Compare May 6, 2018 22:17

latelatif added 2 commits May 7, 2018 22:19

Merge branch 'master' into logging

6b152ff

Conflicts: src/codegen/updater.cpp src/concurrency/timestamp_ordering_transaction_manager.cpp src/storage/data_table.cpp test/brain/query_logger_test.cpp

Incorporating schema/namespace changes with logging/recovery

0b68865

Zeninma reviewed May 12, 2018

View reviewed changes

nwang57 reviewed May 12, 2018

View reviewed changes

cmu-db deleted a comment from nwang57 May 13, 2018

Add logging and recovery tests

2793bfa

apavlo mentioned this pull request Jul 8, 2018

peloton not support persistence #1456

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[15721] Logging and Recovery #1348

[15721] Logging and Recovery #1348

db-ol commented May 5, 2018 •

edited

Loading

Zeninma May 12, 2018

Zeninma May 12, 2018

Zeninma May 12, 2018

Zeninma May 12, 2018

nwang57 left a comment

nwang57 May 12, 2018

nwang57 May 12, 2018

nwang57 May 12, 2018

nwang57 May 12, 2018

nwang57 May 12, 2018

nwang57 May 12, 2018

nwang57 May 12, 2018

nwang57 May 12, 2018

[15721] Logging and Recovery #1348

Are you sure you want to change the base?

[15721] Logging and Recovery #1348

Conversation

db-ol commented May 5, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nwang57 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

db-ol commented May 5, 2018 •

edited

Loading