Fixed blob speed knob initialization #23

satherton · 2018-03-05T07:09:33Z

No description provided.

…g opposite knobs.

/home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:86:8: runtime error: load of value 1231493777, which is not a valid value for type 'limitReason_t' #0 0x310e961 in StorageQueueInfo::StorageQueueInfo(StorageQueueInfo&&) /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:86 #1 0x310eacd in MapPair<UID, StorageQueueInfo>::MapPair<UID, StorageQueueInfo>(UID&&, StorageQueueInfo&&) /home/anoyes/workspace/foundationdb/flow/IndexedSet.h:242 #2 0x310b35e in MapPair<std::decay<UID>::type, std::decay<StorageQueueInfo>::type> mapPair<UID, StorageQueueInfo>(UID&&, StorageQueueInfo&&) /home/anoyes/workspace/foundationdb/flow/IndexedSet.h:258 #3 0x30a8b79 in a_body1 /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:195 apple#4 0x309b529 in TrackStorageServerQueueInfoActor /home/anoyes/build/foundationdb/fdbserver/Ratekeeper.actor.g.cpp:495 apple#5 0x309b9be in trackStorageServerQueueInfo(RatekeeperData* const&, StorageServerInterface const&) /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:194 apple#6 0x30cff63 in a_body1loopBody1when1cont1 /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:303 apple#7 0x30cd9da in a_body1loopBody1when1when1 /home/anoyes/build/foundationdb/fdbserver/Ratekeeper.actor.g.cpp:1170 apple#8 0x30ed4dd in a_callback_fire /home/anoyes/build/foundationdb/fdbserver/Ratekeeper.actor.g.cpp:1185 apple#9 0x30e6d81 in fire /home/anoyes/workspace/foundationdb/flow/flow.h:998 apple#10 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /home/anoyes/workspace/foundationdb/flow/flow.h:447 apple#11 0x959891 in void Promise<Void>::send<Void>(Void&&) const /home/anoyes/workspace/foundationdb/flow/flow.h:778 apple#12 0x7b4b018 in Sim2::execTask(Sim2::Task&) (/home/anoyes/build/foundationdb/bin/fdbserver+0x7b4b018) apple#13 0x7bf9168 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1cont1(Void const&, int) /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:979 apple#14 0x7be7b68 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1when1(Void const&, int) /home/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5391 apple#15 0x7c329ff in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_callback_fire(ActorCallback<Sim2::RunLoopActor, 0, Void>*, Void) /home/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5406 apple#16 0x7c1fc73 in ActorCallback<Sim2::RunLoopActor, 0, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:998 apple#17 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /home/anoyes/workspace/foundationdb/flow/flow.h:447 apple#18 0x959891 in void Promise<Void>::send<Void>(Void&&) const /home/anoyes/workspace/foundationdb/flow/flow.h:778 apple#19 0x7fe74a4 in N2::PromiseTask::operator()() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:481 apple#20 0x7fb6ff7 in N2::Net2::run() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:657 apple#21 0x7b71bd3 in Sim2::_runActorState<Sim2::_runActor>::a_body1(int) /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:989 apple#22 0x7b2ee51 in Sim2::_runActor::_runActor(Sim2* const&) /home/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5608 apple#23 0x7b2f268 in Sim2::_run(Sim2* const&) /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:987 apple#24 0x7b2f2c8 in Sim2::run() /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:996 apple#25 0x21040a6 in main /home/anoyes/workspace/foundationdb/fdbserver/fdbserver.actor.cpp:1793 apple#26 0x7f03492ba504 in __libc_start_main (/lib64/libc.so.6+0x22504) apple#27 0x464914 (/home/anoyes/build/foundationdb/bin/fdbserver+0x464914)

/Users/anoyes/workspace/foundationdb/fdbserver/TLogInterface.h:149:8: runtime error: load of value 232, which is not a valid value for type 'bool' #0 0xc608fb in TLogPeekReply::TLogPeekReply(TLogPeekReply const&) /Users/anoyes/workspace/foundationdb/fdbserver/TLogInterface.h:149 #1 0x242bf87 in ILogSystem::ServerPeekCursor::ServerPeekCursor(TLogPeekReply const&, LogMessageVersion const&, LogMessageVersion const&, int, int, bool, long, Tag) /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:35 #2 0x242da77 in ILogSystem::ServerPeekCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:47 #3 0x24362d5 in ILogSystem::MergedPeekCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:325 apple#4 0x244bf45 in ILogSystem::MultiCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:838 apple#5 0x36b5a36 in a_body1cont5loopBody1 /Users/anoyes/workspace/foundationdb/fdbserver/storageserver.actor.cpp:2621 apple#6 0x36b3110 in a_body1cont5loopHead1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8664 apple#7 0x36b07fe in a_body1cont5 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8576 apple#8 0x36abda8 in a_body1cont4when1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8582 apple#9 0x36a8dc2 in a_body1cont4 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8454 apple#10 0x36a4bf6 in a_body1cont3break1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8489 apple#11 0x36a2c01 in a_body1cont3loopBody1cont1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8505 apple#12 0x369fd36 in a_body1cont3loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8513 apple#13 0x3700dcb in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8528 apple#14 0x36e5210 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#15 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479 apple#16 0x2484b07 in a_body1loopBody1cont1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1526 apple#17 0x24822cf in a_body1loopBody1cont2 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1535 apple#18 0x247e228 in a_body1loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1541 apple#19 0x249be87 in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1556 apple#20 0x249668f in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#21 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479 apple#22 0x80557e in a_body1when1 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11591 apple#23 0x8916ef in a_callback_fire /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11620 apple#24 0x8735f5 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#25 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479 apple#26 0x24820f8 in a_body1cont1loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:860 apple#27 0x249c852 in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:886 apple#28 0x249786c in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#29 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479 apple#30 0x248b39f in a_body1cont2 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11858 apple#31 0x2489d02 in a_body1when1 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11865 apple#32 0x249a150 in a_callback_fire /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11880 apple#33 0x2492a4f in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#34 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479 apple#35 0x248df9b in a_body1cont2 /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2762 apple#36 0x248b7da in a_body1when1 /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2769 apple#37 0x2499c88 in a_callback_fire /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2784 apple#38 0x2492371 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#39 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479 apple#40 0xc60fb3 in void SAV<TLogPeekReply>::sendAndDelPromiseRef<TLogPeekReply&>(TLogPeekReply&) /Users/anoyes/workspace/foundationdb/flow/flow.h:472 apple#41 0xc1137a in NetSAV<TLogPeekReply>::receive(ArenaObjectReader&) /Users/anoyes/workspace/foundationdb/fdbrpc/fdbrpc.h:111 apple#42 0x78eda75 in a_body1cont1 /Users/anoyes/workspace/foundationdb/fdbrpc/FlowTransport.actor.cpp:652 apple#43 0x78f7967 in a_body1cont2 /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2369 apple#44 0x78ed4d8 in a_body1when1 /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2375 apple#45 0x791af45 in a_callback_fire /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2390 apple#46 0x7914670 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#47 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /Users/anoyes/workspace/foundationdb/flow/flow.h:447 apple#48 0x959891 in void Promise<Void>::send<Void>(Void&&) const /Users/anoyes/workspace/foundationdb/flow/flow.h:778 apple#49 0x7b4b022 in Sim2::execTask(Sim2::Task&) (/Users/anoyes/build/foundationdb/bin/fdbserver+0x7b4b022) apple#50 0x7bf9172 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1cont1(Void const&, int) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:979 apple#51 0x7be7b72 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1when1(Void const&, int) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5391 apple#52 0x7c32a09 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_callback_fire(ActorCallback<Sim2::RunLoopActor, 0, Void>*, Void) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5406 apple#53 0x7c1fc7d in ActorCallback<Sim2::RunLoopActor, 0, Void>::fire(Void const&) /Users/anoyes/workspace/foundationdb/flow/flow.h:998 apple#54 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /Users/anoyes/workspace/foundationdb/flow/flow.h:447 apple#55 0x959891 in void Promise<Void>::send<Void>(Void&&) const /Users/anoyes/workspace/foundationdb/flow/flow.h:778 apple#56 0x7fe74ae in N2::PromiseTask::operator()() /Users/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:481 apple#57 0x7fb7001 in N2::Net2::run() /Users/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:657 apple#58 0x7b71bdd in Sim2::_runActorState<Sim2::_runActor>::a_body1(int) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:989 apple#59 0x7b2ee5b in Sim2::_runActor::_runActor(Sim2* const&) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5608 apple#60 0x7b2f272 in Sim2::_run(Sim2* const&) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:987 apple#61 0x7b2f2d2 in Sim2::run() /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:996 apple#62 0x2104064 in main /Users/anoyes/workspace/foundationdb/fdbserver/fdbserver.actor.cpp:1793 apple#63 0x7fb7c6561504 in __libc_start_main (/lib64/libc.so.6+0x22504) apple#64 0x464914 (/Users/anoyes/build/foundationdb/bin/fdbserver+0x464914)

SNOW-152673 Set rpath to /usr/lib64/fdb6/

TSAN output: 3: WARNING: ThreadSanitizer: data race (pid=63) 3: Read of size 1 at 0x7b6800000970 by thread T1: 3: #0 N2::Net2::onMainThread(Promise<Void>&&, TaskPriority) /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1734:6 (libfdb_c.so+0xca3d9e) 3: apple#1 void onMainThreadVoid<ThreadSingleAssignmentVarBase::cancel()::'lambda'()>(ThreadSingleAssignmentVarBase::cancel()::'lambda'(), Error*, TaskPriority) /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:47:13 (libfdb_c.so+0x3fe73a) 3: apple#2 ThreadSingleAssignmentVarBase::cancel() /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:317:3 (libfdb_c.so+0x3fe290) 3: apple#3 fdb_future_cancel /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:141:96 (libfdb_c.so+0x3f7896) 3: apple#4 DLThreadSingleAssignmentVar<long>::cancel() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionAssignmentVars.h:175:4 (libfdb_c.so+0x45bd4d) 3: apple#5 MapSingleAssignmentVar<long, Void>::cancel() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionAssignmentVars.h:241:20 (libfdb_c.so+0x475ba7) 3: apple#6 FlatMapSingleAssignmentVar<Void, Void>::cancel() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionAssignmentVars.h:301:27 (libfdb_c.so+0x4742ce) 3: apple#7 ThreadFuture<Void>::cancel() /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:444:32 (libfdb_c.so+0x441237) 3: apple#8 MultiVersionDatabase::Connector::cancel() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:969:20 (libfdb_c.so+0x441237) 3: apple#9 MultiVersionDatabase::DatabaseState::cancelConnections()::$_22::operator()() const /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1082:11 (libfdb_c.so+0x441237) 3: apple#10 (anonymous namespace)::DoOnMainThreadVoidActorState<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22, (anonymous namespace)::DoOnMainThreadVoidActor<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22> >::a_body1cont1(Void const&, int) /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:559:2 (libfdb_c.so+0x441237) 3: apple#11 (anonymous namespace)::DoOnMainThreadVoidActorState<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22, (anonymous namespace)::DoOnMainThreadVoidActor<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22> >::a_body1when1(Void const&, int) /home/anoyes/build/foundationdb-tsan/flow/ThreadHelper.actor.g.h:992:15 (libfdb_c.so+0x441237) 3: apple#12 (anonymous namespace)::DoOnMainThreadVoidActorState<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22, (anonymous namespace)::DoOnMainThreadVoidActor<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22> >::a_callback_fire(ActorCallback<(anonymous namespace)::DoOnMainThreadVoidActor<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22>, 0, Void>*, Void const&) /home/anoyes/build/foundationdb-tsan/flow/ThreadHelper.actor.g.h:1013:4 (libfdb_c.so+0x440dff) 3: apple#13 ActorCallback<(anonymous namespace)::DoOnMainThreadVoidActor<MultiVersionDatabase::DatabaseState::cancelConnections()::$_22>, 0, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1016:78 (libfdb_c.so+0x440dff) 3: apple#14 void SAV<Void>::send<Void>(Void&&) /home/anoyes/workspace/foundationdb/flow/flow.h:465:23 (libfdb_c.so+0x4086b2) 3: apple#15 void Promise<Void>::send<Void>(Void&&) const /home/anoyes/workspace/foundationdb/flow/flow.h:782:8 (libfdb_c.so+0xcc079e) 3: apple#16 N2::PromiseTask::operator()() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1157:11 (libfdb_c.so+0xcc079e) 3: apple#17 N2::Net2::run() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1493:5 (libfdb_c.so+0xca1640) 3: apple#18 runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/NativeAPI.actor.cpp:1774:13 (libfdb_c.so+0x499eb4) 3: apple#19 ThreadSafeApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/ThreadSafeTransaction.cpp:429:3 (libfdb_c.so+0xb666cc) 3: apple#20 MultiVersionApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1521:20 (libfdb_c.so+0x41e9dd) 3: apple#21 fdb_run_network /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:129:45 (libfdb_c.so+0x3f7645) 3: apple#22 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3539:173 (fdb_c_unit_tests+0x384856) 3: apple#23 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:273:5 (fdb_c_unit_tests+0x384856) 3: apple#24 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:284:5 (fdb_c_unit_tests+0x384856) 3: 3: Previous write of size 1 at 0x7b6800000970 by thread T2: 3: #0 N2::Net2::stopImmediately() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:253:11 (libfdb_c.so+0xca936b) 3: apple#1 N2::Net2::stop()::'lambda'()::operator()() const /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:171:36 (libfdb_c.so+0xca936b) 3: apple#2 (anonymous namespace)::DoOnMainThreadVoidActorState<N2::Net2::stop()::'lambda'(), (anonymous namespace)::DoOnMainThreadVoidActor<N2::Net2::stop()::'lambda'()> >::a_body1cont1(Void const&, int) /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:559:2 (libfdb_c.so+0xca936b) 3: apple#3 (anonymous namespace)::DoOnMainThreadVoidActorState<N2::Net2::stop()::'lambda'(), (anonymous namespace)::DoOnMainThreadVoidActor<N2::Net2::stop()::'lambda'()> >::a_body1when1(Void const&, int) /home/anoyes/build/foundationdb-tsan/flow/ThreadHelper.actor.g.h:992:15 (libfdb_c.so+0xca936b) 3: apple#4 (anonymous namespace)::DoOnMainThreadVoidActorState<N2::Net2::stop()::'lambda'(), (anonymous namespace)::DoOnMainThreadVoidActor<N2::Net2::stop()::'lambda'()> >::a_callback_fire(ActorCallback<(anonymous namespace)::DoOnMainThreadVoidActor<N2::Net2::stop()::'lambda'()>, 0, Void>*, Void const&) /home/anoyes/build/foundationdb-tsan/flow/ThreadHelper.actor.g.h:1013:4 (libfdb_c.so+0xca9128) 3: apple#5 ActorCallback<(anonymous namespace)::DoOnMainThreadVoidActor<N2::Net2::stop()::'lambda'()>, 0, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1016:78 (libfdb_c.so+0xca9128) 3: apple#6 void SAV<Void>::send<Void>(Void&&) /home/anoyes/workspace/foundationdb/flow/flow.h:465:23 (libfdb_c.so+0x4086b2) 3: apple#7 void Promise<Void>::send<Void>(Void&&) const /home/anoyes/workspace/foundationdb/flow/flow.h:782:8 (libfdb_c.so+0xcc079e) 3: apple#8 N2::PromiseTask::operator()() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1157:11 (libfdb_c.so+0xcc079e) 3: apple#9 N2::Net2::run() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1493:5 (libfdb_c.so+0xca1640) 3: apple#10 runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/NativeAPI.actor.cpp:1774:13 (libfdb_c.so+0x499eb4) 3: apple#11 ThreadSafeApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/ThreadSafeTransaction.cpp:429:3 (libfdb_c.so+0xb666cc) 3: apple#12 MultiVersionApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1521:20 (libfdb_c.so+0x41e9dd) 3: apple#13 fdb_run_network /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:129:45 (libfdb_c.so+0x3f7645) 3: apple#14 DLApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:492:11 (libfdb_c.so+0x40dd18) 3: apple#15 runNetworkThread(void*) /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1495:30 (libfdb_c.so+0x41e895) 3: 3: Location is heap block of size 1304 at 0x7b6800000600 allocated by main thread: 3: #0 operator new(unsigned long) <null> (fdb_c_unit_tests+0x31694e) 3: apple#1 newNet2(TLSConfig const&, bool, bool) /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1969:16 (libfdb_c.so+0xca607d) 3: apple#2 setupNetwork(unsigned long, bool) /home/anoyes/workspace/foundationdb/fdbclient/NativeAPI.actor.cpp:1754:14 (libfdb_c.so+0x499c53) 3: apple#3 ThreadSafeApi::setupNetwork() /home/anoyes/workspace/foundationdb/fdbclient/ThreadSafeTransaction.cpp:423:2 (libfdb_c.so+0xb66675) 3: apple#4 MultiVersionApi::setupNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1462:21 (libfdb_c.so+0x41dc42) 3: apple#5 fdb_setup_network_impl() /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:116:45 (libfdb_c.so+0x3f7525) 3: apple#6 DLApi::setupNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:488:15 (libfdb_c.so+0x40dc88) 3: apple#7 MultiVersionApi::setupNetwork()::$_27::operator()(Reference<ClientInfo>) const /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1481:17 (libfdb_c.so+0x442cd4) 3: apple#8 decltype(std::__1::forward<MultiVersionApi::setupNetwork()::$_27&>(fp)(std::__1::forward<Reference<ClientInfo> >(fp0))) std::__1::__invoke<MultiVersionApi::setupNetwork()::$_27&, Reference<ClientInfo> >(MultiVersionApi::setupNetwork()::$_27&, Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/type_traits:3539:173 (libfdb_c.so+0x442cd4) 3: apple#9 void std::__1::__invoke_void_return_wrapper<void>::__call<MultiVersionApi::setupNetwork()::$_27&, Reference<ClientInfo> >(MultiVersionApi::setupNetwork()::$_27&, Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/__functional_base:348:9 (libfdb_c.so+0x442cd4) 3: apple#10 std::__1::__function::__alloc_func<MultiVersionApi::setupNetwork()::$_27, std::__1::allocator<MultiVersionApi::setupNetwork()::$_27>, void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/functional:1540:16 (libfdb_c.so+0x442cd4) 3: apple#11 std::__1::__function::__func<MultiVersionApi::setupNetwork()::$_27, std::__1::allocator<MultiVersionApi::setupNetwork()::$_27>, void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/functional:1714:12 (libfdb_c.so+0x442cd4) 3: apple#12 std::__1::__function::__value_func<void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>&&) const /usr/local/bin/../include/c++/v1/functional:1867:16 (libfdb_c.so+0x4153ed) 3: apple#13 std::__1::function<void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>) const /usr/local/bin/../include/c++/v1/functional:2473:12 (libfdb_c.so+0x4153ed) 3: apple#14 MultiVersionApi::runOnExternalClients(int, std::__1::function<void (Reference<ClientInfo>)>, bool) /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1117:5 (libfdb_c.so+0x4153ed) 3: apple#15 MultiVersionApi::runOnExternalClientsAllThreads(std::__1::function<void (Reference<ClientInfo>)>, bool) /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1102:3 (libfdb_c.so+0x41992c) 3: apple#16 MultiVersionApi::setupNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1475:3 (libfdb_c.so+0x41dd2a) 3: apple#17 fdb_setup_network_impl() /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:116:45 (libfdb_c.so+0x3f7525) 3: apple#18 main /home/anoyes/workspace/foundationdb/bindings/c/test/unit/unit_tests.cpp:2147:12 (fdb_c_unit_tests+0x35f6b2) 3: 3: Thread T1 (tid=65, running) created by main thread at: 3: #0 pthread_create <null> (fdb_c_unit_tests+0x29df32) 3: apple#1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:394:10 (fdb_c_unit_tests+0x372da6) 3: apple#2 std::__1::thread::thread<int (*)(), void>(int (*&&)()) /usr/local/bin/../include/c++/v1/thread:300:16 (fdb_c_unit_tests+0x372da6) 3: apple#3 main /home/anoyes/workspace/foundationdb/bindings/c/test/unit/unit_tests.cpp:2148:14 (fdb_c_unit_tests+0x35f6dd) 3: 3: Thread T2 (tid=66, finished) created by thread T1 at: 3: #0 pthread_create <null> (fdb_c_unit_tests+0x29df32) 3: apple#1 startThread(void* (*)(void*), void*, int, char const*) /home/anoyes/workspace/foundationdb/flow/Platform.actor.cpp:2745:2 (libfdb_c.so+0xd08d74) 3: apple#2 N2::Net2::startThread(void* (*)(void*), void*) /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1749:9 (libfdb_c.so+0xca42a4) 3: apple#3 MultiVersionApi::runNetwork()::$_28::operator()(Reference<ClientInfo>) const /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1516:34 (libfdb_c.so+0x443106) 3: apple#4 decltype(std::__1::forward<MultiVersionApi::runNetwork()::$_28&>(fp)(std::__1::forward<Reference<ClientInfo> >(fp0))) std::__1::__invoke<MultiVersionApi::runNetwork()::$_28&, Reference<ClientInfo> >(MultiVersionApi::runNetwork()::$_28&, Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/type_traits:3539:173 (libfdb_c.so+0x443106) 3: apple#5 void std::__1::__invoke_void_return_wrapper<void>::__call<MultiVersionApi::runNetwork()::$_28&, Reference<ClientInfo> >(MultiVersionApi::runNetwork()::$_28&, Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/__functional_base:348:9 (libfdb_c.so+0x443106) 3: apple#6 std::__1::__function::__alloc_func<MultiVersionApi::runNetwork()::$_28, std::__1::allocator<MultiVersionApi::runNetwork()::$_28>, void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/functional:1540:16 (libfdb_c.so+0x443106) 3: apple#7 std::__1::__function::__func<MultiVersionApi::runNetwork()::$_28, std::__1::allocator<MultiVersionApi::runNetwork()::$_28>, void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>&&) /usr/local/bin/../include/c++/v1/functional:1714:12 (libfdb_c.so+0x443106) 3: apple#8 std::__1::__function::__value_func<void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>&&) const /usr/local/bin/../include/c++/v1/functional:1867:16 (libfdb_c.so+0x4153ed) 3: apple#9 std::__1::function<void (Reference<ClientInfo>)>::operator()(Reference<ClientInfo>) const /usr/local/bin/../include/c++/v1/functional:2473:12 (libfdb_c.so+0x4153ed) 3: apple#10 MultiVersionApi::runOnExternalClients(int, std::__1::function<void (Reference<ClientInfo>)>, bool) /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1117:5 (libfdb_c.so+0x4153ed) 3: apple#11 MultiVersionApi::runOnExternalClientsAllThreads(std::__1::function<void (Reference<ClientInfo>)>, bool) /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1102:3 (libfdb_c.so+0x41992c) 3: apple#12 MultiVersionApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1514:3 (libfdb_c.so+0x41ea53) 3: apple#13 fdb_run_network /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:129:45 (libfdb_c.so+0x3f7645) 3: apple#14 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3539:173 (fdb_c_unit_tests+0x384856) 3: apple#15 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:273:5 (fdb_c_unit_tests+0x384856) 3: apple#16 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:284:5 (fdb_c_unit_tests+0x384856) 3: 3: SUMMARY: ThreadSanitizer: data race /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1734:6 in N2::Net2::onMainThread(Promise<Void>&&, TaskPriority) 3: ================== 3: ThreadSanitizer: reported 1 warnings

Fix gcc build

* Add timeout logics to client requests Description Testing * Add the request timeout in client to clientKnobs Description Testing * Change the name of timeout param in clients Description Testing * Timeout mechanism for the getStatus operation in the server Description Testing * Add timeout logic for list operation Description Testing * Add timeout logic to cleanup operation Description Testing * Timeout logics in abort,status and list Description 1. abort: add timeout determine logic in the abort server side 2. status: provide approach for clients to check if error happens in the server side when uncompleted data is returned 3. list: provide approach for clients to check if error happens in the server side when uncompleted data is returned Testing * Add a server-side timeout configuration Description Testing * Add timeout check for operation locks in the server-side, and add timeoutFunction details in timeout logs Description Testing * Add logic to check if the peer database is alive in finish Description Testing * Timeout for the finish operation in the server-side Description Testing * Put timeout trace event logs at the single place Description Testing * Add timeout logic to the start operation in the server-side Description Testing * Unify traceEvent, fix timeout limit etc Description Testing * Fix timeout for finish operation and so on Description Testing * Record last running function Description Testing * If recovery fails in the start operation, then record the error in memory Description Testing * Remove the message string about last timeout function Description Testing * Add trySaveMovementRecord function to save record in memory, and attempt to set it into persistense Description Testing * Rewrite some logs and parallel actors in getStatusAndUpdateMovementRecord Description Testing * Use store to substitude the function waitAndAssign Description Testing * Remove the depreciated cleanup command's description in the usage text Description Testing * Fix bugs inside getActiveMovements and getStatusAndUpdateMovementRecord Description Testing * Fix bug in the backup.actor.cpp about convert maxLag. And remove onAbort in the clean operation in TenantBalancer Description Testing * simpilfy the code in the getStatusAndUpdateMovementRecord Description Testing * Format the outputs of status and list when dr status is not completed. Description Testing * test Description Testing * Fix issues with repeat error messages and remove errorMessage instance in the TenantMovementStatus Description Testing * Remove the test file, rename some variables etc. Description Testing

==70==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! ================================================================= ==70==ERROR: AddressSanitizer: heap-use-after-free on address 0x606001116b10 at pc 0x000004d25930 bp 0x7ffecf4763a0 sp 0x7ffecf476398 READ of size 8 at 0x606001116b10 thread T0 #0 0x4d2592f in SAV<QuorumVersion>::sendError(Error) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:672:23 apple#1 0x4d3054b in void Promise<QuorumVersion>::sendError<Error>(Error const&) const /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:910:8 apple#2 0x4d3054b in GetCommittedVersionQuorum::GetCommittedVersionActorActorState<GetCommittedVersionQuorum::GetCommittedVersionActorActor>::a_body1Catch2(Error const&, int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:211:29 apple#3 0x4d538a7 in GetCommittedVersionQuorum::GetCommittedVersionActorActorState<GetCommittedVersionQuorum::GetCommittedVersionActorActor>::a_callback_error(ActorCallback<GetCommittedVersionQuorum::GetCommittedVersionActorActor, 0, ConfigFollowerGetCommittedVersionReply>*, Error) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:922:4 apple#4 0x39d157c in SAV<ConfigFollowerGetCommittedVersionReply>::sendErrorAndDelPromiseRef(Error) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:717:23 apple#5 0x4d01df4 in (anonymous namespace)::TimeoutErrorActorState<ConfigFollowerGetCommittedVersionReply, (anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply> >::a_body1Catch1(Error, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/flow/genericactors.actor.g.h:3089:42 apple#6 0x4d01df4 in (anonymous namespace)::TimeoutErrorActorState<ConfigFollowerGetCommittedVersionReply, (anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply> >::a_body1when2(Void const&, int) /mnt/ephemeral/anoyes/foundationdb/flow/genericactors.actor.h:207:10 apple#7 0x4d01df4 in (anonymous namespace)::TimeoutErrorActorState<ConfigFollowerGetCommittedVersionReply, (anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply> >::a_callback_fire(ActorCallback<(anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply>, 1, Void>*, Void const&) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/flow/genericactors.actor.g.h:3191:4 apple#8 0x4d01df4 in ActorCallback<(anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply>, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:1318:34 apple#9 0x1d3d63f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:660:23 apple#10 0xa9daba9 in void Promise<Void>::send<Void>(Void&&) const /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:906:8 apple#11 0xa9daba9 in Sim2::execTask(Sim2::Task&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2155:14 apple#12 0xa9d9f16 in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1198:10 apple#13 0x437f9b3 in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2078:16 apple#14 0x7fcc15e40554 in __libc_start_main (/lib64/libc.so.6+0x22554) apple#15 0x1bd9028 in _start (/mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/bin/fdbserver+0x1bd9028) 0x606001116b10 is located 16 bytes inside of 64-byte region [0x606001116b00,0x606001116b40) freed by thread T0 here: #0 0x1c55352 in free /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:111:3 apple#1 0x4d2aff4 in FastAllocated<SAV<QuorumVersion> >::operator delete(void*) /mnt/ephemeral/anoyes/foundationdb/flow/FastAlloc.h:237:4 apple#2 0x4d2aff4 in SAV<QuorumVersion>::destroy() /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:755:3 apple#3 0x4d2568d in SAV<QuorumVersion>::delPromiseRef() /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:735:5 apple#4 0x4d2a502 in Promise<QuorumVersion>::operator=(Promise<QuorumVersion>&&) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:941:10 apple#5 0x4d2a502 in GetCommittedVersionQuorum::operator=(GetCommittedVersionQuorum&&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:42:7 apple#6 0x4d28556 in PaxosConfigConsumerImpl::resetCommittedVersionQuorum() /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:420:29 apple#7 0x4d6dc3e in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1Catch1cont2(int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:399:9 apple#8 0x4d6cb0f in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1Catch1(Error const&, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3156:18 apple#9 0x4d73ab4 in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_callback_error(ActorCallback<PaxosConfigConsumerImpl::FetchChangesActor, 1, long>*, Error) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3280:4 apple#10 0x207f3fc in SAV<long>::sendErrorAndDelPromiseRef(Error) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:717:23 apple#11 0x4d57363 in PaxosConfigConsumerImpl::GetCommittedVersionActorState<PaxosConfigConsumerImpl::GetCommittedVersionActor>::a_body1Catch1(Error, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:1617:49 apple#12 0x4d57363 in PaxosConfigConsumerImpl::GetCommittedVersionActorState<PaxosConfigConsumerImpl::GetCommittedVersionActor>::a_callback_error(ActorCallback<PaxosConfigConsumerImpl::GetCommittedVersionActor, 0, QuorumVersion>*, Error) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:1715:4 apple#13 0x4d2591c in SAV<QuorumVersion>::sendError(Error) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:673:23 apple#14 0x4d3054b in void Promise<QuorumVersion>::sendError<Error>(Error const&) const /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:910:8 apple#15 0x4d3054b in GetCommittedVersionQuorum::GetCommittedVersionActorActorState<GetCommittedVersionQuorum::GetCommittedVersionActorActor>::a_body1Catch2(Error const&, int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:211:29 apple#16 0x4d538a7 in GetCommittedVersionQuorum::GetCommittedVersionActorActorState<GetCommittedVersionQuorum::GetCommittedVersionActorActor>::a_callback_error(ActorCallback<GetCommittedVersionQuorum::GetCommittedVersionActorActor, 0, ConfigFollowerGetCommittedVersionReply>*, Error) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:922:4 apple#17 0x39d157c in SAV<ConfigFollowerGetCommittedVersionReply>::sendErrorAndDelPromiseRef(Error) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:717:23 apple#18 0x4d01df4 in (anonymous namespace)::TimeoutErrorActorState<ConfigFollowerGetCommittedVersionReply, (anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply> >::a_body1Catch1(Error, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/flow/genericactors.actor.g.h:3089:42 apple#19 0x4d01df4 in (anonymous namespace)::TimeoutErrorActorState<ConfigFollowerGetCommittedVersionReply, (anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply> >::a_body1when2(Void const&, int) /mnt/ephemeral/anoyes/foundationdb/flow/genericactors.actor.h:207:10 apple#20 0x4d01df4 in (anonymous namespace)::TimeoutErrorActorState<ConfigFollowerGetCommittedVersionReply, (anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply> >::a_callback_fire(ActorCallback<(anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply>, 1, Void>*, Void const&) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/flow/genericactors.actor.g.h:3191:4 apple#21 0x4d01df4 in ActorCallback<(anonymous namespace)::TimeoutErrorActor<ConfigFollowerGetCommittedVersionReply>, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:1318:34 apple#22 0x1d3d63f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:660:23 apple#23 0xa9daba9 in void Promise<Void>::send<Void>(Void&&) const /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:906:8 apple#24 0xa9daba9 in Sim2::execTask(Sim2::Task&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2155:14 apple#25 0xa9d9f16 in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1198:10 apple#26 0x437f9b3 in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2078:16 apple#27 0x7fcc15e40554 in __libc_start_main (/lib64/libc.so.6+0x22554) previously allocated by thread T0 here: #0 0x1c55eb2 in aligned_alloc /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:176:3 apple#1 0x4d6fe7d in FastAllocated<SAV<QuorumVersion> >::operator new(unsigned long) /mnt/ephemeral/anoyes/foundationdb/flow/FastAlloc.h:221:14 apple#2 0x4d6fe7d in Promise<QuorumVersion>::Promise() /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:922:18 apple#3 0x4d6fe7d in GetCommittedVersionQuorum::GetCommittedVersionQuorum(std::__1::vector<ConfigFollowerInterface, std::__1::allocator<ConfigFollowerInterface> > const&, long) /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:219:11 apple#4 0x4d6fe7d in PaxosConfigConsumerImpl::resetCommittedVersionQuorum() /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:420:31 apple#5 0x4d6fe7d in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont12(int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/PaxosConfigConsumer.actor.cpp:415:9 apple#6 0x4d6fe7d in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont13cont1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3857:16 apple#7 0x4d6ed1d in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont13(Void const&, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3781:15 apple#8 0x4d6ed1d in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont1when1(Void const&, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3793:15 apple#9 0x4d6ed1d in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3107:154 apple#10 0x4d732cc in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont11cont1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3519:16 apple#11 0x4d75e66 in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont11(Void const&, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3443:15 apple#12 0x4d75e66 in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_body1cont1loopBody1cont3when1(Void const&, int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3455:15 apple#13 0x4d75e66 in PaxosConfigConsumerImpl::FetchChangesActorState<PaxosConfigConsumerImpl::FetchChangesActor>::a_callback_fire(ActorCallback<PaxosConfigConsumerImpl::FetchChangesActor, 3, Void>*, Void const&) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.x86_64/fdbserver/PaxosConfigConsumer.actor.g.cpp:3476:4 apple#14 0x1d3d63f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:660:23 apple#15 0xa9daba9 in void Promise<Void>::send<Void>(Void&&) const /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:906:8 apple#16 0xa9daba9 in Sim2::execTask(Sim2::Task&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2155:14 apple#17 0xa9d9f16 in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1198:10 apple#18 0x437f9b3 in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2078:16 apple#19 0x7fcc15e40554 in __libc_start_main (/lib64/libc.so.6+0x22554) SUMMARY: AddressSanitizer: heap-use-after-free /mnt/ephemeral/anoyes/foundationdb/flow/flow.h:672:23 in SAV<QuorumVersion>::sendError(Error) Shadow bytes around the buggy address: 0x0c0c8021ad10: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c8021ad20: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c8021ad30: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd 0x0c0c8021ad40: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c8021ad50: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa =>0x0c0c8021ad60: fd fd[fd]fd fd fd fd fd fa fa fa fa fd fd fd fd 0x0c0c8021ad70: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c8021ad80: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c8021ad90: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd 0x0c0c8021ada0: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c8021adb0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==70==ABORTING

Previously we did not null out "attributes" in `Span& Span::operator=(Span&& o);`, but we destroyed the arena owning memory referenced by "attributes". Fix that by nulling out "attributes", and rewrite it in a way that's (hopefully) less error-prone. ASAN diagnostic: ``` ==24==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! ================================================================= ==24==ERROR: AddressSanitizer: heap-use-after-free on address 0x60300033fa06 at pc 0x0000026eadcf bp 0x7ffca646fe50 sp 0x7ffca646f618 READ of size 9 at 0x60300033fa06 thread T0 #0 0x26eadce in __asan_memmove /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 apple#1 0xd43db76 in __copy<const unsigned char, unsigned char> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:59:9 apple#2 0xd43db76 in copy<const unsigned char *, unsigned char *> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:72:13 apple#3 0xd43db76 in write_bytes /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:45:3 apple#4 0xd43db76 in serialize_string(unsigned char const*, int, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:109:6 apple#5 0xd43d208 in void serialize_map<SmallVectorRef<KeyValueRef, 1> >(SmallVectorRef<KeyValueRef, 1> const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:154:3 apple#6 0xd42e2a2 in (anonymous namespace)::UDPTracer::serialize_span(Span const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:157:3 apple#7 0xd42c8f4 in (anonymous namespace)::FastUDPTracer::trace(Span const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:301:3 apple#8 0xd41dfe2 in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:355:13 apple#9 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 apple#10 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 apple#11 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 apple#12 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 apple#13 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 apple#14 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 apple#15 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 apple#16 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 apple#17 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 apple#18 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) apple#19 0x266f028 in _start (/mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/bin/fdbserver+0x266f028) 0x60300033fa06 is located 6 bytes inside of 32-byte region [0x60300033fa00,0x60300033fa20) freed by thread T0 here: #0 0x26eb352 in free /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:111:3 apple#1 0xdd90118 in ArenaBlock::destroy() /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:466:6 apple#2 0xdd8f0b0 in delref /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:173:3 apple#3 0xdd8f0b0 in delref<ArenaBlock> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:95:7 apple#4 0xdd8f0b0 in operator= /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:147:5 apple#5 0xdd8f0b0 in Arena::operator=(Arena&&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:119:36 apple#6 0xd41dfed in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:357:8 apple#7 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 apple#8 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 apple#9 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 apple#10 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 apple#11 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 apple#12 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 apple#13 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 apple#14 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 apple#15 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 apple#16 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) previously allocated by thread T0 here: #0 0x26ebeb2 in aligned_alloc /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:176:3 apple#1 0xdd8e5e0 in ArenaBlock::create(int, Reference<ArenaBlock>&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:339:21 apple#2 0xdd91139 in ArenaBlock::allocate(Reference<ArenaBlock>&, int) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:322:7 apple#3 0x2bbf49b in operator new[] /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:206:9 apple#4 0x2bbf49b in StringRef /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:446:54 apple#5 0x2bbf49b in Span::Span(SpanContext const&, Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:141:33 apple#6 0x2bbef25 in Span::Span(Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:148:6 apple#7 0xc1bd941 in Span /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:160:44 apple#8 0xc1bd941 in ReadVersionBatcherActorState /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7125:6 apple#9 0xc1bd941 in ReadVersionBatcherActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32621:6 apple#10 0xc1bd941 in readVersionBatcher(DatabaseContext* const&, FutureStream<DatabaseContext::VersionRequest> const&, TransactionPriority const&, unsigned int const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7098:26 apple#11 0xc1b316a in Transaction::getReadVersion(unsigned int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7366:8 apple#12 0xc19055f in getReadVersion /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/NativeAPI.actor.h:315:44 apple#13 0xc19055f in Transaction::get(Standalone<StringRef> const&, Snapshot) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:5314:13 apple#14 0x5eca2f1 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:190:55 apple#15 0x5eca2f1 in ReadMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1001:9 apple#16 0x5eca2f1 in readMoveKeysLock(Transaction* const&, MoveKeysLock* const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:188:26 apple#17 0x5ee590d in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:228:39 apple#18 0x5ee590d in (anonymous namespace)::TakeMoveKeysLockActorState<(anonymous namespace)::TakeMoveKeysLockActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1421:49 apple#19 0x5ecb801 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1400:16 apple#20 0x5ecb801 in TakeMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1807:9 apple#21 0x5ecb801 in takeMoveKeysLock(Database const&, UID const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:216:34 apple#22 0x542a421 in DDTxnProcessor::takeMoveKeysLock(UID const&) const /mnt/ephemeral/anoyes/foundationdb/fdbserver/DDTxnProcessor.actor.cpp:620:9 apple#23 0x562aa75 in takeMoveKeysLock /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:330:69 apple#24 0x562aa75 in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1loopBody1(int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:370:44 apple#25 0x562a0ba in a_body1loopHead1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1116:49 apple#26 0x562a0ba in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1083:16 apple#27 0x55088c0 in InitActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1715:9 apple#28 0x55088c0 in init /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:367:26 apple#29 0x55088c0 in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:611:39 apple#30 0x55088c0 in (anonymous namespace)::DataDistributionActorState<(anonymous namespace)::DataDistributionActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2772:49 apple#31 0x54e9cf1 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2751:16 apple#32 0x54e9cf1 in DataDistributionActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:3976:9 apple#33 0x54e9cf1 in dataDistribution(Reference<DataDistributor> const&, PromiseStream<GetMetricsListRequest> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:586:26 apple#34 0x54f8758 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1533:39 apple#35 0x54f8758 in DataDistributorActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:11473:9 apple#36 0x54f8758 in dataDistributor(DataDistributorInterface const&, Reference<AsyncVar<ServerDBInfo> const> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1517:26 apple#37 0x88f3f88 in (anonymous namespace)::WorkerServerActorState<(anonymous namespace)::WorkerServerActor>::a_body1cont10loopBody1when6(InitializeDataDistributorRequest&&, int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/worker.actor.cpp:2219:42 apple#38 0x8871c15 in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/worker.actor.g.cpp:13169:4 apple#39 0x8871c15 in ActorSingleCallback<(anonymous namespace)::WorkerServerActor, 6, InitializeDataDistributorRequest>::fire(InitializeDataDistributorRequest&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1338:34 apple#40 0x33e13c0 in send<InitializeDataDistributorRequest> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1004:29 apple#41 0x33e13c0 in NetNotifiedQueue<InitializeDataDistributorRequest, false>::receive(ArenaObjectReader&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/fdbrpc.h:702:10 apple#42 0xd91b6c6 in (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_body1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/FlowTransport.actor.cpp:1042:15 apple#43 0xd91a37d in a_body1cont2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4333:15 apple#44 0xd91a37d in a_body1when1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4345:15 apple#45 0xd91a37d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4366:4 apple#46 0xd91a37d in ActorCallback<(anonymous namespace)::DeliverActor, 0, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 apple#47 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 apple#48 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 apple#49 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 apple#50 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 apple#51 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 apple#52 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) SUMMARY: AddressSanitizer: heap-use-after-free /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 in __asan_memmove Shadow bytes around the buggy address: 0x0c068005fef0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd 0x0c068005ff00: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa 0x0c068005ff10: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff20: fa fa fd fd fd fa fa fa fd fd fd fd fa fa fd fd 0x0c068005ff30: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa =>0x0c068005ff40:[fd]fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff50: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd 0x0c068005ff60: fd fa fa fa fd fd fd fa fa fa fd fd fd fd fa fa 0x0c068005ff70: fd fd fd fd fa fa fd fd fd fa fa fa 00 00 00 00 0x0c068005ff80: fa fa fd fd fd fd fa fa 00 00 00 00 fa fa fd fd 0x0c068005ff90: fd fd fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ```

Previously we did not null out "attributes" in `Span& Span::operator=(Span&& o);`, but we destroyed the arena owning memory referenced by "attributes". Fix that by nulling out "attributes", and rewrite it in a way that's (hopefully) less error-prone. ASAN diagnostic: ``` ==24==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! ================================================================= ==24==ERROR: AddressSanitizer: heap-use-after-free on address 0x60300033fa06 at pc 0x0000026eadcf bp 0x7ffca646fe50 sp 0x7ffca646f618 READ of size 9 at 0x60300033fa06 thread T0 #0 0x26eadce in __asan_memmove /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 #1 0xd43db76 in __copy<const unsigned char, unsigned char> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:59:9 #2 0xd43db76 in copy<const unsigned char *, unsigned char *> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:72:13 #3 0xd43db76 in write_bytes /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:45:3 #4 0xd43db76 in serialize_string(unsigned char const*, int, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:109:6 #5 0xd43d208 in void serialize_map<SmallVectorRef<KeyValueRef, 1> >(SmallVectorRef<KeyValueRef, 1> const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:154:3 #6 0xd42e2a2 in (anonymous namespace)::UDPTracer::serialize_span(Span const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:157:3 #7 0xd42c8f4 in (anonymous namespace)::FastUDPTracer::trace(Span const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:301:3 #8 0xd41dfe2 in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:355:13 #9 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 #10 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 #11 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 #12 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 #13 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 #14 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 #15 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 #16 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 #17 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 #18 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) #19 0x266f028 in _start (/mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/bin/fdbserver+0x266f028) 0x60300033fa06 is located 6 bytes inside of 32-byte region [0x60300033fa00,0x60300033fa20) freed by thread T0 here: #0 0x26eb352 in free /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:111:3 #1 0xdd90118 in ArenaBlock::destroy() /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:466:6 #2 0xdd8f0b0 in delref /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:173:3 #3 0xdd8f0b0 in delref<ArenaBlock> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:95:7 #4 0xdd8f0b0 in operator= /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:147:5 #5 0xdd8f0b0 in Arena::operator=(Arena&&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:119:36 #6 0xd41dfed in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:357:8 #7 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 #8 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 #9 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 #10 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 #11 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 #12 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 #13 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 #14 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 #15 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 #16 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) previously allocated by thread T0 here: #0 0x26ebeb2 in aligned_alloc /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:176:3 #1 0xdd8e5e0 in ArenaBlock::create(int, Reference<ArenaBlock>&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:339:21 #2 0xdd91139 in ArenaBlock::allocate(Reference<ArenaBlock>&, int) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:322:7 #3 0x2bbf49b in operator new[] /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:206:9 #4 0x2bbf49b in StringRef /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:446:54 #5 0x2bbf49b in Span::Span(SpanContext const&, Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:141:33 #6 0x2bbef25 in Span::Span(Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:148:6 #7 0xc1bd941 in Span /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:160:44 #8 0xc1bd941 in ReadVersionBatcherActorState /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7125:6 #9 0xc1bd941 in ReadVersionBatcherActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32621:6 #10 0xc1bd941 in readVersionBatcher(DatabaseContext* const&, FutureStream<DatabaseContext::VersionRequest> const&, TransactionPriority const&, unsigned int const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7098:26 #11 0xc1b316a in Transaction::getReadVersion(unsigned int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7366:8 #12 0xc19055f in getReadVersion /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/NativeAPI.actor.h:315:44 #13 0xc19055f in Transaction::get(Standalone<StringRef> const&, Snapshot) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:5314:13 #14 0x5eca2f1 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:190:55 #15 0x5eca2f1 in ReadMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1001:9 #16 0x5eca2f1 in readMoveKeysLock(Transaction* const&, MoveKeysLock* const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:188:26 #17 0x5ee590d in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:228:39 #18 0x5ee590d in (anonymous namespace)::TakeMoveKeysLockActorState<(anonymous namespace)::TakeMoveKeysLockActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1421:49 #19 0x5ecb801 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1400:16 #20 0x5ecb801 in TakeMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1807:9 #21 0x5ecb801 in takeMoveKeysLock(Database const&, UID const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:216:34 #22 0x542a421 in DDTxnProcessor::takeMoveKeysLock(UID const&) const /mnt/ephemeral/anoyes/foundationdb/fdbserver/DDTxnProcessor.actor.cpp:620:9 #23 0x562aa75 in takeMoveKeysLock /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:330:69 #24 0x562aa75 in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1loopBody1(int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:370:44 #25 0x562a0ba in a_body1loopHead1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1116:49 #26 0x562a0ba in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1083:16 #27 0x55088c0 in InitActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1715:9 #28 0x55088c0 in init /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:367:26 #29 0x55088c0 in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:611:39 #30 0x55088c0 in (anonymous namespace)::DataDistributionActorState<(anonymous namespace)::DataDistributionActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2772:49 #31 0x54e9cf1 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2751:16 #32 0x54e9cf1 in DataDistributionActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:3976:9 #33 0x54e9cf1 in dataDistribution(Reference<DataDistributor> const&, PromiseStream<GetMetricsListRequest> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:586:26 #34 0x54f8758 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1533:39 #35 0x54f8758 in DataDistributorActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:11473:9 #36 0x54f8758 in dataDistributor(DataDistributorInterface const&, Reference<AsyncVar<ServerDBInfo> const> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1517:26 #37 0x88f3f88 in (anonymous namespace)::WorkerServerActorState<(anonymous namespace)::WorkerServerActor>::a_body1cont10loopBody1when6(InitializeDataDistributorRequest&&, int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/worker.actor.cpp:2219:42 #38 0x8871c15 in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/worker.actor.g.cpp:13169:4 #39 0x8871c15 in ActorSingleCallback<(anonymous namespace)::WorkerServerActor, 6, InitializeDataDistributorRequest>::fire(InitializeDataDistributorRequest&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1338:34 #40 0x33e13c0 in send<InitializeDataDistributorRequest> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1004:29 #41 0x33e13c0 in NetNotifiedQueue<InitializeDataDistributorRequest, false>::receive(ArenaObjectReader&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/fdbrpc.h:702:10 #42 0xd91b6c6 in (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_body1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/FlowTransport.actor.cpp:1042:15 #43 0xd91a37d in a_body1cont2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4333:15 #44 0xd91a37d in a_body1when1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4345:15 #45 0xd91a37d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4366:4 #46 0xd91a37d in ActorCallback<(anonymous namespace)::DeliverActor, 0, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 #47 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 #48 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 #49 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 #50 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 #51 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 #52 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) SUMMARY: AddressSanitizer: heap-use-after-free /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 in __asan_memmove Shadow bytes around the buggy address: 0x0c068005fef0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd 0x0c068005ff00: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa 0x0c068005ff10: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff20: fa fa fd fd fd fa fa fa fd fd fd fd fa fa fd fd 0x0c068005ff30: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa =>0x0c068005ff40:[fd]fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff50: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd 0x0c068005ff60: fd fa fa fa fd fd fd fa fa fa fd fd fd fd fa fa 0x0c068005ff70: fd fd fd fd fa fa fd fd fd fa fa fa 00 00 00 00 0x0c068005ff80: fa fa fd fd fd fd fa fa 00 00 00 00 fa fa fd fd 0x0c068005ff90: fd fd fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ```

Previously we did not null out "attributes" in `Span& Span::operator=(Span&& o);`, but we destroyed the arena owning memory referenced by "attributes". Fix that by nulling out "attributes", and rewrite it in a way that's (hopefully) less error-prone. ASAN diagnostic: ``` ==24==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! ================================================================= ==24==ERROR: AddressSanitizer: heap-use-after-free on address 0x60300033fa06 at pc 0x0000026eadcf bp 0x7ffca646fe50 sp 0x7ffca646f618 READ of size 9 at 0x60300033fa06 thread T0 #0 0x26eadce in __asan_memmove /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 #1 0xd43db76 in __copy<const unsigned char, unsigned char> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:59:9 #2 0xd43db76 in copy<const unsigned char *, unsigned char *> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:72:13 #3 0xd43db76 in write_bytes /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:45:3 #4 0xd43db76 in serialize_string(unsigned char const*, int, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:109:6 #5 0xd43d208 in void serialize_map<SmallVectorRef<KeyValueRef, 1> >(SmallVectorRef<KeyValueRef, 1> const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:154:3 #6 0xd42e2a2 in (anonymous namespace)::UDPTracer::serialize_span(Span const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:157:3 #7 0xd42c8f4 in (anonymous namespace)::FastUDPTracer::trace(Span const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:301:3 #8 0xd41dfe2 in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:355:13 #9 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 #10 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 #11 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 apple#12 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 apple#13 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 apple#14 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 apple#15 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 apple#16 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 apple#17 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 apple#18 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) apple#19 0x266f028 in _start (/mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/bin/fdbserver+0x266f028) 0x60300033fa06 is located 6 bytes inside of 32-byte region [0x60300033fa00,0x60300033fa20) freed by thread T0 here: #0 0x26eb352 in free /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:111:3 #1 0xdd90118 in ArenaBlock::destroy() /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:466:6 #2 0xdd8f0b0 in delref /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:173:3 #3 0xdd8f0b0 in delref<ArenaBlock> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:95:7 #4 0xdd8f0b0 in operator= /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:147:5 #5 0xdd8f0b0 in Arena::operator=(Arena&&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:119:36 #6 0xd41dfed in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:357:8 #7 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 #8 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 #9 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 #10 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 #11 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 apple#12 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 apple#13 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 apple#14 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 apple#15 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 apple#16 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) previously allocated by thread T0 here: #0 0x26ebeb2 in aligned_alloc /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:176:3 #1 0xdd8e5e0 in ArenaBlock::create(int, Reference<ArenaBlock>&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:339:21 #2 0xdd91139 in ArenaBlock::allocate(Reference<ArenaBlock>&, int) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:322:7 #3 0x2bbf49b in operator new[] /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:206:9 #4 0x2bbf49b in StringRef /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:446:54 #5 0x2bbf49b in Span::Span(SpanContext const&, Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:141:33 #6 0x2bbef25 in Span::Span(Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:148:6 #7 0xc1bd941 in Span /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:160:44 #8 0xc1bd941 in ReadVersionBatcherActorState /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7125:6 #9 0xc1bd941 in ReadVersionBatcherActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32621:6 #10 0xc1bd941 in readVersionBatcher(DatabaseContext* const&, FutureStream<DatabaseContext::VersionRequest> const&, TransactionPriority const&, unsigned int const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7098:26 #11 0xc1b316a in Transaction::getReadVersion(unsigned int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7366:8 apple#12 0xc19055f in getReadVersion /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/NativeAPI.actor.h:315:44 apple#13 0xc19055f in Transaction::get(Standalone<StringRef> const&, Snapshot) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:5314:13 apple#14 0x5eca2f1 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:190:55 apple#15 0x5eca2f1 in ReadMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1001:9 apple#16 0x5eca2f1 in readMoveKeysLock(Transaction* const&, MoveKeysLock* const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:188:26 apple#17 0x5ee590d in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:228:39 apple#18 0x5ee590d in (anonymous namespace)::TakeMoveKeysLockActorState<(anonymous namespace)::TakeMoveKeysLockActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1421:49 apple#19 0x5ecb801 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1400:16 apple#20 0x5ecb801 in TakeMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1807:9 apple#21 0x5ecb801 in takeMoveKeysLock(Database const&, UID const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:216:34 apple#22 0x542a421 in DDTxnProcessor::takeMoveKeysLock(UID const&) const /mnt/ephemeral/anoyes/foundationdb/fdbserver/DDTxnProcessor.actor.cpp:620:9 apple#23 0x562aa75 in takeMoveKeysLock /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:330:69 apple#24 0x562aa75 in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1loopBody1(int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:370:44 apple#25 0x562a0ba in a_body1loopHead1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1116:49 apple#26 0x562a0ba in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1083:16 apple#27 0x55088c0 in InitActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1715:9 apple#28 0x55088c0 in init /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:367:26 apple#29 0x55088c0 in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:611:39 apple#30 0x55088c0 in (anonymous namespace)::DataDistributionActorState<(anonymous namespace)::DataDistributionActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2772:49 apple#31 0x54e9cf1 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2751:16 apple#32 0x54e9cf1 in DataDistributionActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:3976:9 apple#33 0x54e9cf1 in dataDistribution(Reference<DataDistributor> const&, PromiseStream<GetMetricsListRequest> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:586:26 apple#34 0x54f8758 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1533:39 apple#35 0x54f8758 in DataDistributorActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:11473:9 apple#36 0x54f8758 in dataDistributor(DataDistributorInterface const&, Reference<AsyncVar<ServerDBInfo> const> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1517:26 apple#37 0x88f3f88 in (anonymous namespace)::WorkerServerActorState<(anonymous namespace)::WorkerServerActor>::a_body1cont10loopBody1when6(InitializeDataDistributorRequest&&, int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/worker.actor.cpp:2219:42 apple#38 0x8871c15 in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/worker.actor.g.cpp:13169:4 apple#39 0x8871c15 in ActorSingleCallback<(anonymous namespace)::WorkerServerActor, 6, InitializeDataDistributorRequest>::fire(InitializeDataDistributorRequest&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1338:34 apple#40 0x33e13c0 in send<InitializeDataDistributorRequest> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1004:29 apple#41 0x33e13c0 in NetNotifiedQueue<InitializeDataDistributorRequest, false>::receive(ArenaObjectReader&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/fdbrpc.h:702:10 apple#42 0xd91b6c6 in (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_body1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/FlowTransport.actor.cpp:1042:15 apple#43 0xd91a37d in a_body1cont2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4333:15 apple#44 0xd91a37d in a_body1when1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4345:15 apple#45 0xd91a37d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4366:4 apple#46 0xd91a37d in ActorCallback<(anonymous namespace)::DeliverActor, 0, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 apple#47 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 apple#48 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 apple#49 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 apple#50 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 apple#51 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 apple#52 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) SUMMARY: AddressSanitizer: heap-use-after-free /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 in __asan_memmove Shadow bytes around the buggy address: 0x0c068005fef0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd 0x0c068005ff00: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa 0x0c068005ff10: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff20: fa fa fd fd fd fa fa fa fd fd fd fd fa fa fd fd 0x0c068005ff30: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa =>0x0c068005ff40:[fd]fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff50: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd 0x0c068005ff60: fd fa fa fa fd fd fd fa fa fa fd fd fd fd fa fa 0x0c068005ff70: fd fd fd fd fa fa fd fd fd fa fa fa 00 00 00 00 0x0c068005ff80: fa fa fd fd fd fd fa fa 00 00 00 00 fa fa fd fd 0x0c068005ff90: fd fd fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ```

Previously we did not null out "attributes" in `Span& Span::operator=(Span&& o);`, but we destroyed the arena owning memory referenced by "attributes". Fix that by nulling out "attributes", and rewrite it in a way that's (hopefully) less error-prone. ASAN diagnostic: ``` ==24==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! ================================================================= ==24==ERROR: AddressSanitizer: heap-use-after-free on address 0x60300033fa06 at pc 0x0000026eadcf bp 0x7ffca646fe50 sp 0x7ffca646f618 READ of size 9 at 0x60300033fa06 thread T0 #0 0x26eadce in __asan_memmove /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 #1 0xd43db76 in __copy<const unsigned char, unsigned char> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:59:9 #2 0xd43db76 in copy<const unsigned char *, unsigned char *> /usr/local/bin/../include/c++/v1/__algorithm/copy.h:72:13 #3 0xd43db76 in write_bytes /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:45:3 #4 0xd43db76 in serialize_string(unsigned char const*, int, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:109:6 #5 0xd43d208 in void serialize_map<SmallVectorRef<KeyValueRef, 1> >(SmallVectorRef<KeyValueRef, 1> const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/Msgpack.h:154:3 #6 0xd42e2a2 in (anonymous namespace)::UDPTracer::serialize_span(Span const&, MsgpackBuffer&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:157:3 #7 0xd42c8f4 in (anonymous namespace)::FastUDPTracer::trace(Span const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:301:3 #8 0xd41dfe2 in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:355:13 #9 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 #10 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 #11 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 #12 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 #13 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 #14 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 #15 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 #16 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 #17 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 #18 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) #19 0x266f028 in _start (/mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/bin/fdbserver+0x266f028) 0x60300033fa06 is located 6 bytes inside of 32-byte region [0x60300033fa00,0x60300033fa20) freed by thread T0 here: #0 0x26eb352 in free /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:111:3 #1 0xdd90118 in ArenaBlock::destroy() /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:466:6 #2 0xdd8f0b0 in delref /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:173:3 #3 0xdd8f0b0 in delref<ArenaBlock> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:95:7 #4 0xdd8f0b0 in operator= /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/FastRef.h:147:5 #5 0xdd8f0b0 in Arena::operator=(Arena&&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:119:36 #6 0xd41dfed in Span::operator=(Span&&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/Tracing.actor.cpp:357:8 #7 0xc3eb122 in (anonymous namespace)::ReadVersionBatcherActorState<(anonymous namespace)::ReadVersionBatcherActor>::a_body1loopBody1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7180:9 #8 0xc3e386d in a_body1loopBody1when2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32322:15 #9 0xc3e386d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32434:4 #10 0xc3e386d in ActorCallback<(anonymous namespace)::ReadVersionBatcherActor, 1, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 #11 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 #12 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 #13 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 #14 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 #15 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 #16 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) previously allocated by thread T0 here: #0 0x26ebeb2 in aligned_alloc /tmp/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:176:3 #1 0xdd8e5e0 in ArenaBlock::create(int, Reference<ArenaBlock>&) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:339:21 #2 0xdd91139 in ArenaBlock::allocate(Reference<ArenaBlock>&, int) /mnt/ephemeral/anoyes/foundationdb/flow/Arena.cpp:322:7 #3 0x2bbf49b in operator new[] /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:206:9 #4 0x2bbf49b in StringRef /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/Arena.h:446:54 #5 0x2bbf49b in Span::Span(SpanContext const&, Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:141:33 #6 0x2bbef25 in Span::Span(Location const&, SpanContext const&, std::initializer_list<SpanContext> const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:148:6 #7 0xc1bd941 in Span /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/Tracing.h:160:44 #8 0xc1bd941 in ReadVersionBatcherActorState /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7125:6 #9 0xc1bd941 in ReadVersionBatcherActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbclient/NativeAPI.actor.g.cpp:32621:6 #10 0xc1bd941 in readVersionBatcher(DatabaseContext* const&, FutureStream<DatabaseContext::VersionRequest> const&, TransactionPriority const&, unsigned int const&) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7098:26 #11 0xc1b316a in Transaction::getReadVersion(unsigned int) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:7366:8 #12 0xc19055f in getReadVersion /mnt/ephemeral/anoyes/foundationdb/fdbclient/include/fdbclient/NativeAPI.actor.h:315:44 #13 0xc19055f in Transaction::get(Standalone<StringRef> const&, Snapshot) /mnt/ephemeral/anoyes/foundationdb/fdbclient/NativeAPI.actor.cpp:5314:13 #14 0x5eca2f1 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:190:55 #15 0x5eca2f1 in ReadMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1001:9 #16 0x5eca2f1 in readMoveKeysLock(Transaction* const&, MoveKeysLock* const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:188:26 #17 0x5ee590d in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:228:39 #18 0x5ee590d in (anonymous namespace)::TakeMoveKeysLockActorState<(anonymous namespace)::TakeMoveKeysLockActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1421:49 #19 0x5ecb801 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1400:16 #20 0x5ecb801 in TakeMoveKeysLockActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/MoveKeys.actor.g.cpp:1807:9 #21 0x5ecb801 in takeMoveKeysLock(Database const&, UID const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/MoveKeys.actor.cpp:216:34 #22 0x542a421 in DDTxnProcessor::takeMoveKeysLock(UID const&) const /mnt/ephemeral/anoyes/foundationdb/fdbserver/DDTxnProcessor.actor.cpp:620:9 #23 0x562aa75 in takeMoveKeysLock /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:330:69 #24 0x562aa75 in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1loopBody1(int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:370:44 #25 0x562a0ba in a_body1loopHead1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1116:49 #26 0x562a0ba in DataDistributor::InitActorState<DataDistributor::InitActor>::a_body1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1083:16 #27 0x55088c0 in InitActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:1715:9 #28 0x55088c0 in init /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:367:26 #29 0x55088c0 in a_body1loopBody1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:611:39 #30 0x55088c0 in (anonymous namespace)::DataDistributionActorState<(anonymous namespace)::DataDistributionActor>::a_body1loopHead1(int) /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2772:49 #31 0x54e9cf1 in a_body1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:2751:16 #32 0x54e9cf1 in DataDistributionActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:3976:9 #33 0x54e9cf1 in dataDistribution(Reference<DataDistributor> const&, PromiseStream<GetMetricsListRequest> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:586:26 #34 0x54f8758 in a_body1 /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1533:39 #35 0x54f8758 in DataDistributorActor /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/DataDistribution.actor.g.cpp:11473:9 #36 0x54f8758 in dataDistributor(DataDistributorInterface const&, Reference<AsyncVar<ServerDBInfo> const> const&) /mnt/ephemeral/anoyes/foundationdb/fdbserver/DataDistribution.actor.cpp:1517:26 #37 0x88f3f88 in (anonymous namespace)::WorkerServerActorState<(anonymous namespace)::WorkerServerActor>::a_body1cont10loopBody1when6(InitializeDataDistributorRequest&&, int) /mnt/ephemeral/anoyes/foundationdb/fdbserver/worker.actor.cpp:2219:42 #38 0x8871c15 in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbserver/worker.actor.g.cpp:13169:4 #39 0x8871c15 in ActorSingleCallback<(anonymous namespace)::WorkerServerActor, 6, InitializeDataDistributorRequest>::fire(InitializeDataDistributorRequest&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1338:34 #40 0x33e13c0 in send<InitializeDataDistributorRequest> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1004:29 #41 0x33e13c0 in NetNotifiedQueue<InitializeDataDistributorRequest, false>::receive(ArenaObjectReader&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/include/fdbrpc/fdbrpc.h:702:10 #42 0xd91b6c6 in (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_body1cont1(int) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/FlowTransport.actor.cpp:1042:15 #43 0xd91a37d in a_body1cont2 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4333:15 #44 0xd91a37d in a_body1when1 /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4345:15 #45 0xd91a37d in a_callback_fire /mnt/ephemeral/anoyes/build/foundationdb.linux.clang.asan.x86_64/fdbrpc/FlowTransport.actor.g.cpp:4366:4 #46 0xd91a37d in ActorCallback<(anonymous namespace)::DeliverActor, 0, Void>::fire(Void const&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:1316:34 #47 0x278af3f in void SAV<Void>::send<Void>(Void&&) /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:655:23 #48 0xdd2dc36 in send<Void> /mnt/ephemeral/anoyes/foundationdb/flow/include/flow/flow.h:901:8 #49 0xdd2dc36 in Sim2::execTask(Sim2::PromiseTask&) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:2319:15 #50 0xdd2cf0e in Sim2::runLoop(Sim2*) /mnt/ephemeral/anoyes/foundationdb/fdbrpc/sim2.actor.cpp:1279:11 #51 0x7ebdf2e in main /mnt/ephemeral/anoyes/foundationdb/fdbserver/fdbserver.actor.cpp:2276:17 #52 0x7f62c7be6554 in __libc_start_main (/lib64/libc.so.6+0x22554) SUMMARY: AddressSanitizer: heap-use-after-free /tmp/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3 in __asan_memmove Shadow bytes around the buggy address: 0x0c068005fef0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd 0x0c068005ff00: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa 0x0c068005ff10: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff20: fa fa fd fd fd fa fa fa fd fd fd fd fa fa fd fd 0x0c068005ff30: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa =>0x0c068005ff40:[fd]fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd 0x0c068005ff50: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd 0x0c068005ff60: fd fa fa fa fd fd fd fa fa fa fd fd fd fd fa fa 0x0c068005ff70: fd fd fd fd fa fa fd fd fd fa fa fa 00 00 00 00 0x0c068005ff80: fa fa fd fd fd fd fa fa 00 00 00 00 fa fa fd fd 0x0c068005ff90: fd fd fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ```

Blob client send and receive speed limits were being initialized usin…

d0e122f

…g opposite knobs.

satherton changed the title ~~Blob speed knob swap~~ Fixed blob speed knob initialization Mar 5, 2018

satherton merged commit cae075e into apple:release-5.1 Mar 5, 2018

satherton deleted the fix-speed-knobs branch March 5, 2018 07:11

sfc-gh-anoyes added a commit to sfc-gh-anoyes/foundationdb that referenced this pull request Jul 1, 2020

Merge pull request apple#23 from snowflakedb/anoyes/set-java-rpath

0a895ff

SNOW-152673 Set rpath to /usr/lib64/fdb6/

sfc-gh-ljoswiak added a commit that referenced this pull request May 3, 2021

Merge pull request #23 from sfc-gh-ljoswiak/fixes/gcc

1c1125c

Fix gcc build

sfc-gh-jslocum mentioned this pull request Aug 1, 2022

ASAN test failing on main #7748

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed blob speed knob initialization #23

Fixed blob speed knob initialization #23

satherton commented Mar 5, 2018

Fixed blob speed knob initialization #23

Fixed blob speed knob initialization #23

Conversation

satherton commented Mar 5, 2018