Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

More fearless features #63

Merged
merged 11 commits into from
Mar 14, 2018
Merged

More fearless features #63

merged 11 commits into from
Mar 14, 2018

Conversation

etschannen
Copy link
Contributor

No description provided.

@etschannen
changed remoteDcId to a vector of ids, to support future configuratio…
470f5c0 
…ns where there are multiple remote databases
@etschannen
added region-based configuration to support a large variety of fearle…
1194e3a 
…ss setups. Currently only 1 primary 1 remote setups are allowed.
@etschannen
fix: we must commit to the number of log routers we are going to use …
8c88041 
…when recruiting the primary, because it determines the number of log router tags that will be attached to mutations
@etschannen
fix: sim2 logic for when a kill is safe was incorrect
68606c7
@etschannen
fix: inactive cursors are still useful if their version is larger tha…
9d4cdc8 
…n the current version
@etschannen
fix: shards affected by team failure did not properly handle separate…
fa7eaea 
… teams for the remote and primary data centers
@etschannen
Merge branch 'master' into feature-remote-logs
3abf4d7
@etschannen
fix: merge problem
c74211b
@etschannen
fix: do not serialize an a tlog interface without a unique id
72d56a7
@etschannen
fix: the recovery actor was holding a copy of the tlogInterface after…
f6a22c1 
… the tlog was removed
alexmiller-apple
alexmiller-apple suggested changes Mar 13, 2018
View reviewed changes
fdbserver/DatabaseConfiguration.cpp Outdated
@@ -75,13 +128,15 @@ void DatabaseConfiguration::setDefaultReplicationPolicy() {
if(remoteTLogReplicationFactor > 0 && !remoteTLogPolicy) {
remoteTLogPolicy = IRepPolicyRef(new PolicyAcross(remoteTLogReplicationFactor, "zoneid", IRepPolicyRef(new PolicyOne())));
}
if(satelliteTLogReplicationFactor > 0 && !satelliteTLogPolicy) {
satelliteTLogPolicy = IRepPolicyRef(new PolicyAcross(satelliteTLogReplicationFactor, "zoneid", IRepPolicyRef(new PolicyOne())));
for(auto r : regions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're missing a reference here? Your satellite policy might thus disappearing into the ether?

fdbserver/DatabaseConfiguration.cpp
return false;
}
dcIds.insert(r.dcId);
priorities.insert(r.priority);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priorities doesn't then get used anywhere?

fdbserver/DatabaseConfiguration.cpp Outdated
std::set<Key> dcIds;
std::set<int> priorities;
dcIds.insert(Key());
for(auto r : regions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think r and s are a correctness problem here, but a const & would do no harm.

fdbserver/DatabaseConfiguration.cpp Outdated
result["logs"] = format("%d", desiredTLogCount);
if(regions.size()) {
StatusArray regionArr;
for( auto r : regions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RegionInfo copy

fdbserver/DatabaseConfiguration.cpp Outdated
result["satellite_logs"] = format("%d", satelliteDesiredTLogCount);
if(r.satellites.size()) {
StatusArray satellitesArr;
for(auto s : r.satellites) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SatelliteInfo copy

fdbserver/DatabaseConfiguration.h Outdated
if(!primaryDcId.present()) return 1;
return 2 + primarySatelliteDcIds.size() + remoteSatelliteDcIds.size();
int minRequired = 0;
for(auto r : regions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RegionInfo copy. And many more, so I'm going to stop commenting on them one by one.

fdbclient/ManagementAPI.actor.cpp Outdated
out[p+key] = value;
if( key == "regions" ) {
json_spirit::mValue mv;
json_spirit::read_string( value, mv );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is JSON the permanent plan here...?

fdbserver/DataDistribution.actor.cpp
state Reference<InitialDataDistribution> result = Reference<InitialDataDistribution>(new InitialDataDistribution);
state Key beginKey = allKeys.begin;

state bool succeeded;

state Transaction tr( cx );

state std::map<UID, Optional<Key>> server_dc;
state std::map<vector<UID>, std::pair<vector<UID>, vector<UID>>> team_cache;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to double check, the possible number of distinct vector keys that team_cache can have is pretty constrained, so the fact that we never remove entries isn't a memory concern?

@etschannen etschannen merged commit 8045b73 into apple:master Mar 14, 2018
etschannen added a commit to etschannen/foundationdb that referenced this pull request Mar 26, 2019
@etschannen
Merge pull request apple#63 from etschannen/feature-remote-logs
ce5b908 
More fearless features
atn34 added a commit to atn34/foundationdb that referenced this pull request Dec 3, 2019
@atn34
Fix UBSAN report
47de9d9 
/Users/anoyes/workspace/foundationdb/fdbserver/TLogInterface.h:149:8: runtime error: load of value 232, which is not a valid value for type 'bool'
    #0 0xc608fb in TLogPeekReply::TLogPeekReply(TLogPeekReply const&) /Users/anoyes/workspace/foundationdb/fdbserver/TLogInterface.h:149
    #1 0x242bf87 in ILogSystem::ServerPeekCursor::ServerPeekCursor(TLogPeekReply const&, LogMessageVersion const&, LogMessageVersion const&, int, int, bool, long, Tag) /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:35
    #2 0x242da77 in ILogSystem::ServerPeekCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:47
    #3 0x24362d5 in ILogSystem::MergedPeekCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:325
    apple#4 0x244bf45 in ILogSystem::MultiCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:838
    apple#5 0x36b5a36 in a_body1cont5loopBody1 /Users/anoyes/workspace/foundationdb/fdbserver/storageserver.actor.cpp:2621
    apple#6 0x36b3110 in a_body1cont5loopHead1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8664
    apple#7 0x36b07fe in a_body1cont5 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8576
    apple#8 0x36abda8 in a_body1cont4when1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8582
    apple#9 0x36a8dc2 in a_body1cont4 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8454
    apple#10 0x36a4bf6 in a_body1cont3break1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8489
    apple#11 0x36a2c01 in a_body1cont3loopBody1cont1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8505
    apple#12 0x369fd36 in a_body1cont3loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8513
    apple#13 0x3700dcb in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8528
    apple#14 0x36e5210 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#15 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    apple#16 0x2484b07 in a_body1loopBody1cont1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1526
    apple#17 0x24822cf in a_body1loopBody1cont2 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1535
    apple#18 0x247e228 in a_body1loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1541
    apple#19 0x249be87 in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1556
    apple#20 0x249668f in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#21 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    apple#22 0x80557e in a_body1when1 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11591
    apple#23 0x8916ef in a_callback_fire /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11620
    apple#24 0x8735f5 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#25 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    apple#26 0x24820f8 in a_body1cont1loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:860
    apple#27 0x249c852 in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:886
    apple#28 0x249786c in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#29 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    apple#30 0x248b39f in a_body1cont2 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11858
    apple#31 0x2489d02 in a_body1when1 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11865
    apple#32 0x249a150 in a_callback_fire /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11880
    apple#33 0x2492a4f in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#34 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    apple#35 0x248df9b in a_body1cont2 /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2762
    apple#36 0x248b7da in a_body1when1 /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2769
    apple#37 0x2499c88 in a_callback_fire /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2784
    apple#38 0x2492371 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#39 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    apple#40 0xc60fb3 in void SAV<TLogPeekReply>::sendAndDelPromiseRef<TLogPeekReply&>(TLogPeekReply&) /Users/anoyes/workspace/foundationdb/flow/flow.h:472
    apple#41 0xc1137a in NetSAV<TLogPeekReply>::receive(ArenaObjectReader&) /Users/anoyes/workspace/foundationdb/fdbrpc/fdbrpc.h:111
    apple#42 0x78eda75 in a_body1cont1 /Users/anoyes/workspace/foundationdb/fdbrpc/FlowTransport.actor.cpp:652
    apple#43 0x78f7967 in a_body1cont2 /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2369
    apple#44 0x78ed4d8 in a_body1when1 /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2375
    apple#45 0x791af45 in a_callback_fire /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2390
    apple#46 0x7914670 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#47 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /Users/anoyes/workspace/foundationdb/flow/flow.h:447
    apple#48 0x959891 in void Promise<Void>::send<Void>(Void&&) const /Users/anoyes/workspace/foundationdb/flow/flow.h:778
    apple#49 0x7b4b022 in Sim2::execTask(Sim2::Task&) (/Users/anoyes/build/foundationdb/bin/fdbserver+0x7b4b022)
    apple#50 0x7bf9172 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1cont1(Void const&, int) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:979
    apple#51 0x7be7b72 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1when1(Void const&, int) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5391
    apple#52 0x7c32a09 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_callback_fire(ActorCallback<Sim2::RunLoopActor, 0, Void>*, Void) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5406
    apple#53 0x7c1fc7d in ActorCallback<Sim2::RunLoopActor, 0, Void>::fire(Void const&) /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    apple#54 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /Users/anoyes/workspace/foundationdb/flow/flow.h:447
    apple#55 0x959891 in void Promise<Void>::send<Void>(Void&&) const /Users/anoyes/workspace/foundationdb/flow/flow.h:778
    apple#56 0x7fe74ae in N2::PromiseTask::operator()() /Users/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:481
    apple#57 0x7fb7001 in N2::Net2::run() /Users/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:657
    apple#58 0x7b71bdd in Sim2::_runActorState<Sim2::_runActor>::a_body1(int) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:989
    apple#59 0x7b2ee5b in Sim2::_runActor::_runActor(Sim2* const&) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5608
    apple#60 0x7b2f272 in Sim2::_run(Sim2* const&) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:987
    apple#61 0x7b2f2d2 in Sim2::run() /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:996
    apple#62 0x2104064 in main /Users/anoyes/workspace/foundationdb/fdbserver/fdbserver.actor.cpp:1793
    apple#63 0x7fb7c6561504 in __libc_start_main (/lib64/libc.so.6+0x22504)
    apple#64 0x464914  (/Users/anoyes/build/foundationdb/bin/fdbserver+0x464914)
atn34 pushed a commit to atn34/foundationdb that referenced this pull request Dec 13, 2019
@sfc-gh-anoyes
Merge pull request apple#63 from apple/release-6.2
da618c4 
Merge release-6.2
1inker referenced this pull request in owtech/foundationdb Feb 22, 2022
@1inker
Fixed testcases from 'restarting' group
3702139 
Test  #63: restarting/from_7.0.0/ConfigureTestRestart-1
Test  #64: restarting/from_5.0.0/CycleTestRestart-1
Test  #65: restarting/from_5.0.0/StorefrontTestRestart-1
Test  #66: restarting/from_6.2.33/SnapTestAttrition-1
Test  #67: restarting/from_6.2.33/SnapTestSimpleRestart-1
Test  #68: restarting/from_6.2.33/SnapTestRestart-1
Test  #69: restarting/from_6.2.33/SnapCycleRestart-1
Test  #70: restarting/from_5.1.7/DrUpgradeRestart-1
Test  #71: restarting/from_5.2.0/ClientTransactionProfilingCorrectness-1
Test  #72: restarting/to_6.2.33/CycleTestRestart-1
sfc-gh-ljoswiak pushed a commit to sfc-gh-ljoswiak/foundationdb that referenced this pull request Dec 17, 2022
@sfc-gh-vgasiunas
Merge pull request apple#63 from sfc-gh-mpilman/vgasiunas/flow-serial…
355493d 
…izer

Fix flow buffer test generator to support all types
vishesh pushed a commit to vishesh/foundationdb that referenced this pull request Apr 6, 2023
@hyp
Merge pull request apple#63 from FoundationDB/eng/gen-cdb
33165d1 
[interop] generate swift compile commands
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexmiller-apple alexmiller-apple alexmiller-apple requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@etschannen @alexmiller-apple