Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have frontend proxy persist events when using redis #870

Merged
merged 11 commits into from
Jun 2, 2020
+155 −109
Merged

Have frontend proxy persist events when using redis #870

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
b6e2ccc
Have frontend proxy persit events when using redis
erikjohnston May 15, 2020
97e9a2d
Use renamed variable
erikjohnston May 18, 2020
ed33ee2
Merge branch 'develop' of github.com:matrix-org/sytest into erikj/mov…
erikjohnston May 18, 2020
705d037
Fiddle some timeouts
erikjohnston May 18, 2020
1c0844d
Remove password
erikjohnston May 19, 2020
80c993d
Add some retry_until_success to fix some races
erikjohnston May 21, 2020
67916d5
Always sync power level changes
erikjohnston May 21, 2020
b249839
More flakey test fixes
erikjohnston May 22, 2020
32d06f9
Flakey test
erikjohnston May 22, 2020
70f6ca5
Flakey test
erikjohnston May 22, 2020
95045ff
More flakey test fixes
erikjohnston May 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion lib/SyTest/Homeserver/Synapse.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,17 @@ sub start
},
) : (),

instance_map => {
"frontend_proxy1" => {
host => "$bind_host",
port => $self->{ports}{frontend_proxy},
},
},

stream_writers => {
events => $self->{redis_host} ne '' ? "frontend_proxy1" : "master",
},

# We use a high limit so the limit is never reached, but enabling the
# limit ensures that the code paths get hit. This helps testing the
# feature with worker mode.
Expand Down Expand Up @@ -988,6 +999,7 @@ sub wrap_synapse_command
{
my $frontend_proxy_config_path = $self->write_yaml_file( "frontend_proxy.yaml" => {
"worker_app" => "synapse.app.frontend_proxy",
"worker_name" => "frontend_proxy1",
"worker_pid_file" => "$hsdir/frontend_proxy.pid",
"worker_log_config" => $self->configure_logger("frontend_proxy"),
"worker_replication_host" => "$bind_host",
Expand All @@ -997,7 +1009,7 @@ sub wrap_synapse_command
"worker_listeners" => [
{
type => "http",
resources => [{ names => ["client"] }],
resources => [{ names => ["client", "replication"] }],
port => $self->{ports}{frontend_proxy},
bind_address => $bind_host,
},
Expand Down
4 changes: 3 additions & 1 deletion tests/10apidoc/33room-members.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,8 @@ sub matrix_create_and_join_room
} @local_members,
)
})->then( sub {
log_if_fail "Created room_id=$room_id";

Future->done( $room_id,
( $with_alias ? ( $room_alias_fullname ) : () )
);
Expand Down Expand Up @@ -660,7 +662,7 @@ sub matrix_join_room_synced

matrix_do_and_wait_for_sync( $user,
do => sub {
matrix_join_room( $user, $room_id_or_alias, %params );
retry_until_success { matrix_join_room( $user, $room_id_or_alias, %params ) }
},
check => sub { exists $_[0]->{rooms}{join}{$_[1]} },
);
Expand Down
2 changes: 1 addition & 1 deletion tests/10apidoc/36room-levels.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ sub matrix_change_room_power_levels
my ( $levels ) = @_;
$func->( $levels );

matrix_put_room_state( $user, $room_id, type => "m.room.power_levels",
matrix_put_room_state_synced( $user, $room_id, type => "m.room.power_levels",
content => $levels,
);
});
Expand Down
36 changes: 21 additions & 15 deletions tests/30rooms/05aliases.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -219,15 +219,19 @@
$_[0]->{events}->{'m.room.aliases'} = 50;
},
)->then( sub {
matrix_get_room_state( $creator, $room_id,
type => "m.room.power_levels",
);
})->then( sub {
my ( $body ) = @_;
log_if_fail "power levels", $body;
retry_until_success {
matrix_get_room_state( $creator, $room_id,
type => "m.room.power_levels",
)->then( sub {
my ( $body ) = @_;
log_if_fail "power levels", $body;

assert_eq( $body->{events}->{'m.room.aliases'}, 50 );
assert_eq( $body->{events}->{'m.room.aliases'}, 50 );

Future->done( 1 )
})
}
})->then( sub {
do_request_json_for( $other_user,
method => "PUT",
uri => "/r0/directory/room/$alias",
Expand Down Expand Up @@ -364,7 +368,7 @@ sub _test_can_create_and_delete_alias {
content => { room_id => $room_id },
)
})->then( sub {
matrix_put_room_state( $creator, $room_id,
matrix_put_room_state_synced( $creator, $room_id,
type => "m.room.canonical_alias",
content => { alias => $room_alias }
)
Expand All @@ -376,15 +380,17 @@ sub _test_can_create_and_delete_alias {
content => {},
)
})->then( sub {
matrix_get_room_state( $creator, $room_id,
type => "m.room.canonical_alias",
)
})->then( sub {
my ( $body ) = @_;
retry_until_success {
matrix_get_room_state( $creator, $room_id,
type => "m.room.canonical_alias",
)->then( sub {
my ( $body ) = @_;

not defined $body->{alias} or die "Expected canonical alias to be empty";
not defined $body->{alias} or die "Expected canonical alias to be empty";

Future->done( 1 );
Future->done( 1 );
})
}
})
};

Expand Down
2 changes: 1 addition & 1 deletion tests/30rooms/06invite.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ sub invited_user_can_reject_invite

matrix_invite_user_to_room( $creator, $invitee, $room_id )
->then( sub {
matrix_leave_room( $invitee, $room_id )
matrix_leave_room_synced( $invitee, $room_id )
})->then( sub {
matrix_get_room_state( $creator, $room_id,
type => "m.room.member",
Expand Down
40 changes: 24 additions & 16 deletions tests/30rooms/07ban.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,19 @@

content => { user_id => $banned_user->user_id, reason => "testing" },
)->then( sub {
matrix_get_room_state( $creator, $room_id,
type => "m.room.member",
state_key => $banned_user->user_id,
)
retry_until_success {
matrix_get_room_state( $creator, $room_id,
type => "m.room.member",
state_key => $banned_user->user_id,
)->then( sub {
my ( $body ) = @_;
$body->{membership} eq "ban" or
die "Expected banned user membership to be 'ban'";

Future->done( 1 )
})
}
})->then( sub {
my ( $body ) = @_;
$body->{membership} eq "ban" or
die "Expected banned user membership to be 'ban'";

matrix_join_room( $banned_user, $room_id )
->main::expect_http_403; # Must be unbanned first
})->then( sub {
Expand Down Expand Up @@ -75,15 +79,19 @@

content => { user_id => $banned_user->user_id, reason => "testing" },
)->then( sub {
matrix_get_room_state( $creator, $room_id,
type => "m.room.member",
state_key => $banned_user->user_id,
)
retry_until_success {
matrix_get_room_state( $creator, $room_id,
type => "m.room.member",
state_key => $banned_user->user_id,
)->then( sub {
my ( $body ) = @_;
$body->{membership} eq "ban" or
die "Expected banned user membership to be 'ban'";

Future->done( 1 )
})
}
})->then( sub {
my ( $body ) = @_;
$body->{membership} eq "ban" or
die "Expected banned user membership to be 'ban'";

repeat_until_true {
matrix_get_room_state( $banned_user, $room_id,
type => "m.room.member",
Expand Down
54 changes: 32 additions & 22 deletions tests/30rooms/12thirdpartyinvite.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,16 +78,18 @@
},
);
})->then( sub {
matrix_get_room_state( $inviter, $room_id,
type => "m.room.member",
state_key => $invitee_mxid,
);
})->on_done( sub {
my ( $body ) = @_;

log_if_fail "Body", $body;
assert_eq( $body->{membership}, "invite",
'invited user membership' );
retry_until_success {
matrix_get_room_state( $inviter, $room_id,
type => "m.room.member",
state_key => $invitee_mxid,
)->on_done( sub {
my ( $body ) = @_;

log_if_fail "Body", $body;
assert_eq( $body->{membership}, "invite",
'invited user membership' );
})
}
});
};

Expand Down Expand Up @@ -236,13 +238,17 @@ sub can_invite_unbound_3pid
log_if_fail "m.room.member invite", $body;
assert_eq( $body->{third_party_invite}{display_name}, 'Bob', 'invite display name' );

matrix_join_room( $invitee, $room_id )
retry_until_success {
matrix_join_room( $invitee, $room_id )
}
})->then( sub {
matrix_get_room_state( $inviter, $room_id,
type => "m.room.member",
state_key => $invitee->user_id,
)
})->followed_by( assert_membership( "join" ) );
retry_until_success {
matrix_get_room_state( $inviter, $room_id,
type => "m.room.member",
state_key => $invitee->user_id,
)->followed_by( assert_membership( "join" ) )
}
})
}

test "Can invite unbound 3pid over federation with users from both servers",
Expand Down Expand Up @@ -289,7 +295,9 @@ sub can_invite_unbound_3pid
log_if_fail "m.room.member invite", $body;
assert_eq( $body->{third_party_invite}{display_name}, 'Bob', 'invite display name' );

matrix_join_room( $invitee, $room_id )
retry_until_success {
matrix_join_room( $invitee, $room_id )
}
})->then( sub {
await_event_for( $inviter, filter => sub {
my ( $event ) = @_;
Expand All @@ -301,11 +309,13 @@ sub can_invite_unbound_3pid
return 1;
})
})->then( sub {
matrix_get_room_state( $inviter, $room_id,
type => "m.room.member",
state_key => $invitee->user_id,
)
})->followed_by( assert_membership( "join" ) );
retry_until_success {
matrix_get_room_state( $inviter, $room_id,
type => "m.room.member",
state_key => $invitee->user_id,
)->followed_by( assert_membership( "join" ) )
}
});
};

test "Can accept unbound 3pid invite after inviter leaves",
Expand Down
28 changes: 16 additions & 12 deletions tests/30rooms/60version_upgrade.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -674,7 +674,7 @@ sub upgrade_room_synced {
);
})->then( sub {
# alias 1 is the canonical alias.
matrix_put_room_state( $creator, $room_id,
matrix_put_room_state_synced( $creator, $room_id,
type => "m.room.canonical_alias",
content => {
alias => $room_alias_1,
Expand Down Expand Up @@ -757,7 +757,7 @@ sub upgrade_room_synced {
$creator, $remote_user, $room_id
)->then( sub {
# Have the remote user join the room
matrix_join_room( $remote_user, $room_id );
matrix_join_room_synced( $remote_user, $room_id );
})->then( sub {
# Have the remote user add an alias
do_request_json_for(
Expand All @@ -781,20 +781,24 @@ sub upgrade_room_synced {
);
})->then( sub {
# Have the remote user join the upgraded room
matrix_join_room( $remote_user, $new_room_id );
matrix_join_room_synced( $remote_user, $new_room_id );
})->then( sub {
# Check that the remote alias points to the new room id
do_request_json_for(
$remote_user,
method => "GET",
uri => "/r0/directory/room/$remote_room_alias",
);
})->then( sub {
my ( $body ) = @_;
retry_until_success {
do_request_json_for(
$remote_user,
method => "GET",
uri => "/r0/directory/room/$remote_room_alias",
)->then( sub {
my ( $body ) = @_;

assert_eq( $body->{room_id}, $new_room_id, "room_id for remote alias" );
log_if_fail "Got room ID for alias", $body->{room_id};

Future->done(1);
assert_eq( $body->{room_id}, $new_room_id, "room_id for remote alias" );

Future->done(1);
})
}
});
};

Expand Down
4 changes: 2 additions & 2 deletions tests/31sync/06state.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -774,7 +774,7 @@
qw( can_sync ) ],

# sending 50 messages can take a while
timeout => 20000,
timeout => 20,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this made tests go faster or was this just an unnecessarily large number to begin with?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I forgot about these. I think there was a confusion for the units here, they're in seconds not milliseconds (a timeout of 6 hours seems excessive).


check => sub {
my ( $creator, $syncer, $invitee ) = @_;
Expand Down Expand Up @@ -834,7 +834,7 @@
qw( can_sync ) ],

# sending 50 messages can take a while
timeout => 20000,
timeout => 20,

check => sub {
my ( $creator, $syncer, $invitee ) = @_;
Expand Down
2 changes: 1 addition & 1 deletion tests/31sync/15lazy-members.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@

test "Gapped incremental syncs include all state changes",
# sending 50 messages can take a while
timeout => 20000,
timeout => 20,

requires => [ local_user_fixtures( 4 ),
qw( can_sync ) ],
Expand Down
Loading