Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for MCLAG #453

Merged
merged 7 commits into from
Feb 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions config/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,14 @@ def _abort_if_false(ctx, param, value):
if not value:
ctx.abort()

def _get_optional_services():
config_db = ConfigDBConnector()
config_db.connect()
optional_services_dict = config_db.get_table('FEATURE')
if not optional_services_dict:
return None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shine4chen please fix a bug:

TASK [execute cli "config load_minigraph -y" to apply new minigraph] ***********
Tuesday 25 February 2020  19:54:51 +0000 (0:00:00.067)       0:00:18.526 ****** 

fatal: [vlab-01]: FAILED! => {"changed": true, "cmd": "config load_minigraph -y", "delta": "0:00:55.103499", "end": "2020-02-25 19:55:46.147827", "msg": "non-zero return code", "rc": 1, "start": "2020-02-25 19:54:51.044328", "stderr": "Traceback (most recent call last):\n  File \"/usr/bin/config\", line 12, in <module>\n    sys.exit(config())\n  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 722, in __call__\n    return self.main(*args, **kwargs)\n  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 697, in main\n    rv = self.invoke(ctx)\n  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 1066, in invoke\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\n  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 895, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 535, in invoke\n    return callback(*args, **kwargs)\n  File \"/usr/lib/python2.7/dist-packages/config/main.py\", line 684, in load_minigraph\n    _stop_services()\n  File \"/usr/lib/python2.7/dist-packages/config/main.py\", line 451, in _stop_services\n    for service in _get_optional_services():\nTypeError: 'NoneType' object is not iterable", "stderr_lines": ["Traceback (most recent call last):", "  File \"/usr/bin/config\", line 12, in <module>", "    sys.exit(config())", "  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 722, in __call__", "    return self.main(*args, **kwargs)", "  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 697, in main", "    rv = self.invoke(ctx)", "  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 1066, in invoke", "    return _process_result(sub_ctx.command.invoke(sub_ctx))", "  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 895, in invoke", "    return ctx.invoke(self.callback, **ctx.params)", "  File \"/usr/lib/python2.7/dist-packages/click/core.py\", line 535, in invoke", "    return callback(*args, **kwargs)", "  File \"/usr/lib/python2.7/dist-packages/config/main.py\", line 684, in load_minigraph", "    _stop_services()", "  File \"/usr/lib/python2.7/dist-packages/config/main.py\", line 451, in _stop_services", "    for service in _get_optional_services():", "TypeError: 'NoneType' object is not iterable"], "stdout": "Stopping service swss ...\nStopping service lldp ...\nStopping service pmon ...\nStopping service bgp ...\nStopping service hostcfgd ...\nStopping service nat ...", "stdout_lines": ["Stopping service swss ...", "Stopping service lldp ...", "Stopping service pmon ...", "Stopping service bgp ...", "Stopping service hostcfgd ...", "Stopping service nat ..."]}

@qiluo-msft / @yxieca FYI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems github website has some error that we can't open PRs now. I will fix it after github service recover.

return optional_services_dict.keys()
Copy link
Contributor

@jleveque jleveque Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this function could return the entire dictionary, then the consumers below can determine whether or not to stop/start the service based on whether it is enabled or disabled in the DB, in conjunction with the result of running systemctl is-enabled. Theoretically, the two states should match, but I'm trying to consider the corner case just in case they are out-of-sync.

Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If out of sync I prefer the result from systemctl is-enable. In general this out of sync should be caused by user behavior such as directly running systemctl enable xxx. This is the reason I choose systemctl is-enabled instead of config-db.
Anyway I am open for this. Giving priority to config-db is acceptable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jleveque any comment? If no could you approve this PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with this approach. I will approve the PR, but I would also like to get @lguohan's approval before merging.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks


def _stop_services():
services_to_stop = [
'swss',
Expand All @@ -331,6 +339,17 @@ def _stop_services():
log_error("Stopping {} failed with error {}".format(service, e))
raise

# For optional services they don't start by default
for service in _get_optional_services():
(out, err) = run_command("systemctl status {}".format(service), return_output = True)
if not err and 'Active: active (running)' in out:
try:
click.echo("Stopping service {} ...".format(service))
run_command("systemctl stop {}".format(service))
except SystemExit as e:
log_error("Stopping {} failed with error {}".format(service, e))
raise

def _reset_failed_services():
services_to_reset = [
'bgp',
Expand All @@ -357,6 +376,17 @@ def _reset_failed_services():
log_error("Failed to reset failed status for service {}".format(service))
raise

# For optional services they don't start by default
for service in _get_optional_services():
(out, err) = run_command("systemctl is-enabled {}".format(service), return_output = True)
if not err and 'enabled' in out:
try:
click.echo("Resetting failed status for service {} ...".format(service))
run_command("systemctl reset-failed {}".format(service))
except SystemExit as e:
log_error("Failed to reset failed status for service {}".format(service))
raise

def _restart_services():
services_to_restart = [
'hostname-config',
Expand All @@ -378,6 +408,17 @@ def _restart_services():
log_error("Restart {} failed with error {}".format(service, e))
raise

# For optional services they don't start by default
for service in _get_optional_services():
(out, err) = run_command("systemctl is-enabled {}".format(service), return_output = True)
if not err and 'enabled' in out:
try:
click.echo("Restarting service {} ...".format(service))
run_command("systemctl restart {}".format(service))
except SystemExit as e:
log_error("Restart {} failed with error {}".format(service, e))
raise

def is_ipaddress(val):
""" Validate if an entry is a valid IP """
if not val:
Expand Down
18 changes: 18 additions & 0 deletions scripts/fast-reboot
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,24 @@ debug "Stopped bgp ..."
docker kill lldp > /dev/null
systemctl stop lldp

if [[ "$REBOOT_TYPE" = "fast-reboot" ]]; then
if echo $(docker ps) | grep -q iccpd; then
docker kill iccpd > /dev/null || [ $? == 1 ]
fi
fi

# Stop iccpd gracefully
if [[ "$REBOOT_TYPE" = "warm-reboot" || "$REBOOT_TYPE" = "fastfast-reboot" ]]; then
if echo $(docker ps) | grep -q iccpd; then
debug "Stopping iccpd ..."
# Send USR1 signal to iccpd to stop it
# It will prepare iccpd for warm-reboot
# Note: We must send USR1 signal before syncd, or some state of iccpd maybe lost
docker exec -i iccpd pkill -USR1 iccpd || [ $? == 1 ] > /dev/null
debug "Stopped iccpd ..."
fi
fi

if [[ "$REBOOT_TYPE" = "fast-reboot" ]]; then
# Kill teamd processes inside of teamd container with SIGUSR2 to allow them to send last LACP frames
# We call `docker kill teamd` to ensure the container stops as quickly as possible,
Expand Down