Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SONiC Generic Update and Rollback - HLD #736

Merged

Conversation

ghooo
Copy link
Contributor

@ghooo ghooo commented Jan 19, 2021

Repo PR Title State
sonic-utilities [config][generic-update] Adding apply-patch, rollback, checkpoints commands GitHub issue/pull request detail
sonic-buildimage [sonic-yang] Backlinks are none when there is 0 backlinks GitHub issue/pull request detail
sonic-utilities [config][generic-update] Implementing patch sorting GitHub issue/pull request detail
sonic-buildimage [yang-models] Removing sonic-acl import from sonic-nat.yang GitHub issue/pull request detail
sonic-utilities [config][generic-update] Implementing patch sorting (2nd attempt) GitHub issue/pull request detail
sonic-utilities [config][generic-update] Fix PatchApplier init order GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Cropping current/target config before sorting GitHub issue/pull request detail
sonic-utilities [generic_config_updater] Logging GitHub issue/pull request detail
sonic-utilities Generic_upater: Apply JSON change GitHub issue/pull request detail
sonic-utilities generic_updater: Fix service validator related issues GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Adding logging to config-{replace, rollback, checkpoint, list-checkpoints} GitHub issue/pull request detail
sonic-mgmt draft for dhcp server testcase GitHub issue/pull request detail
sonic-mgmt Skip DUTs that don't support apply-patch feature GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Handling empty tables while sorting a patch GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Logging change just before applying it GitHub issue/pull request detail
sonic-utilities generic_config_updater: Filename changed & VLAN validator added GitHub issue/pull request detail
sonic-mgmt skip_release DUTs that don't support apply-patch feature GitHub issue/pull request detail
sonic-mgmt Generic Config Updater cacl test GitHub issue/pull request detail
sonic-mgmt Generic Config Updater Syslog server test GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Adding non-strict mode GitHub issue/pull request detail
sonic-buildimage [yang-models] Validating 'services' exist if ACL type is 'CTRLPLANE' GitHub issue/pull request detail
sonic-buildimage [yang-model] Using 'leafref' instead of 'must' for loopback GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Improving CreateOnly validator and marking /LOOPBACK_INTERFACE/LOOPBACK#/vrf_name as create-only GitHub issue/pull request detail
sonic-utilities [GCU] Implementing DryRun by printing patch-sorter steps/imitating config_db GitHub issue/pull request detail
sonic-utilities [GCU] Moving PatchSorter unit-test to json file to make it easier to read/maintain GitHub issue/pull request detail
sonic-mgmt GCU loopback interface test GitHub issue/pull request detail
sonic-utilities [GCU] Loading yang-models only once GitHub issue/pull request detail
sonic-mgmt Make GCU syslog's test independent from previous test GitHub issue/pull request detail
sonic-mgmt GCU vlan interface test GitHub issue/pull request detail
sonic-mgmt GCU cacl test combination GitHub issue/pull request detail
sonic-utilities [GCU] Different apply-patch runs should produce same sorted steps GitHub issue/pull request detail
sonic-utilities [GCU] Copying config_db before calling sonic_yang.loadData GitHub issue/pull request detail
sonic-utilities [GCU] Using simulated config instead of target config when validating replace operation in NoDependencyMoveValidator GitHub issue/pull request detail
sonic-mgmt GCU portchannel interface test GitHub issue/pull request detail
sonic-buildimage [sonic-yang-mgmt] Adding flag to disable/enable log printing GitHub issue/pull request detail
sonic-utilities [GCU] Show default option for '--format' GitHub issue/pull request detail
sonic-utilities [GCU] Disallowing DeleteInsteadOfReplaceMoveExtender from generating delete whole config move GitHub issue/pull request detail
sonic-utilities [build-failure-fix][GCU] Sort referrer paths alphabetically directly on 202111 GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Handle failed service restarts GitHub issue/pull request detail
sonic-utilities [generic_config_updater] Minor update - No logical code change GitHub issue/pull request detail
sonic-mgmt Add previous GCU tests to kvm daily test GitHub issue/pull request detail
sonic-utilities [GCU] Mark children of bgp_neighbor as create-only GitHub issue/pull request detail
sonic-mgmt GCU AAA test GitHub issue/pull request detail
sonic-mgmt GCU test for BGPListener(BGPMonitor) GitHub issue/pull request detail
sonic-mgmt Remove test reset check for GCU test GitHub issue/pull request detail
sonic-mgmt Add incremental QOS config tests GitHub issue/pull request detail
sonic-mgmt Skip GCU test on dut version that older than 202112 GitHub issue/pull request detail
sonic-buildimage [yang] Adding sonic-bgp-monitor to setup.py GitHub issue/pull request detail
sonic-buildimage [yang] Fixing groupings when grouping is in the same module file GitHub issue/pull request detail
sonic-mgmt Revert GCU cacl test due to random failure and add sleep in cacl for local test GitHub issue/pull request detail
sonic-mgmt Added support of barefoot asic in incremental_qos test case GitHub issue/pull request detail
sonic-mgmt Add pfcwd interval config tests GitHub issue/pull request detail
sonic-mgmt [tests/generic_config_updater] Replace IP testcase for BGP Listener GitHub issue/pull request detail
sonic-mgmt Incremental QOS config updater test update GitHub issue/pull request detail
sonic-mgmt [generic-config-updater][Add/Remove rack] - Updated test to include add/remove rack via patch-apply GitHub issue/pull request detail
sonic-utilities [GCU] Turning port admin down before some critical port changes GitHub issue/pull request detail
sonic-utilities [GCU] Supporting Groupings during path-xpath translation GitHub issue/pull request detail
sonic-mgmt [GCU] Add config ipv6 BGP neighbor test GitHub issue/pull request detail
sonic-mgmt AddRack: Skip multi asic & remove hack GitHub issue/pull request detail
sonic-utilities [GCU] Marking fields under BGP_PEER_RANGE, BGP_MONITORS as create-only GitHub issue/pull request detail
sonic-mgmt GCU test for bgp speaker GitHub issue/pull request detail
sonic-mgmt GCU test for BGP prefix GitHub issue/pull request detail
sonic-mgmt [GCU] Add ethernet interface tests GitHub issue/pull request detail
sonic-utilities [GCU] Marking fields under MIRROR_SESSION as create-only GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Add caclrule validator GitHub issue/pull request detail
sonic-mgmt GCU test for Monitor Config GitHub issue/pull request detail
sonic-buildimage [yang] In ACL_RULE PRIORITY is mandatory and also PACKET_ACTION for CTRLPLANE ACLs GitHub issue/pull request detail
sonic-mgmt Add loganalyzer ignore regex for GCU GitHub issue/pull request detail
sonic-utilities [GCU] Optimizing moves by adding generators for keys/tables GitHub issue/pull request detail
sonic-mgmt [GCU] add ipv6, ethernet interface to kvm test suite GitHub issue/pull request detail
sonic-mgmt [tests/generic_config_updater]skip test on release 202111 GitHub issue/pull request detail
sonic-mgmt [generic_config_updater]Add empty input fixture to verify configDB before GCU tests GitHub issue/pull request detail
sonic-mgmt [generic_config_updater] add GCU ignore regex GitHub issue/pull request detail
sonic-utilities [GCU] Supressing YANG errors from libyang while sorting GitHub issue/pull request detail
sonic-buildimage [SY] Adding exceptlionList to validation exception GitHub issue/pull request detail
sonic-utilities [GCU] Validate peer_group_range ip_range are correct GitHub issue/pull request detail
sonic-mgmt Fix test case: generic_config_updater/test_ipv6.py::test_ipv6_neighbor_admin_change GitHub issue/pull request detail
sonic-utilities [GCU] Handling non-compliant leaf-list with string values GitHub issue/pull request detail
sonic-mgmt Fix test case: generic_config_updater/test_eth_interface.py::test_replace_mtu GitHub issue/pull request detail
sonic-utilities [GCU] Handling type1 lists GitHub issue/pull request detail
sonic-mgmt [configlet] Add test_add_rack to KVM daily test GitHub issue/pull request detail
sonic-mgmt [tests/generic_config_updater] improve GCU portchannel test GitHub issue/pull request detail
sonic-utilities [generic-config-updater] Add NTP validator GitHub issue/pull request detail
sonic-mgmt [GCU] Shorten GCU ignored LA list GitHub issue/pull request detail
sonic-mgmt [GCU] Add rollback validation test GitHub issue/pull request detail
sonic-mgmt Avoid long run test in GCU. GitHub issue/pull request detail
sonic-mgmt Bypass replace_lane to unblock submodule update GitHub issue/pull request detail
sonic-mgmt Fix GCU update speed case to use supported speed get from the dut GitHub issue/pull request detail
sonic-utilities [GCU] Moving UniqueLanes from only validating moves, to be a supplemental YANG validator GitHub issue/pull request detail

@ghooo ghooo marked this pull request as ready for review February 10, 2021 15:03
@ghooo ghooo changed the title SONiC Generic Update and Rollback - Design SONiC Generic Update and Rollback - HLD Feb 11, 2021
Copy link
Member

@praveen-li praveen-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will complete entire review today/

Copy link
Member

@praveen-li praveen-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me, let me know if there is a section for lower-level design and the section which describes the unit test in more detail.

ghooo added a commit to sonic-net/sonic-utilities that referenced this pull request Apr 26, 2021
…mmands (#1536)

#### What I did
Adding apply-patch, rollback, replace, checkpoint, delete-checkpoint, list-checkpoints functionality.

#### How I did it
This PR is implementing the first step in in README.md in the design document: sonic-net/SONiC#736  

#### How to verify it
Using unit-tests

#### Previous command output (if the output of a command-line utility has changed)

#### New command output (if the output of a command-line utility has changed)

```sh
admin@sonic:~$ sudo config apply-patch --help
Usage: config apply-patch [OPTIONS] PATCH_FILE_PATH

  Apply given patch of updates to Config. A patch is a JsonPatch which
  follows rfc6902. This command can be used do partial updates to the config
  with minimum disruption to running processes. It allows addition as well
  as deletion of configs. The patch file represents a diff of ConfigDb(ABNF)
  format or SonicYang format.

  <patch-file-path>: Path to the patch file on the file-system.

Options:
  -f, --format [CONFIGDB|SONICYANG]
                                  format of config of the patch is either
                                  ConfigDb(ABNF) or SonicYang
  -d, --dry-run                   test out the command without affecting
                                  config state
  -v, --verbose                   print additional details of what the
                                  operation is doing
  -h, -?, --help                  Show this message and exit.



admin@sonic:~$ sudo config replace --help
Usage: config replace [OPTIONS] TARGET_FILE_PATH

  Replace the whole config with the specified config. The config is replaced
  with minimum disruption e.g. if ACL config is different between current
  and target config only ACL config is updated, and other config/services
  such as DHCP will not be affected. **WARNING** The target config file
  should be the whole config, not just the part intended to be updated.

  <target-file-path>: Path to the target file on the file-system.

Options:
  -f, --format [CONFIGDB|SONICYANG]
                                  format of target config is either
                                  ConfigDb(ABNF) or SonicYang
  -d, --dry-run                   test out the command without affecting
                                  config state
  -v, --verbose                   print additional details of what the
                                  operation is doing
  -h, -?, --help                  Show this message and exit.



admin@sonic:~$ sudo config rollback --help
Usage: config rollback [OPTIONS] CHECKPOINT_NAME

  Rollback the whole config to the specified checkpoint. The config is
  rolled back with minimum disruption e.g. if ACL config is different
  between current and checkpoint config only ACL config is updated, and
  other config/services such as DHCP will not be affected.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -d, --dry-run   test out the command without affecting config state
  -v, --verbose   print additional details of what the operation is doing
  -?, -h, --help  Show this message and exit.



admin@sonic:~$ sudo config checkpoint --help
Usage: config checkpoint [OPTIONS] CHECKPOINT_NAME

  Take a checkpoint of the whole current config with the specified
  checkpoint name.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -h, -?, --help  Show this message and exit.



admin@sonic:~$ sudo config delete-checkpoint --help
Usage: config delete-checkpoint [OPTIONS] CHECKPOINT_NAME

  Delete a checkpoint with the specified checkpoint name.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -h, -?, --help  Show this message and exit.



admin@sonic:~$ sudo config list-checkpoints --help
Usage: config list-checkpoints [OPTIONS]

  List the config checkpoints available.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -?, -h, --help  Show this message and exit.
```
Copy link
Member

@praveen-li praveen-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me, let me know if any pending questions, where my involvement is needed. Thx.

@qiluo-msft qiluo-msft merged commit 9b37027 into sonic-net:master May 3, 2021
gitsabari pushed a commit to gitsabari/sonic-utilities that referenced this pull request Jun 15, 2021
…mmands (sonic-net#1536)

#### What I did
Adding apply-patch, rollback, replace, checkpoint, delete-checkpoint, list-checkpoints functionality.

#### How I did it
This PR is implementing the first step in in README.md in the design document: sonic-net/SONiC#736  

#### How to verify it
Using unit-tests

#### Previous command output (if the output of a command-line utility has changed)

#### New command output (if the output of a command-line utility has changed)

```sh
admin@sonic:~$ sudo config apply-patch --help
Usage: config apply-patch [OPTIONS] PATCH_FILE_PATH

  Apply given patch of updates to Config. A patch is a JsonPatch which
  follows rfc6902. This command can be used do partial updates to the config
  with minimum disruption to running processes. It allows addition as well
  as deletion of configs. The patch file represents a diff of ConfigDb(ABNF)
  format or SonicYang format.

  <patch-file-path>: Path to the patch file on the file-system.

Options:
  -f, --format [CONFIGDB|SONICYANG]
                                  format of config of the patch is either
                                  ConfigDb(ABNF) or SonicYang
  -d, --dry-run                   test out the command without affecting
                                  config state
  -v, --verbose                   print additional details of what the
                                  operation is doing
  -h, -?, --help                  Show this message and exit.



admin@sonic:~$ sudo config replace --help
Usage: config replace [OPTIONS] TARGET_FILE_PATH

  Replace the whole config with the specified config. The config is replaced
  with minimum disruption e.g. if ACL config is different between current
  and target config only ACL config is updated, and other config/services
  such as DHCP will not be affected. **WARNING** The target config file
  should be the whole config, not just the part intended to be updated.

  <target-file-path>: Path to the target file on the file-system.

Options:
  -f, --format [CONFIGDB|SONICYANG]
                                  format of target config is either
                                  ConfigDb(ABNF) or SonicYang
  -d, --dry-run                   test out the command without affecting
                                  config state
  -v, --verbose                   print additional details of what the
                                  operation is doing
  -h, -?, --help                  Show this message and exit.



admin@sonic:~$ sudo config rollback --help
Usage: config rollback [OPTIONS] CHECKPOINT_NAME

  Rollback the whole config to the specified checkpoint. The config is
  rolled back with minimum disruption e.g. if ACL config is different
  between current and checkpoint config only ACL config is updated, and
  other config/services such as DHCP will not be affected.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -d, --dry-run   test out the command without affecting config state
  -v, --verbose   print additional details of what the operation is doing
  -?, -h, --help  Show this message and exit.



admin@sonic:~$ sudo config checkpoint --help
Usage: config checkpoint [OPTIONS] CHECKPOINT_NAME

  Take a checkpoint of the whole current config with the specified
  checkpoint name.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -h, -?, --help  Show this message and exit.



admin@sonic:~$ sudo config delete-checkpoint --help
Usage: config delete-checkpoint [OPTIONS] CHECKPOINT_NAME

  Delete a checkpoint with the specified checkpoint name.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -h, -?, --help  Show this message and exit.



admin@sonic:~$ sudo config list-checkpoints --help
Usage: config list-checkpoints [OPTIONS]

  List the config checkpoints available.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -?, -h, --help  Show this message and exit.
```
malletvapid23 added a commit to malletvapid23/Sonic-Utility that referenced this pull request Aug 3, 2023
…mmands (#1536)

#### What I did
Adding apply-patch, rollback, replace, checkpoint, delete-checkpoint, list-checkpoints functionality.

#### How I did it
This PR is implementing the first step in in README.md in the design document: sonic-net/SONiC#736  

#### How to verify it
Using unit-tests

#### Previous command output (if the output of a command-line utility has changed)

#### New command output (if the output of a command-line utility has changed)

```sh
admin@sonic:~$ sudo config apply-patch --help
Usage: config apply-patch [OPTIONS] PATCH_FILE_PATH

  Apply given patch of updates to Config. A patch is a JsonPatch which
  follows rfc6902. This command can be used do partial updates to the config
  with minimum disruption to running processes. It allows addition as well
  as deletion of configs. The patch file represents a diff of ConfigDb(ABNF)
  format or SonicYang format.

  <patch-file-path>: Path to the patch file on the file-system.

Options:
  -f, --format [CONFIGDB|SONICYANG]
                                  format of config of the patch is either
                                  ConfigDb(ABNF) or SonicYang
  -d, --dry-run                   test out the command without affecting
                                  config state
  -v, --verbose                   print additional details of what the
                                  operation is doing
  -h, -?, --help                  Show this message and exit.



admin@sonic:~$ sudo config replace --help
Usage: config replace [OPTIONS] TARGET_FILE_PATH

  Replace the whole config with the specified config. The config is replaced
  with minimum disruption e.g. if ACL config is different between current
  and target config only ACL config is updated, and other config/services
  such as DHCP will not be affected. **WARNING** The target config file
  should be the whole config, not just the part intended to be updated.

  <target-file-path>: Path to the target file on the file-system.

Options:
  -f, --format [CONFIGDB|SONICYANG]
                                  format of target config is either
                                  ConfigDb(ABNF) or SonicYang
  -d, --dry-run                   test out the command without affecting
                                  config state
  -v, --verbose                   print additional details of what the
                                  operation is doing
  -h, -?, --help                  Show this message and exit.



admin@sonic:~$ sudo config rollback --help
Usage: config rollback [OPTIONS] CHECKPOINT_NAME

  Rollback the whole config to the specified checkpoint. The config is
  rolled back with minimum disruption e.g. if ACL config is different
  between current and checkpoint config only ACL config is updated, and
  other config/services such as DHCP will not be affected.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -d, --dry-run   test out the command without affecting config state
  -v, --verbose   print additional details of what the operation is doing
  -?, -h, --help  Show this message and exit.



admin@sonic:~$ sudo config checkpoint --help
Usage: config checkpoint [OPTIONS] CHECKPOINT_NAME

  Take a checkpoint of the whole current config with the specified
  checkpoint name.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -h, -?, --help  Show this message and exit.



admin@sonic:~$ sudo config delete-checkpoint --help
Usage: config delete-checkpoint [OPTIONS] CHECKPOINT_NAME

  Delete a checkpoint with the specified checkpoint name.

  <checkpoint-name>: The checkpoint name, use `config list-checkpoints`
  command to see available checkpoints.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -h, -?, --help  Show this message and exit.



admin@sonic:~$ sudo config list-checkpoints --help
Usage: config list-checkpoints [OPTIONS]

  List the config checkpoints available.

Options:
  -v, --verbose   print additional details of what the operation is doing
  -?, -h, --help  Show this message and exit.
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants