Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iotedge check update #4153

Merged
merged 20 commits into from
Jan 22, 2021
Merged

iotedge check update #4153

merged 20 commits into from
Jan 22, 2021

Conversation

daprilik
Copy link
Contributor

@daprilik daprilik commented Dec 23, 2020

Updates iotedge check to work with the new underlying aziot architecture.

This PR builds on-top of the functionality introduced in the downstream aziot PR at Azure/iot-identity-service#83


In a nutshell, many of the checks that used to be in iotedge check have been ported over into the new aziot check tool. Please see the associated downstream PR for more details. That being said, we don't want to make users invoke multiple check commands, so this PR also includes functionality whereby iotedge will shell-out to aziot when running check and check-list, transparently displaying aziot's check results alongside iotedge-specific checks.

As with the downstream PR, this PR should be reviewed along two axes:

  1. Refactoring work around the actual iotedge check/check-list subcommands
  2. The actual checks themselves (i.e: confirming that all removed checks have indeed been migrated / aren't applicable anymore).

I would really appreciate it if the reviewers actually pulled these changes locally and tested them out, as subtle things such as check output formatting and/or checking check pass/failure cases aren't easy to spot by looking at the code alone.

in the upcoming commits, i'll add the code to shell-out to `aziot
check`, which will populate the `iothub_hostname` variable (which is
currently set to None).
it works, but it could really do with some cleanup. notably, it
currently duplicates all the wire-protocol types from `aziot`, whereas
it really aught to use a shared library to keep things consistent.
the aziot PR has to land before updating the submodule.
kodiakhq bot pushed a commit to Azure/iot-identity-service that referenced this pull request Jan 7, 2021
Adds a new `aziot check` subcommand to `aziot`, modeled after `iotedge check`.

Should be reviewed in conjunction with the upstream PR at Azure/iotedge#4153

Please note that the `aziot check` implementation is _not_ a straight copy-paste from `iotedge check`, and while the two implementations have a lot in common, `aziot check`'s underlying check framework has been substantially refactored, simplified, and modernized. Notable changes include the use of `async_trait` on the `Check` trait, the use of `serde_erased` to simplify checker serialization, and removing all the Windows console specific text formatting code. 

While the internals of `aziot check` are quite different from `iotedge check`, I've made sure that the user-facing `text` and `json` output formats have remained exactly the same between the two tools.

One notable addition to `aziot check` is the new `json-stream` output format, which will enable `iotedge check` to invoke `iotedge check` as a sub-process. Aside from streamlining check flow (users won't have to invoke both `check` commands separately), the `additional_info` payload will be used to share certain bits of provisioning-specific information with `iotedge` that it wouldn't otherwise have access to (e.g: the `iothub_hostname` configuration option used in the host connectivity checks in `iotedge check` is part of the `identityd` config, which `iotedge` should _not_ be able to access directly).

* * *

This PR should be reviewed along two axes:

1. The `aziot check` subcommand, framework, and implementation quality
2. The actual checks themselves

The checks are divided into two categories:

1. Checks ported over from `iotedge check` (due to certain functionality being moved down the stack into `aziot`)
    - `host_connect_dps_endpoint`
    - `host_local_time`
    - `hostname`
    - `identity_certificate_expiry`
    - `well_formed_configs`
2. New checks specific to the new `aziot` architecture
    - `certs_preloaded`
    - `dameons_running`

While the primary goal of this PR is to get `aziot check` "up and running" in an effort to fix `iotedge check`, if there are any checks that I may have missed that are "critical" to merge as part of this initial PR, please let me know.

* * *

This PR will be un-drafted once the corresponding `iotedge check` PR has been submitted, as there's a non-zero chance the `json-streaming` output format will require changes as part of the integration work.
@daprilik daprilik marked this pull request as ready for review January 8, 2021 17:16
edgelet/iotedge/src/check/checks/container_engine_ipv6.rs Outdated Show resolved Hide resolved
edgelet/iotedge/src/main.rs Outdated Show resolved Hide resolved
{
let aziot_check_out = std::process::Command::new(aziot_bin)
.arg("check-list")
.arg("-j")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check-list ought to be changed to be consistent with check and use --output json instead of --json

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough, though given that this isn't a flag that users will ever be expected to use directly, I don't think this bit of inconsistency is too big of a deal. Plus, it'll be a bit of a PITA to update aziot check now that the first PR has already been merged 😉

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sorry about that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sooooo I should make the change then 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

edgelet/iotedge/src/check/mod.rs Outdated Show resolved Hide resolved
edgelet/iotedge/src/check/mod.rs Show resolved Hide resolved
massand
massand previously approved these changes Jan 15, 2021
@massand
Copy link
Contributor

massand commented Jan 15, 2021

:shipit:

@massand
Copy link
Contributor

massand commented Jan 15, 2021

Hold off on any submodule update before merge because I might have exposed a bug in the module runtime with the latest aziot version. I'll post a fix for it shortly. If you aren't updating the aziot submodule, you should be fine.

@massand
Copy link
Contributor

massand commented Jan 15, 2021

#4240 fixes the above issue. If you don't update the submodule, I can do so in my PR.

@daprilik
Copy link
Contributor Author

Hold off on any submodule update before merge because I might have exposed a bug in the module runtime with the latest aziot version. I'll post a fix for it shortly. If you aren't updating the aziot submodule, you should be fine.

I still need to fix #4153 (comment), so I'll end up bumping the submodule regardless.

@massand
Copy link
Contributor

massand commented Jan 16, 2021

One note on testing these iotedge check changes locally (@micahl can provide guidance): iotedge check compares installed version with the latest versions here. We should coordinate on adding aziot-identityd and adding the version check in this PR perhaps?

@arsing
Copy link
Member

arsing commented Jan 16, 2021 via email

@daprilik daprilik dismissed massand’s stale review January 20, 2021 17:27

The base branch was changed.

kodiakhq bot pushed a commit to Azure/iot-identity-service that referenced this pull request Jan 21, 2021
@daprilik daprilik requested review from arsing and massand January 21, 2021 17:52
@huguesBouvier huguesBouvier self-requested a review January 21, 2021 18:53
Copy link
Contributor

@huguesBouvier huguesBouvier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be tested in nested configuration. Check it out with me

@daprilik daprilik force-pushed the iotedge-check-update branch from 3bc24e2 to bb2e0a7 Compare January 21, 2021 19:08
arsing
arsing previously approved these changes Jan 21, 2021
@daprilik daprilik force-pushed the iotedge-check-update branch from 61c1abb to 5ea1255 Compare January 22, 2021 17:11
@daprilik daprilik force-pushed the iotedge-check-update branch from 5ea1255 to 8868430 Compare January 22, 2021 17:36
Copy link
Contributor

@huguesBouvier huguesBouvier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best careful, the run on the pipeline is not compiling:
use edgelet_docker::{Settings, UPSTREAM_PARENT_KEYWORD};
| ^^^^^^^^^^^^^^^^^^^^^^^ no UPSTREAM_PARENT_KEYWORD in the root

@daprilik
Copy link
Contributor Author

Best careful, the run on the pipeline is not compiling:
use edgelet_docker::{Settings, UPSTREAM_PARENT_KEYWORD};
| ^^^^^^^^^^^^^^^^^^^^^^^ no UPSTREAM_PARENT_KEYWORD in the root

Yes indeed, I realized that and force pushed a fix.

Copy link
Contributor

@massand massand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had hit some check errors due to diagnostics module misbehaving. But, the other checks seemed fine. Let us test this after merge, since it will be easier. Thanks!

@kodiakhq kodiakhq bot merged commit 6c7fc9b into Azure:master Jan 22, 2021
@daprilik daprilik deleted the iotedge-check-update branch January 25, 2021 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants