Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Make State/Status/Resource More Readable #1181

Merged
merged 6 commits into from
Oct 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

SELF_DIR=$(dir $(lastword $(MAKEFILE_LIST)))
GOOS=$(shell go env GOOS)
GOPATH=$(shell go env GOPATH)
Expand Down
4 changes: 2 additions & 2 deletions docs/commands/verify.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ If not, it tries to give you a hint that maybe you forgot to run `dtm init` firs

`dtm verify` also tries to create a state manager that operates a backend. If something is wrong with the state, it generates an error telling you what exactly the error is.

## 4 Config / State / Resource
## 4 Config / State / ResourceStatus

For definitions of _Config_, _State_, and _Resource_, see [Core Concepts](../core-concepts/core-concepts.md).
For definitions of _Config_, _State_, and _ResourceStatus_, see [Core Concepts](../core-concepts/core-concepts.md).

`dtm verify` tries to see if the _Config_ matches the _State_ and the _Resource_ or not. If not, it tells you what exactly is not the same, and what would happen if you run `dtm apply`.

Expand Down
6 changes: 3 additions & 3 deletions docs/commands/verify.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@

`dtm verify` 也会尝试创建一个操作 _backend_ 的 _State_ 管理器。若 _State_ 出现问题,将提示错误所在。

## 4 Config / State / Resource
## 4 Config / State / ResourceStatus

关于 _Config_ 、 _State_ 和 _Resource_ 的定义,见[核心概念](../core-concepts/core-concepts.zh.md)。
关于 _Config_ 、 _State_ 和 _ResourceStatus_ 的定义,见[核心概念](../core-concepts/core-concepts.zh.md)。

`dtm verify` 尝试判断 _Config_ 是否与 _State_ 和 _Resource_ 匹配。如果不匹配,它会告诉你到底是什么不一样,以及如果你运行 `dtm apply` 会发生什么。
`dtm verify` 尝试判断 _Config_ 是否与 _State_ 和 _ResourceStatus_ 匹配。如果不匹配,它会告诉你到底是什么不一样,以及如果你运行 `dtm apply` 会发生什么。

如果所有上述检查都成功,`dtm verify` 最后将输出 "Verify succeeded." 日志提示。

Expand Down
10 changes: 5 additions & 5 deletions docs/core-concepts/core-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The architecture image below shows how DevStream works in general:

## Workflow

![config state resource workflow](../images/config_state_resource.png)
![config state resource-status workflow](../images/config_state_resource.png)

## Config, Tool, State and Resource

Expand Down Expand Up @@ -55,10 +55,10 @@ The _State_ records the current status of your DevOps toolchain. It contains the
- The _State_ is actually a map of states, as defined [here](https://github.com/devstream-io/devstream/blob/main/internal/pkg/statemanager/state.go#L24).
- Each state in the map is a struct containing Name, Plugin, Options, and Resource, as defined [here](https://github.com/devstream-io/devstream/blob/main/internal/pkg/statemanager/state.go#L16).

### 4. Resource
### 4. ResourceStatus

- We call what the plugin created a _Resource_, and the `Read()` interface of that plugin returns a description of that resource, which is in turn stored as part of the state.
- We call what the plugin created a _Resource_, and the `Read()` interface of that plugin returns a description of that resource status, which is in turn stored as part of the state.

Config-State-Resource workflow:
Config-State-ResourceStatus workflow:

![config state resource workflow](../images/config_state_resource.png)
![config state resource-status workflow](../images/config_state_resource.png)
14 changes: 7 additions & 7 deletions docs/core-concepts/core-concepts.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@

## 工作流程

![config state resource workflow](../images/config_state_resource.png)
![config state resource-status workflow](../images/config_state_resource.png)

## 配置,工具,状态和资源
## 配置,工具,状态和资源状态

构架文档阐述了 DevStream 的基本工作原理。请确保你在阅读文档其他部分之前完成此部分的阅读。

Expand Down Expand Up @@ -49,15 +49,15 @@ _注意: 你可以将多个YAML文件合并为同一个并用三个短横线(`--

### 3. 状态(State)

_State_ 记录了当下开发运维工具链的状况,包括了每个工具的配置以及当下状态。
_State_ 记录了当前 DevOps 工具链的状态,包括了每个工具的配置以及当下状态。

- _State_ 实际上是一个记录了状态的 Map, 定义在[这里](https://github.com/devstream-io/devstream/blob/main/internal/pkg/statemanager/state.go#L24)。
- Map 中的每一个状态都是一个包含了名称、插件、选项和资源的结构体,定义在[这里](https://github.com/devstream-io/devstream/blob/main/internal/pkg/statemanager/state.go#L16)。

### 4. 资源(Resource)
daniel-hutao marked this conversation as resolved.
Show resolved Hide resolved
### 4. 资源状态(ResourceStatus)

- 我们称插件创建了 _资源(Resource)_,而插件的 `Read()` 接口返回了此资源的描述,该描述也作为状态(Stat)的一部分保存。
- 我们称插件创建了 _资源(Resource)_,而插件的 `Read()` 接口返回了此资源的描述,该描述也作为资源的状态(State)的一部分保存。

配置-状态-资源 工作流:
配置-状态-资源状态 工作流:

![config state resource workflow](../images/config_state_resource.png)
![config state resource-status workflow](../images/config_state_resource.png)
10 changes: 5 additions & 5 deletions docs/core-concepts/stateconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ tools:
org: ""
repo: [[ repoName ]]
branch: [[ defaultBranch ]]
image_repo: [[ dockerhubUsername ]]/[[ repoName ]]
imageRepo: [[ dockerhubUsername ]]/[[ repoName ]]
```

### Using the S3 Backend
Expand Down Expand Up @@ -161,15 +161,15 @@ And if we open the downloaded file, we will see something similar to the followi
```yaml
repo-scaffolding_default:
name: repo-scaffolding
instanceid: default
dependson: []
instanceID: default
dependsOn: []
options:
branch: main
image_repo: ironcore864/dtm-test-go
imageRepo: ironcore864/dtm-test-go
org: ""
owner: IronCore864
repo: dtm-test-go
resource:
resourceStatus:
org: ""
outputs:
org: ""
Expand Down
2 changes: 1 addition & 1 deletion docs/plugins/githubactions-golang.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ tools:
owner: IronCore864
repo: go-webapp-devstream-demo
branch: main
image_repo: ironcore864/go-webapp-devstream-demo
imageRepo: ironcore864/go-webapp-devstream-demo
- name: githubactions-golang
instanceID: default
dependsOn: ["repo-scaffolding.golang-github"]
Expand Down
6 changes: 3 additions & 3 deletions docs/plugins/jenkins.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,8 +284,8 @@ jenkins 插件会保存如下状态:
```yaml
jenkins_default:
name: jenkins
instanceid: default
dependson: []
instanceID: default
dependsOn: []
options:
chart:
valuesYaml: |
Expand All @@ -297,7 +297,7 @@ jenkins_default:
ingress:
enabled: true
hostName: jenkins.example.com
resource:
resourceStatus:
outputs:
jenkins_url: http://jenkins.jenkins:8080
valuesYaml: |
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/configmanager/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ tools:
owner: YOUR_GITHUB_USERNAME_CASE_SENSITIVE
repo: go-webapp-devstream-demo
branch: main
image_repo: YOUR_DOCKER_USERNAME/go-webapp-devstream-demo`)
imageRepo: YOUR_DOCKER_USERNAME/go-webapp-devstream-demo`)
notValidCoreBytes = []byte(`
varFile: tests
toolFile: test
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/configmanager/toolconfig_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ tools:
owner: [[ owner ]]
repo: go-webapp-devstream-demo
branch: main
image_repo: YOUR_DOCKER_USERNAME/go-webapp-devstream-demo`)
imageRepo: YOUR_DOCKER_USERNAME/go-webapp-devstream-demo`)

It("should return error if yaml not valid", func() {
varBytes := []byte(`ownertest_owner`)
Expand Down
4 changes: 2 additions & 2 deletions internal/pkg/develop/plugin/template/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ func Create(options map[string]interface{}) (map[string]interface{}, error) {
TerminateOperations: plugininstaller.TerminateOperations{
// TODO(dtm): Add your TerminateOperations here.
},
GetStateOperation: func(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
// TODO(dtm): Add your GetStateOperation here.
GetStatusOperation: func(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
// TODO(dtm): Add your GetStatusOperation here.
return nil, nil
},
}
Expand Down
4 changes: 2 additions & 2 deletions internal/pkg/develop/plugin/template/read.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ func Read(options map[string]interface{}) (map[string]interface{}, error) {
PreExecuteOperations: plugininstaller.PreExecuteOperations{
// TODO(dtm): Add your PreExecuteOperations here.
},
GetStateOperation: func(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
// TODO(dtm): Add your GetStateOperation here.
GetStatusOperation: func(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
// TODO(dtm): Add your GetStatusOperation here.
return nil, nil
},
}
Expand Down
4 changes: 2 additions & 2 deletions internal/pkg/develop/plugin/template/update.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ func Update(options map[string]interface{}) (map[string]interface{}, error) {
TerminateOperations: plugininstaller.TerminateOperations{
// TODO(dtm): Add your TerminateOperations here.
},
GetStateOperation: func(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
// TODO(dtm): Add your GetStateOperation here.
GetStatusOperation: func(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
// TODO(dtm): Add your GetStatusOperation here.
return nil, nil
},
}
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/argocd/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ func Create(options map[string]interface{}) (map[string]interface{}, error) {
},
ExecuteOperations: helm.DefaultCreateOperations,
TerminateOperations: helm.DefaultTerminateOperations,
GetStateOperation: helm.GetPluginAllState,
GetStatusOperation: helm.GetAllResourcesStatus,
}

// Execute all Operations in Operator
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/argocd/read.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ func Read(options map[string]interface{}) (map[string]interface{}, error) {
helm.SetDefaultConfig(&defaultHelmConfig),
helm.Validate,
},
GetStateOperation: helm.GetPluginAllState,
GetStatusOperation: helm.GetAllResourcesStatus,
}

// 2. get plugin status
Expand Down
4 changes: 2 additions & 2 deletions internal/pkg/plugin/argocd/update.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ func Update(options map[string]interface{}) (map[string]interface{}, error) {
helm.SetDefaultConfig(&defaultHelmConfig),
helm.Validate,
},
ExecuteOperations: helm.DefaultUpdateOperations,
GetStateOperation: helm.GetPluginAllState,
ExecuteOperations: helm.DefaultUpdateOperations,
GetStatusOperation: helm.GetAllResourcesStatus,
}

// 2. update by helm config and get status
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/argocdapp/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ func Create(options map[string]interface{}) (map[string]interface{}, error) {
ExecuteOperations: plugininstaller.ExecuteOperations{
kubectl.ProcessByContent(kubectlUtil.Create, templateFileLoc),
},
GetStateOperation: getStaticState,
GetStatusOperation: getStaticStatus,
}

// Execute all Operations in Operator
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/argocdapp/read.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ func Read(options map[string]interface{}) (map[string]interface{}, error) {
PreExecuteOperations: plugininstaller.PreExecuteOperations{
validate,
},
GetStateOperation: getDynamicState,
GetStatusOperation: getDynamicStatus,
}

// Execute all Operations in Operator
Expand Down
10 changes: 5 additions & 5 deletions internal/pkg/plugin/argocdapp/state.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"github.com/devstream-io/devstream/pkg/util/k8s"
)

func getStaticState(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
func getStaticStatus(options plugininstaller.RawOptions) (statemanager.ResourceStatus, error) {
opts, err := NewOptions(options)
if err != nil {
return nil, err
Expand All @@ -36,15 +36,15 @@ func getStaticState(options plugininstaller.RawOptions) (statemanager.ResourceSt
return res, nil
}

func getDynamicState(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
func getDynamicStatus(options plugininstaller.RawOptions) (statemanager.ResourceStatus, error) {
opts, err := NewOptions(options)
if err != nil {
return nil, err
}

state := make(map[string]interface{})
retStatus := make(statemanager.ResourceStatus)
operation := func() error {
err := getArgoCDAppFromK8sAndSetState(state, opts.App.Name, opts.App.Namespace)
err := getArgoCDAppFromK8sAndSetState(retStatus, opts.App.Name, opts.App.Namespace)
if err != nil {
return err
}
Expand All @@ -56,7 +56,7 @@ func getDynamicState(options plugininstaller.RawOptions) (statemanager.ResourceS
if err != nil {
return nil, err
}
return state, nil
return retStatus, nil
}

func getArgoCDAppFromK8sAndSetState(state map[string]interface{}, name, namespace string) error {
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/artifactory/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ func Create(options map[string]interface{}) (map[string]interface{}, error) {
},
ExecuteOperations: helm.DefaultCreateOperations,
TerminateOperations: helm.DefaultTerminateOperations,
GetStateOperation: helm.GetPluginAllState,
GetStatusOperation: helm.GetAllResourcesStatus,
}

// Execute all Operations in Operator
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/artifactory/read.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ func Read(options map[string]interface{}) (map[string]interface{}, error) {
helm.SetDefaultConfig(&defaultHelmConfig),
helm.Validate,
},
GetStateOperation: helm.GetPluginAllState,
GetStatusOperation: helm.GetAllResourcesStatus,
}

status, err := operator.Execute(plugininstaller.RawOptions(options))
Expand Down
4 changes: 2 additions & 2 deletions internal/pkg/plugin/artifactory/update.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ func Update(options map[string]interface{}) (map[string]interface{}, error) {
helm.SetDefaultConfig(&defaultHelmConfig),
helm.Validate,
},
ExecuteOperations: helm.DefaultUpdateOperations,
GetStateOperation: helm.GetPluginAllState,
ExecuteOperations: helm.DefaultUpdateOperations,
GetStatusOperation: helm.GetAllResourcesStatus,
}

// Execute all Operations in Operator
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/cigeneric/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ func Create(options map[string]interface{}) (map[string]interface{}, error) {
ExecuteOperations: plugininstaller.ExecuteOperations{
ci.PushCIFiles,
},
GetStateOperation: ci.GetCIFileStatus,
GetStatusOperation: ci.GetCIFileStatus,
}

// Execute all Operations in Operator
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/cigeneric/read.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ func Read(options map[string]interface{}) (map[string]interface{}, error) {
PreExecuteOperations: plugininstaller.PreExecuteOperations{
ci.Validate,
},
GetStateOperation: ci.GetCIFileStatus,
GetStatusOperation: ci.GetCIFileStatus,
}

status, err := operator.Execute(plugininstaller.RawOptions(options))
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/devlake/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ func Create(options map[string]interface{}) (map[string]interface{}, error) {
},
ExecuteOperations: helm.DefaultCreateOperations,
TerminateOperations: helm.DefaultTerminateOperations,
GetStateOperation: genDevLakeState,
GetStatusOperation: genDevLakeStatus,
}

// Execute all Operations in Operator
Expand Down
10 changes: 5 additions & 5 deletions internal/pkg/plugin/devlake/devlake.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ var defaultHelmConfig = helm.Options{
},
}

func genDevLakeState(options plugininstaller.RawOptions) (statemanager.ResourceState, error) {
resState, err := helm.GetPluginAllState(options)
func genDevLakeStatus(options plugininstaller.RawOptions) (statemanager.ResourceStatus, error) {
resStatus, err := helm.GetAllResourcesStatus(options)
if err != nil {
return nil, err
}
Expand All @@ -42,7 +42,7 @@ func genDevLakeState(options plugininstaller.RawOptions) (statemanager.ResourceS
return nil, err
}
valuesYaml := opt.GetHelmParam().Chart.ValuesYaml
resState["valuesYaml"] = valuesYaml
resStatus["valuesYaml"] = valuesYaml

// TODO(daniel-hutao): Use Ingress later.
ip, err := getDevLakeClusterIP(opt.Chart.Namespace, DevLakeSvcName)
Expand All @@ -53,9 +53,9 @@ func genDevLakeState(options plugininstaller.RawOptions) (statemanager.ResourceS
outputs := map[string]interface{}{
"devlake_url": url,
}
resState.SetOutputs(outputs)
resStatus.SetOutputs(outputs)

return resState, nil
return resStatus, nil
}

func getDevLakeClusterIP(namespace, name string) (string, error) {
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/devlake/read.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ func Read(options map[string]interface{}) (map[string]interface{}, error) {
helm.SetDefaultConfig(&defaultHelmConfig),
helm.Validate,
},
GetStateOperation: genDevLakeState,
GetStatusOperation: genDevLakeStatus,
}

// Execute all Operations in Operator
Expand Down
4 changes: 2 additions & 2 deletions internal/pkg/plugin/devlake/update.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ func Update(options map[string]interface{}) (map[string]interface{}, error) {
helm.SetDefaultConfig(&defaultHelmConfig),
helm.Validate,
},
ExecuteOperations: helm.DefaultUpdateOperations,
GetStateOperation: genDevLakeState,
ExecuteOperations: helm.DefaultUpdateOperations,
GetStatusOperation: genDevLakeStatus,
}

// Execute all Operations in Operator
Expand Down
2 changes: 1 addition & 1 deletion internal/pkg/plugin/devlakeconfig/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ func Create(options map[string]interface{}) (map[string]interface{}, error) {
TerminateOperations: plugininstaller.TerminateOperations{
// TODO(dtm): Add your TerminateOperations here.
},
GetStateOperation: GetState,
GetStatusOperation: GetStatus,
}

// Execute all Operations in Operator
Expand Down
Loading