Skip to content

Commit

Permalink
Merge pull request #968 from merico-dev/refactor-1
Browse files Browse the repository at this point in the history
refactor: configloader -> configmanager
  • Loading branch information
daniel-hutao authored Aug 9, 2022
2 parents add2cfd + 0a7c823 commit a45b3e8
Show file tree
Hide file tree
Showing 35 changed files with 232 additions and 228 deletions.
4 changes: 2 additions & 2 deletions cmd/devstream/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"github.com/spf13/cobra"

"github.com/devstream-io/devstream/internal/pkg/completion"
"github.com/devstream-io/devstream/internal/pkg/configloader"
"github.com/devstream-io/devstream/internal/pkg/configmanager"
"github.com/devstream-io/devstream/internal/pkg/pluginengine"
"github.com/devstream-io/devstream/internal/pkg/pluginmanager"
"github.com/devstream-io/devstream/internal/pkg/version"
Expand All @@ -21,7 +21,7 @@ var initCMD = &cobra.Command{
}

func initCMDFunc(_ *cobra.Command, _ []string) {
cfg, err := configloader.LoadConfig(configFile)
cfg, err := configmanager.NewManager(configFile).LoadConfig()
if err != nil {
log.Errorf("Error: %s.", err)
return
Expand Down
2 changes: 1 addition & 1 deletion docs/core-concepts/core-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ _Note: you can put multiple YAML files into the same one with three dashes (`---
### 2. Tool

- Each _Tool_ corresponds to a plugin, which can either be used to install, configure, or integrate some DevOps tools.
- Each _Tool_ has its Name, InstanceID, and Options, as defined [here](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configloader/toolconfig.go#L13).
- Each _Tool_ has its Name, InstanceID, and Options, as defined [here](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configmanager/toolconfig.go#L13).
- Each _Tool_ can have its dependencies, which are specified by the `dependsOn` keyword.

The dependency `dependsOn` is an array of strings, with each element being a dependency. Each dependency is named in the format of "TOOL_NAME.INSTANCE_ID".
Expand Down
2 changes: 1 addition & 1 deletion docs/core-concepts/core-concepts.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ _注意: 你可以将多个YAML文件合并为同一个并用三个短横线(`--
### 2. 工具(Tool)

- 每个 _Tool_ 对应一个插件, 即可用于安装和配置,也可用于整合 DevOps 的工具。
- 每个 _Tool_ 有名称, (实例ID)InstanceID 和选项(Options), 定义在[这里](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configloader/toolconfig.go#L13)
- 每个 _Tool_ 有名称, (实例ID)InstanceID 和选项(Options), 定义在[这里](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configmanager/toolconfig.go#L13)
- 每个 _Tool_ 可以使用`dependsOn` 字段指定其依赖项。

`dependsOn` 是一个字符串数组, 其中每一个元素都是一个依赖。 每个依赖项都以 "TOOL_NAME.INSTANCE_ID" 为格式命名。
Expand Down
4 changes: 2 additions & 2 deletions docs/development/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Every time a user runs the `dtm` program, the execution transfers immediately in

Then, each command calls the plugin engine package under [`internal/pkg`](https://github.com/devstream-io/devstream/tree/main/internal/pkg/pluginengine).

The `pluginengine` calls the [config manager package](https://github.com/devstream-io/devstream/tree/main/internal/pkg/configloader) first to read the local YAML config file into a struct.
The `pluginengine` calls the [config manager package](https://github.com/devstream-io/devstream/tree/main/internal/pkg/configmanager) first to read the local YAML config file into a struct.

Then it calls the [`pluginmanager` package](https://github.com/devstream-io/devstream/tree/main/internal/pkg/pluginmanager) to download the required plugins.

Expand All @@ -40,7 +40,7 @@ It achieves the goal by calling the following modules:

### 2.1 Config Manager

Model types in package [`configloader`](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configloader/config.go#L23) represent the top-level configuration structure.
Model types in package [`configmanager`](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configmanager/config.go#L23) represent the top-level configuration structure.

### 2.2 Plugin Manager

Expand Down
4 changes: 2 additions & 2 deletions docs/development/architecture.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ _注意:为了简单起见,CLI被命名为`dtm`(DevOps Toolchain Manager)

然后,每个命令调用[`internal/pkg`](https://github.com/devstream-io/devstream/tree/main/internal/pkg/pluginengine)下的`pluginengine`包。

`pluginengine`首先调用`configloader`,将本地YAML配置文件读取到一个结构体中,然后调用`pluginmanager`来下载所需的插件。
`pluginengine`首先调用`configmanager`,将本地YAML配置文件读取到一个结构体中,然后调用`pluginmanager`来下载所需的插件。

之后,`pluginengine`调用`statemanager`来计算congfig、状态和实际DevOps工具的状态之间的"差异"。最后,`pluginengine`根据这变更执行对应的操作,并更新状态。在执行过程中,`pluginengine`加载每个插件(`*.so`文件)并根据每个变更调用相应的接口。

Expand All @@ -38,7 +38,7 @@ Plugin engine通过调用以下模块来实现目标:

### 2.1 配置管理器

[`configloader`](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configloader/config.go#L23)中的模型结构体用来表示顶级配置的结构。
[`configmanager`](https://github.com/devstream-io/devstream/blob/main/internal/pkg/configmanager/config.go#L23)中的模型结构体用来表示顶级配置的结构。

### 2.2 插件管理器

Expand Down
4 changes: 2 additions & 2 deletions internal/pkg/backend/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (

"github.com/devstream-io/devstream/internal/pkg/backend/local"
"github.com/devstream-io/devstream/internal/pkg/backend/s3"
"github.com/devstream-io/devstream/internal/pkg/configloader"
"github.com/devstream-io/devstream/internal/pkg/configmanager"
"github.com/devstream-io/devstream/pkg/util/log"
)

Expand All @@ -25,7 +25,7 @@ type Backend interface {
}

// GetBackend will return a Backend by the given name.
func GetBackend(stateConfig configloader.State) (Backend, error) {
func GetBackend(stateConfig configmanager.State) (Backend, error) {
typeName := Type(stateConfig.Backend)
switch typeName {
case Local:
Expand Down
6 changes: 3 additions & 3 deletions internal/pkg/backend/backend_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ import (
. "github.com/onsi/gomega"

"github.com/devstream-io/devstream/internal/pkg/backend"
"github.com/devstream-io/devstream/internal/pkg/configloader"
"github.com/devstream-io/devstream/internal/pkg/configmanager"
)

var _ = Describe("GetBackend", func() {
When("use local backend", func() {
It("should return local backend struct", func() {
state := configloader.State{Backend: "local"}
state := configmanager.State{Backend: "local"}
_, err := backend.GetBackend(state)
Expect(err).Error().ShouldNot(HaveOccurred())
})
Expand All @@ -20,7 +20,7 @@ var _ = Describe("GetBackend", func() {
// TODO: add mock s3 test
When("use unknown backend", func() {
It("should return err", func() {
state := configloader.State{Backend: "not_exist_plug"}
state := configmanager.State{Backend: "not_exist_plug"}
_, err := backend.GetBackend(state)
Expect(err).Error().Should(HaveOccurred())
Expect(err.Error()).Should(Equal("the backend type < not_exist_plug > is illegal"))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package configloader
package configmanager

import (
"bytes"
Expand Down Expand Up @@ -39,7 +39,7 @@ state:
stateFile: devstream.state`)
})
Describe("LoadConfig yaml", func() {
configStateObj, err := LoadConfig("../../../examples/quickstart.yaml")
configStateObj, err := NewManager("../../../examples/quickstart.yaml").LoadConfig()
Context("when the Yaml parses successfully", func() {
It("should state filed correctly", func() {
Expect(configStateObj.State.Backend).To(Or(Equal("local"), Equal("s3")))
Expand All @@ -65,13 +65,13 @@ state:
It("should return error if data is not valid yaml string", func() {
notValidStr := "this is not valid yaml"
notValidBytes := []byte(notValidStr)
config, err := renderConfigs(notValidBytes, emptyVariable, emptyVariable)
config, err := NewManager("").renderConfigs(notValidBytes, emptyVariable, emptyVariable)
Expect(err).Error().Should(HaveOccurred())
Expect(config).Should(BeNil())
})

It("should return error if core config is not valid", func() {
config, err := renderConfigs(notValidCoreBytes, emptyVariable, emptyVariable)
config, err := NewManager("").renderConfigs(notValidCoreBytes, emptyVariable, emptyVariable)
Expect(err).Error().Should(HaveOccurred())
Expect(config).Should(BeNil())
})
Expand All @@ -80,7 +80,7 @@ state:

Context("when config all valid", func() {
It("should generate config", func() {
config, err := renderConfigs(validCoreBytes, emptyVariable, validToolBytes)
config, err := NewManager("").renderConfigs(validCoreBytes, emptyVariable, validToolBytes)
Expect(err).Error().ShouldNot(HaveOccurred())
Expect(config).ShouldNot(BeNil())
})
Expand All @@ -91,7 +91,7 @@ state:
Context("when config error", func() {
It("should return error if tool file is empty and tools config file is empty", func() {
config := CoreConfig{}
tools, err := renderToolsFromCoreConfigAndConfigBytes(&config, emptyVariable, emptyVariable)
tools, err := NewManager("").renderToolsFromCoreConfigAndConfigBytes(&config, emptyVariable, emptyVariable)
Expect(err).Error().Should(HaveOccurred())
Expect(err.Error()).Should(Equal("tools config is empty"))
Expect(tools).Should(BeEmpty())
Expand All @@ -101,7 +101,7 @@ state:
Context("when tool config valid", func() {
It("should generate tools array", func() {
config := CoreConfig{}
tools, err := renderToolsFromCoreConfigAndConfigBytes(&config, validToolBytes, emptyVariable)
tools, err := NewManager("").renderToolsFromCoreConfigAndConfigBytes(&config, validToolBytes, emptyVariable)
Expect(err).Error().ShouldNot(HaveOccurred())
Expect(tools).ShouldNot(BeEmpty())
})
Expand All @@ -112,21 +112,21 @@ state:
Context("when file name error", func() {
It("should return error", func() {
errorFileName := "error_file_name.yaml"
_, err := loadOriginalConfigFile(errorFileName)
_, err := NewManager(errorFileName).loadOriginalConfigFile()
Expect(err).Error().Should(HaveOccurred())
})
})
})

Describe("SplitConfigFileBytes func", func() {
Describe("splitConfigFileBytes func", func() {
Context("when input text not valid", func() {
It("should return error if fomat not valid", func() {
notValidContent := []byte(`
---
---
---
`)
_, _, _, err := SplitConfigFileBytes(notValidContent)
_, _, _, err := NewManager("").splitConfigFileBytes(notValidContent)
Expect(err).Error().Should(HaveOccurred())
Expect(err.Error()).Should(Equal("invalid config format"))
})
Expand All @@ -140,7 +140,7 @@ tools:
tools:
- name: repo-scaffolding
instanceID: default`)
_, _, _, err := SplitConfigFileBytes(notValidContent)
_, _, _, err := NewManager("").splitConfigFileBytes(notValidContent)
Expect(err).Error().Should(HaveOccurred())
Expect(err.Error()).Should(ContainSubstring("multiple sections"))
})
Expand All @@ -155,7 +155,7 @@ tools:
validToolBytes,
}
validInput := bytes.Join(validArray, []byte("\n"))
coreConfigBytes, _, toolConfigBytes, err := SplitConfigFileBytes(validInput)
coreConfigBytes, _, toolConfigBytes, err := NewManager("").splitConfigFileBytes(validInput)
Expect(err).Error().ShouldNot(HaveOccurred())
Expect(coreConfigBytes).ShouldNot(BeEmpty())
Expect(toolConfigBytes).ShouldNot(BeEmpty())
Expand All @@ -170,22 +170,22 @@ tools:
test:
- name: repo-scaffolding
instanceID: default`)
isValid, err := checkConfigType(notValidType, "core")
isValid, err := NewManager("").checkConfigType(notValidType, "core")
Expect(err).Error().ShouldNot(HaveOccurred())
Expect(isValid).Should(BeFalse())
})

It("should return error if data is not valid yaml string", func() {
notValidStr := "this is not valid yaml"
notValidBytes := []byte(notValidStr)
isValid, err := checkConfigType(notValidBytes, "core")
isValid, err := NewManager("").checkConfigType(notValidBytes, "core")
Expect(err).Error().Should(HaveOccurred())
Expect(isValid).Should(BeFalse())
})
})
Context("when input is right", func() {
It("should return true and error is nil", func() {
isValid, err := checkConfigType(validCoreBytes, "core")
isValid, err := NewManager("").checkConfigType(validCoreBytes, "core")
Expect(err).Error().ShouldNot(HaveOccurred())
Expect(isValid).Should(BeTrue())

Expand Down
Loading

0 comments on commit a45b3e8

Please sign in to comment.