Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Update docs for default pairs #196

Merged
merged 3 commits into from
Sep 6, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/go-storage/internal/specs.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,14 +179,16 @@ The name of current service, should be unique.

[go-storage] has two namespace: `service` and `storage`.

`namespace` COULD implement extra interfaces:
`namespace` COULD implement extra interfaces and features:

```toml
[namespace.storage]
features = ["virtual_dir"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to update this?

implement = ["multiparter"]
```

We should also add related `UnimplementedMultiparter` stub into our `Storage` struct.
- For features, they are defined in `features.toml`. We should check the feature manually when required.
- For interfaces, we should also add related `UnimplementedMultiparter` stub into our `Storage` struct.

`namespace` MUST add `new` section to describe how to initiate this namespace:

Expand Down
10 changes: 7 additions & 3 deletions docs/go-storage/operations/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,25 +25,29 @@ We have connection string support.

The format of the connection string is (optional parts marked by squared brackets):

`<type>://[<name>][<work_dir>][?key1=value1&...&keyN=valueN]`
`<type>://[<name>][<work_dir>][?enable_feature1&...&enable_featureN&key1=value1&...&keyN=valueN]`

- name: storage name, e.g., bucket name. MUST NOT contain /
- work_dir: For object storage, it is prefix; for fs, it is directory path. MUST start with / for every storage services.
- For the `enable_feature` query string:
- `feature` is the feature name defined in `toml`, and the format SHOULD be exactly the same.
- For the `key=value` pairs:
- If `=value` is missing, we just ignore the pair. But `key=` means a pair with a blank value.
- The `key` is the pair name defined in `toml` and the format SHOULD be exactly the same.
- If their are multiple pairs with the same key, the first one will be picked.
- Or the `key` is the default pair name prefixed with `default_` and followed by pair name described above.
- If there are multiple pairs with the same key, the first one will be picked.

So a valid connection string could be:

- `s3://bucket_name`
- `s3://bucket_name/prefix`
- `s3://?credential=hmac:xxxx:xxxx&endpoint=http:s3.us-east-2.amazonaws.com`
- `s3://bucket_name/prefix?credential=hmac:xxxx:xxxx&endpoint=http:s3.us-east-2.amazonaws.com`
- `s3://bucket_name/prefix?enable_virtual_dir&credential=hmac:xxxx:xxxx&endpoint=http:s3.us-east-2.amazonaws.com&default_storage_class=STANDARD`
- `fs://`
- `fs:///tmp`

For more details, take a look at [GSP-90](https://github.com/beyondstorage/specs/blob/master/rfcs/90-re-support-initialization-via-connection-string.md).
For more details, take a look at [GSP-90](https://github.com/beyondstorage/specs/blob/master/rfcs/90-re-support-initialization-via-connection-string.md) and [GSP-700](https://github.com/beyondstorage/go-storage/blob/master/docs/rfcs/700-config-features-and-defaultpairs-via-connection-string.md).

:::caution

Expand Down
90 changes: 88 additions & 2 deletions docs/go-storage/pairs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ We will document all global pair here and leave system pairs in service's docume

`go-storage` provides a mechanism to allow user pass default pairs for every operation during `NewServicer` and `NewStorager`.

Any service that supports this mechanism will generate system pairs called `DefaultServicePairs` and `DefaultStoragePairs`:
Any service that supports this mechanism will generate system pairs called `DefaultServicePairs` and `DefaultStoragePairs`:

```go
type DefaultStoragePairs struct {
Expand Down Expand Up @@ -72,4 +72,90 @@ store, err := s3.NewStorager(
)
```

As in example, every call to `Write` will specify the `storage_class` to `STANDARD_IA`.
As in example, every call to `Write` will specify the `storage_class` to `STANDARD_IA`.

### Defaultable Pair

`defaultable` property for pair is introduced to set default value for pair, and the pair of operations with the same name will share the default value.

Default pairs will be generated for `defaultable` pairs, and the generated pair for defaultable global pair is still global:

```go
// Defaultable global pair in pairs.
func WithDefualtContentType(v string) Pair {
return Pair{
Key: "enable_content_type",
Value: v,
}
}

// Defualtable system pair in service.
func WithDefaultStorageClass(v string) Pair {
return Pair{
Key: "default_storage_class",
Value: v,
}
}
```

User can use default pairs like:

```go
store, err := s3.NewStorager(
s3.WithDefaultStorageClass("STANDARD_IA"),
pairs.WithDefualtContentType("application/octet-stream"),
)
```

As in example, pairs of operations in the service with the same name `storage_class` or `content_type` will share the default values.

## Feature Pairs

`go-storage` provides a mechanism of userland optional abilities for service during `NewServicer` and `NewStorager`.

Any service that supports this mechanism will generate system pairs called `StorageFeatures` and `ServiceFeatures`:

```go
type StorageFeatures struct {
LoosePair bool
VirtualDir bool
}

func WithStorageFeatures(v StorageFeatures) Pair {
return Pair{
Key: "storage_features",
Value: v,
}
}
```

Enable feature pairs will be generated for each feature in service:

```go
func WithEnableVirtualDir() Pair {
return Pair{
Key: "enable_virtual_dir",
Value: true,
}
}
```

User can enable features like this:

```go
store, err := s3.NewStorager(
s3.WithStorageFeatures(s3.StorageFeatures{
VirtualDir: true,
}),
)
```

or:

```go
store, err := s3.NewStorager(
s3.WithEnableVirtualDir(),
)
```

As in the above examples, the service will support simulated dir behavior.