From ef80d0faf806aea1751ffbfa69ea34b8a619e923 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Sat, 7 Sep 2024 13:13:52 -0400 Subject: [PATCH 01/11] docs: add page to opman on getting production-ready --- docs/operation/modules/ROOT/nav.adoc | 1 + .../modules/ROOT/pages/productionizing.adoc | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) create mode 100644 docs/operation/modules/ROOT/pages/productionizing.adoc diff --git a/docs/operation/modules/ROOT/nav.adoc b/docs/operation/modules/ROOT/nav.adoc index 478935ef..4cbddfb4 100644 --- a/docs/operation/modules/ROOT/nav.adoc +++ b/docs/operation/modules/ROOT/nav.adoc @@ -42,3 +42,4 @@ * xref:extensibility.adoc[] * xref:telemetry.adoc[] * xref:migrate-tilestache.adoc[] +* xref:productionizing.adoc[] diff --git a/docs/operation/modules/ROOT/pages/productionizing.adoc b/docs/operation/modules/ROOT/pages/productionizing.adoc new file mode 100644 index 00000000..2e075beb --- /dev/null +++ b/docs/operation/modules/ROOT/pages/productionizing.adoc @@ -0,0 +1,16 @@ += Productionizing + +When running tilegroxy by default it runs in a non-hardened mode most fit for development use-cases. If you want to operate tilegroxy in a production use-case with substantial traffic consider the following action items: + +[%interactive] +* [ ] Deploy tilegroxy in a high-availability manner (that is to say N+1 Load Balanced) +* [ ] Configure Server.Production=true to disable X-Powered-By header and developer documentation +* [ ] Configure Server.Header with any static headers you want returned such as CORS headers. A good reference of headers to consider is available link:https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html[here] +* [ ] Configure Client.UserAgent with a agent that reflects your organization rather than tilegroxy itself +* [ ] Configure logging to output at warning or error levels to avoid a bad signal to noise ratio +* [ ] Consume logs into your aggregator or observability solution with a tool such as fluentbit/filebeat and/or rotate them with a tool such as logrotate +* [ ] Ensure you have authentication configured either inside tilegroxy or upstream +* [ ] Ensure you have monitoring of the health endpoints and the endpoints are not exposed to the internet +* [ ] Ensure cache-ing is configured against a distributed, high availability data-store +* [ ] Consume telemetry data into an observability solution with trace sampling set at a level that won't overwhelm you +* [ ] Verify tilegroxy will be restarted on the unfortunate event it crashes such as via Docker's run=unless-stopped, Kubernete's restart policy, or SystemD's Restart=on-failure From 0e0a5c0e8caf9f5df24a95497cc2c106024e90f0 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Sat, 7 Sep 2024 15:49:35 -0400 Subject: [PATCH 02/11] docs: add documentation on health config --- docs/operation/modules/ROOT/nav.adoc | 1 + .../ROOT/pages/configuration/health.adoc | 197 ++++++++++++++++++ .../ROOT/pages/configuration/server.adoc | 8 +- .../modules/ROOT/pages/content-type.adoc | 3 + 4 files changed, 208 insertions(+), 1 deletion(-) create mode 100644 docs/operation/modules/ROOT/pages/configuration/health.adoc create mode 100644 docs/operation/modules/ROOT/pages/content-type.adoc diff --git a/docs/operation/modules/ROOT/nav.adoc b/docs/operation/modules/ROOT/nav.adoc index 4cbddfb4..6d1aa671 100644 --- a/docs/operation/modules/ROOT/nav.adoc +++ b/docs/operation/modules/ROOT/nav.adoc @@ -29,6 +29,7 @@ *** xref:configuration/secret/aws_secrets_manager.adoc[] ** xref:configuration/server.adoc[] *** xref:configuration/encryption.adoc[] +*** xref:configuration/health.adoc[] ** xref:configuration/client.adoc[] ** xref:configuration/log.adoc[] ** xref:configuration/telemetry.adoc[] diff --git a/docs/operation/modules/ROOT/pages/configuration/health.adoc b/docs/operation/modules/ROOT/pages/configuration/health.adoc new file mode 100644 index 00000000..d0448b0d --- /dev/null +++ b/docs/operation/modules/ROOT/pages/configuration/health.adoc @@ -0,0 +1,197 @@ += Health + +Tilegroxy includes a facility for validating the health of the tilegroxy server. This information is served from a separate port from the main endpoints; this is done for security reasons to avoid the risk of leaking internal details which could be used to create targeted attacks. This port should not be exposed outside your local network. The intent is for the health endpoints to be used with load balancers, monitoring tools, or orchestration systems (such as kubernetes). + +There are two specific endpoints available: `/` and `/health`. + +Hitting `/` will return a 200 once the tilegroxy server is online. + +Hitting `/health` will invoke the full health check. + +The full healthcheck consists of invoking a series of individual checks. The list of specific checks is configurable. Actual execution of these checks is fully asynchronous, which means calling the health endpoint returns the cached results of the most recent check rather than triggering the checks immediately. + +Configuration options: + +[cols="1,3,1,1,1"] +|=== +| Parameter | Description | Type | Required | Default + +| Enabled +| If set to false the port isn't bound to +| boolean +| No +| true + +| Port +| The port to serve health on +| int +| No +| 3000 + +| Host +| The host to bind to +| string +| No +| 0.0.0.0 + +| Checks +| An array defining the specific checks to perform. See <> +| Check[] +| Yes +| None + +|=== + +The following can be supplied as environment variables: + +|=== +| Configuration Parameter | Environment Variable + +| Enabled +| SERVER_HEALTH_ENABLED + +| Port +| SERVER_HEALTH_PORT + +| Host +| SERVER_HEALTH_HOST + +|=== + + +== Checks + +There are a number of different types of checks that can be performed. Like elsewhere in the configuration, these different types of checks are controlled by the `name` parameter and the remaining parameters available depend on which one you select. + +=== Tile + +This check generates a synthetic request for a specific layer and validates the result is as expected. This allows you to detect issues connecting to an upstream provider or alert when broken imagery begins to be returned. These requests are generated internally and do not go through HTTP so will not show up in access logs, do not trigger authentication, and will not have a user ID associated. If telemetry is enabled, these requests will count against per-layer tile metrics but not against total tile metrics. + +THIS CHECK DOES NOT USE THE CACHE. Do not use this check against an intensive layer or one that incurs a monetary cost. + + +Name should be "tile" + +Configuration options: + +[cols="1,3,1,1,1"] +|=== +| Parameter | Description | Type | Required | Default + +| delay +| How long to wait (in seconds) between subsequent checks +| int +| No +| 600 + +| layer +| The layername to send the request against +| string +| Yes +| None + +| z +| The z coordinate (zoom) of the request +| int +| No +| 10 + +| x +| The x coordinate of the request +| int +| No +| 123 + +| y +| The y coordinate of the request +| int +| No +| 534 + +| validation +| How to validate the resulting tile is as expected. Possible options: same, content-type, base-64, file, success. See below for details +| string +| Yes +| same + +| result +| The expected result of the tile. Interpreted based on the value specified in `validation` +| string +| No +| None + +|=== + + +Once the tile is generated there's a few options for how you validate the tile is as you expect: + +[cols="1,3,2"] +|=== +| Validation | Description | How result is interpreted + +| same +| Checks that the tile generated is the same as the previous tile generated. The first check after startup will act the same as `success`. If the value of the tile changes and then stays the same, the first check will return an error but subsequent checks will return back to healthy. +| Result is unused + +| content-type +| Checks the content type of the resulting tile but doesn't check the tile itself. This requires the tile to come back from the provider with a known MIME type. +// See link:https://tilegroxy.michael.davis.name/operation/content-type.html[content-type] for details on how this is handled. +| Result is the exact MIME-type of the tile + +| base-64 +| Checks the generated tile exactly matches a specified value. +| Result is the base64 encoded contents of the tile + +| file +| Checks the generated tile exactly matches a specified value. +| Result is the filepath to retrieve the expected contents of the tile. File is read into memory upon every check. + +| success +| Doesn't check the value of the generated tile. As long as no error is encountered when generating the tile the check is considered healthy. +| Result is unused + +|=== + + +=== Cache + +This check puts a value into the cache, retrieves it, and confirms the value matches. + +The key used to insert into the cache corresponds to a request against layer named "\_\__hc___" at coordinates 0,0,0. The value inserted into the cache changes each time. + +Name should be "cache" + +Configuration options: + +[cols="1,3,1,1,1"] +|=== +| Parameter | Description | Type | Required | Default + +| delay +| How long to wait (in seconds) between subsequent checks +| int +| No +| 600 + +|=== + +== Example + + +---- +server: + health: + port: 3020 + checks: + - name: tile + layer: some_layer + x: 0 + y: 0 + z: 0 + validation: content-type + result: image/png; charset=UTF-8 + delay: 60 + - name: cache + delay: 600 + +---- diff --git a/docs/operation/modules/ROOT/pages/configuration/server.adoc b/docs/operation/modules/ROOT/pages/configuration/server.adoc index d119c9b6..43ee234c 100644 --- a/docs/operation/modules/ROOT/pages/configuration/server.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/server.adoc @@ -57,10 +57,16 @@ Configuration options: | Whether to gzip compress HTTP responses | Encrypt -| <> +| xref:encryption.adoc[Encryption] | No | None | Configuration for enabling TLS (HTTPS). Don't specify to operate without encryption (the default) + +| Health +| xref:health.adoc[Health] +| No +| None +| Configuration to turn on endpoints for validating the health of the server on a secondary port |=== The following can be supplied as environment variables: diff --git a/docs/operation/modules/ROOT/pages/content-type.adoc b/docs/operation/modules/ROOT/pages/content-type.adoc new file mode 100644 index 00000000..19780c1c --- /dev/null +++ b/docs/operation/modules/ROOT/pages/content-type.adoc @@ -0,0 +1,3 @@ += Content Type + +TODO: details about how content-type is handled \ No newline at end of file From ccba90ccbc34df2b303602203ac12a6b89d10dd4 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Sat, 7 Sep 2024 16:25:46 -0400 Subject: [PATCH 03/11] docs: fix xref --- docs/operation/modules/ROOT/pages/configuration/server.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/operation/modules/ROOT/pages/configuration/server.adoc b/docs/operation/modules/ROOT/pages/configuration/server.adoc index 43ee234c..5e15058e 100644 --- a/docs/operation/modules/ROOT/pages/configuration/server.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/server.adoc @@ -57,13 +57,13 @@ Configuration options: | Whether to gzip compress HTTP responses | Encrypt -| xref:encryption.adoc[Encryption] +| xref:configuration/encryption.adoc[Encryption] | No | None | Configuration for enabling TLS (HTTPS). Don't specify to operate without encryption (the default) | Health -| xref:health.adoc[Health] +| xref:configuration/health.adoc[Health] | No | None | Configuration to turn on endpoints for validating the health of the server on a secondary port From c8a372a9c71bb777015374e69fb4bb55ade588ce Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Sat, 7 Sep 2024 16:26:06 -0400 Subject: [PATCH 04/11] refactor: split up server.go as it's getting a bit unwieldy --- internal/server/context_handler.go | 40 +++ internal/server/error.go | 103 ++++++++ internal/server/log.go | 171 +++++++++++++ internal/server/redirect_handler.go | 25 ++ internal/server/server.go | 366 +++++----------------------- 5 files changed, 401 insertions(+), 304 deletions(-) create mode 100644 internal/server/context_handler.go create mode 100644 internal/server/error.go create mode 100644 internal/server/log.go create mode 100644 internal/server/redirect_handler.go diff --git a/internal/server/context_handler.go b/internal/server/context_handler.go new file mode 100644 index 00000000..6240440a --- /dev/null +++ b/internal/server/context_handler.go @@ -0,0 +1,40 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package server + +import ( + "fmt" + "net/http" + "runtime/debug" + + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" +) + +type httpContextHandler struct { + http.Handler + errCfg config.ErrorConfig +} + +func (h httpContextHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { + reqContext := pkg.NewRequestContext(req) + defer func() { + if err := recover(); err != nil { + writeErrorMessage(reqContext, w, &h.errCfg, pkg.TypeOfErrorOther, fmt.Sprint(err), "Unexpected Internal Server Error", debug.Stack()) + } + }() + + h.Handler.ServeHTTP(w, req.WithContext(reqContext)) +} diff --git a/internal/server/error.go b/internal/server/error.go new file mode 100644 index 00000000..80543898 --- /dev/null +++ b/internal/server/error.go @@ -0,0 +1,103 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package server + +import ( + "context" + "errors" + "fmt" + "log/slog" + "net/http" + "runtime/debug" + + "github.com/Michad/tilegroxy/internal/images" + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" +) + +func errorVars(cfg *config.ErrorConfig, errorType pkg.TypeOfError) (int, slog.Level, string) { + var status int + var level slog.Level + var imgPath string + + switch errorType { + case pkg.TypeOfErrorAuth: + level = slog.LevelDebug + status = http.StatusUnauthorized + imgPath = cfg.Images.Authentication + case pkg.TypeOfErrorBounds: + level = slog.LevelDebug + status = http.StatusBadRequest + imgPath = cfg.Images.OutOfBounds + case pkg.TypeOfErrorProvider: + level = slog.LevelInfo + status = http.StatusInternalServerError + imgPath = cfg.Images.Provider + case pkg.TypeOfErrorBadRequest: + level = slog.LevelDebug + status = http.StatusBadRequest + imgPath = cfg.Images.Other + default: + level = slog.LevelWarn + status = http.StatusInternalServerError + imgPath = cfg.Images.Other + } + + if cfg.AlwaysOK { + status = http.StatusOK + } + + return status, level, imgPath +} + +func writeError(ctx context.Context, w http.ResponseWriter, cfg *config.ErrorConfig, err error) { + var te pkg.TypedError + if errors.As(err, &te) { + writeErrorMessage(ctx, w, cfg, te.Type(), te.Error(), te.External(cfg.Messages), debug.Stack()) + } else { + writeErrorMessage(ctx, w, cfg, pkg.TypeOfErrorOther, err.Error(), fmt.Sprintf(cfg.Messages.ServerError, err), debug.Stack()) + } +} + +func writeErrorMessage(ctx context.Context, w http.ResponseWriter, cfg *config.ErrorConfig, errorType pkg.TypeOfError, internalMessage string, externalMessage string, stack []byte) { + status, level, imgPath := errorVars(cfg, errorType) + + slog.Log(ctx, level, internalMessage, "stack", string(stack)) + + switch cfg.Mode { + case config.ModeErrorPlainText: + w.WriteHeader(status) + _, err := w.Write([]byte(externalMessage)) + if err != nil { + slog.WarnContext(ctx, fmt.Sprintf("error writing error %v", err)) + } + case config.ModeErrorImage, config.ModeErrorImageHeader: + if cfg.Mode == config.ModeErrorImageHeader { + w.Header().Add("X-Error-Message", externalMessage) + } + w.WriteHeader(status) + + img, err2 := images.GetStaticImage(imgPath) + if img != nil && err2 == nil { + _, err2 = w.Write(*img) + } + + if err2 != nil { + slog.ErrorContext(ctx, err2.Error()) + } + default: + w.WriteHeader(status) + } +} diff --git a/internal/server/log.go b/internal/server/log.go new file mode 100644 index 00000000..b0354f43 --- /dev/null +++ b/internal/server/log.go @@ -0,0 +1,171 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package server + +import ( + "context" + "fmt" + "io" + "log/slog" + "net/http" + "os" + "slices" + "strings" + + "github.com/Michad/tilegroxy/pkg/config" + "github.com/Michad/tilegroxy/pkg/static" + "github.com/gorilla/handlers" + "go.opentelemetry.io/contrib/bridges/otelslog" +) + +type slogContextHandler struct { + slog.Handler + keys []string +} + +func (h slogContextHandler) Handle(ctx context.Context, r slog.Record) error { + for _, k := range h.keys { + r.AddAttrs(slog.Attr{Key: strings.ToLower(k), Value: slog.AnyValue(ctx.Value(k))}) + } + + return h.Handler.Handle(ctx, r) +} + +func makeLogFileWriter(path string, alsoStdOut bool) (io.Writer, error) { + logFile, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666) + + if err != nil { + return nil, err + } + var out io.Writer + + if alsoStdOut { + out = io.MultiWriter(os.Stdout, logFile) + } else { + out = logFile + } + return out, nil +} + +func configureMainLogging(cfg *config.Config) error { + if !cfg.Logging.Main.Console && len(cfg.Logging.Main.Path) == 0 { + slog.SetLogLoggerLevel(slog.LevelError + 1) + return nil + } + + var err error + var out io.Writer + if len(cfg.Logging.Main.Path) > 0 { + out, err = makeLogFileWriter(cfg.Logging.Main.Path, cfg.Logging.Main.Console) + if err != nil { + return err + } + } else { + out = os.Stdout + } + + var level slog.Level + custLogLevel, ok := config.CustomLogLevel[strings.ToLower(cfg.Logging.Main.Level)] + + if ok { + level = custLogLevel + } else { + err := level.UnmarshalText([]byte(cfg.Logging.Main.Level)) + + if err != nil { + return err + } + } + + opt := slog.HandlerOptions{ + AddSource: true, + Level: level, + ReplaceAttr: func(groups []string, a slog.Attr) slog.Attr { + if groups == nil && a.Key == "msg" { + return slog.Attr{Key: "message", Value: a.Value} + } + return a + }, + } + + var logHandler slog.Handler + + switch cfg.Logging.Main.Format { + case config.MainFormatPlain: + logHandler = slog.NewTextHandler(out, &opt) + case config.MainFormatJSON: + logHandler = slog.NewJSONHandler(out, &opt) + if cfg.Logging.Main.Request == "auto" { + cfg.Logging.Main.Request = "true" + } + default: + return fmt.Errorf(cfg.Error.Messages.InvalidParam, "logging.main.format", cfg.Logging.Main.Format) + } + + var attr []string + + if cfg.Logging.Main.Request == "true" || cfg.Logging.Main.Request == "1" { + attr = slices.Concat(attr, []string{ + "uri", + "path", + "query", + "proto", + "ip", + "method", + "host", + "elapsed", + "user", + }) + } + + attr = slices.Concat(attr, cfg.Logging.Main.Headers) + + logHandler = slogContextHandler{logHandler, attr} + + if cfg.Telemetry.Enabled { + otelHandler := otelslog.NewHandler(static.GetPackage()) + logHandler = MultiHandler{[]slog.Handler{logHandler, otelHandler}} + } + + slog.SetDefault(slog.New(logHandler)) + + return nil +} + +func configureAccessLogging(cfg config.AccessConfig, errorMessages config.ErrorMessages, rootHandler http.Handler) (http.Handler, error) { + if cfg.Console || len(cfg.Path) > 0 { + var out io.Writer + var err error + if len(cfg.Path) > 0 { + out, err = makeLogFileWriter(cfg.Path, cfg.Console) + + if err != nil { + return nil, err + } + } else { + out = os.Stdout + } + + switch cfg.Format { + case config.AccessFormatCommon: + rootHandler = handlers.LoggingHandler(out, rootHandler) + case config.AccessFormatCombined: + rootHandler = handlers.CombinedLoggingHandler(out, rootHandler) + default: + return nil, fmt.Errorf(errorMessages.InvalidParam, "logging.access.format", cfg.Format) + } + } + return rootHandler, nil +} diff --git a/internal/server/redirect_handler.go b/internal/server/redirect_handler.go new file mode 100644 index 00000000..6c7855a8 --- /dev/null +++ b/internal/server/redirect_handler.go @@ -0,0 +1,25 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package server + +import "net/http" + +type httpRedirectHandler struct { + protoAndHost string +} + +func (h httpRedirectHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { + http.Redirect(w, req, h.protoAndHost+req.RequestURI, http.StatusMovedPermanently) +} diff --git a/internal/server/server.go b/internal/server/server.go index bdb53831..92c41904 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -18,26 +18,20 @@ import ( "context" "errors" "fmt" - "io" "log/slog" "net" "net/http" "os" "os/signal" "runtime/debug" - "slices" "strconv" - "strings" "time" - "github.com/Michad/tilegroxy/internal/images" "github.com/Michad/tilegroxy/pkg" "github.com/Michad/tilegroxy/pkg/config" "github.com/Michad/tilegroxy/pkg/entities/authentication" "github.com/Michad/tilegroxy/pkg/entities/layer" - "github.com/Michad/tilegroxy/pkg/static" - "go.opentelemetry.io/contrib/bridges/otelslog" "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp" "golang.org/x/crypto/acme/autocert" @@ -52,244 +46,107 @@ func handleNoContent(w http.ResponseWriter, _ *http.Request) { // not useful in practice due to OS-specific nature of signals var InterruptFlags = []os.Signal{os.Interrupt} -func errorVars(cfg *config.ErrorConfig, errorType pkg.TypeOfError) (int, slog.Level, string) { - var status int - var level slog.Level - var imgPath string - - switch errorType { - case pkg.TypeOfErrorAuth: - level = slog.LevelDebug - status = http.StatusUnauthorized - imgPath = cfg.Images.Authentication - case pkg.TypeOfErrorBounds: - level = slog.LevelDebug - status = http.StatusBadRequest - imgPath = cfg.Images.OutOfBounds - case pkg.TypeOfErrorProvider: - level = slog.LevelInfo - status = http.StatusInternalServerError - imgPath = cfg.Images.Provider - case pkg.TypeOfErrorBadRequest: - level = slog.LevelDebug - status = http.StatusBadRequest - imgPath = cfg.Images.Other - default: - level = slog.LevelWarn - status = http.StatusInternalServerError - imgPath = cfg.Images.Other - } - if cfg.AlwaysOK { - status = http.StatusOK - } +func setupHandlers(config *config.Config, layerGroup *layer.LayerGroup, auth authentication.Authentication) (http.Handler, error) { + r := http.ServeMux{} - return status, level, imgPath -} + var myRootHandler http.Handler + var myTileHandler http.Handler + var myDocumentationHandler http.Handler + defaultHandler := defaultHandler{config: config, auth: auth, layerGroup: layerGroup} -func writeError(ctx context.Context, w http.ResponseWriter, cfg *config.ErrorConfig, err error) { - var te pkg.TypedError - if errors.As(err, &te) { - writeErrorMessage(ctx, w, cfg, te.Type(), te.Error(), te.External(cfg.Messages), debug.Stack()) + if config.Server.Production { + myRootHandler = http.HandlerFunc(handleNoContent) } else { - writeErrorMessage(ctx, w, cfg, pkg.TypeOfErrorOther, err.Error(), fmt.Sprintf(cfg.Messages.ServerError, err), debug.Stack()) - } -} - -func writeErrorMessage(ctx context.Context, w http.ResponseWriter, cfg *config.ErrorConfig, errorType pkg.TypeOfError, internalMessage string, externalMessage string, stack []byte) { - status, level, imgPath := errorVars(cfg, errorType) - - slog.Log(ctx, level, internalMessage, "stack", string(stack)) - - switch cfg.Mode { - case config.ModeErrorPlainText: - w.WriteHeader(status) - _, err := w.Write([]byte(externalMessage)) - if err != nil { - slog.WarnContext(ctx, fmt.Sprintf("error writing error %v", err)) - } - case config.ModeErrorImage, config.ModeErrorImageHeader: - if cfg.Mode == config.ModeErrorImageHeader { - w.Header().Add("X-Error-Message", externalMessage) - } - w.WriteHeader(status) - - img, err2 := images.GetStaticImage(imgPath) - if img != nil && err2 == nil { - _, err2 = w.Write(*img) - } + myRootHandler = &defaultHandler - if err2 != nil { - slog.ErrorContext(ctx, err2.Error()) + if config.Server.DocsPath != "" { + myDocumentationHandler = &documentationHandler{defaultHandler} } - default: - w.WriteHeader(status) } -} - -type slogContextHandler struct { - slog.Handler - keys []string -} - -func (h slogContextHandler) Handle(ctx context.Context, r slog.Record) error { - for _, k := range h.keys { - r.AddAttrs(slog.Attr{Key: strings.ToLower(k), Value: slog.AnyValue(ctx.Value(k))}) - } - - return h.Handler.Handle(ctx, r) -} - -func makeLogFileWriter(path string, alsoStdOut bool) (io.Writer, error) { - logFile, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666) + tilePath := config.Server.RootPath + config.Server.TilePath + "/{layer}/{z}/{x}/{y}" + docsPath := config.Server.RootPath + config.Server.DocsPath + "/{path...}" + handler, err := newTileHandler(defaultHandler) if err != nil { return nil, err } - var out io.Writer - if alsoStdOut { - out = io.MultiWriter(os.Stdout, logFile) - } else { - out = logFile - } - return out, nil -} + myTileHandler = &handler -func configureMainLogging(cfg *config.Config) error { - if !cfg.Logging.Main.Console && len(cfg.Logging.Main.Path) == 0 { - slog.SetLogLoggerLevel(slog.LevelError + 1) - return nil - } + if config.Telemetry.Enabled { + myRootHandler = otelhttp.NewHandler(myRootHandler, config.Server.RootPath, otelhttp.WithMessageEvents(otelhttp.WriteEvents)) + myTileHandler = otelhttp.NewHandler(myTileHandler, tilePath, otelhttp.WithMessageEvents(otelhttp.WriteEvents)) - var err error - var out io.Writer - if len(cfg.Logging.Main.Path) > 0 { - out, err = makeLogFileWriter(cfg.Logging.Main.Path, cfg.Logging.Main.Console) - if err != nil { - return err + if myDocumentationHandler != nil { + myDocumentationHandler = otelhttp.NewHandler(myDocumentationHandler, docsPath, otelhttp.WithMessageEvents(otelhttp.WriteEvents)) } - } else { - out = os.Stdout } - var level slog.Level - custLogLevel, ok := config.CustomLogLevel[strings.ToLower(cfg.Logging.Main.Level)] - - if ok { - level = custLogLevel - } else { - err := level.UnmarshalText([]byte(cfg.Logging.Main.Level)) - - if err != nil { - return err - } - } + r.Handle(config.Server.RootPath, myRootHandler) + r.Handle(tilePath, myTileHandler) + r.Handle(tilePath+"/", myTileHandler) - opt := slog.HandlerOptions{ - AddSource: true, - Level: level, - ReplaceAttr: func(groups []string, a slog.Attr) slog.Attr { - if groups == nil && a.Key == "msg" { - return slog.Attr{Key: "message", Value: a.Value} - } - return a - }, + if myDocumentationHandler != nil { + r.Handle(docsPath, myDocumentationHandler) } - var logHandler slog.Handler + var rootHandler http.Handler - switch cfg.Logging.Main.Format { - case config.MainFormatPlain: - logHandler = slog.NewTextHandler(out, &opt) - case config.MainFormatJSON: - logHandler = slog.NewJSONHandler(out, &opt) - if cfg.Logging.Main.Request == "auto" { - cfg.Logging.Main.Request = "true" - } - default: - return fmt.Errorf(cfg.Error.Messages.InvalidParam, "logging.main.format", cfg.Logging.Main.Format) - } + rootHandler = &r - var attr []string - - if cfg.Logging.Main.Request == "true" || cfg.Logging.Main.Request == "1" { - attr = slices.Concat(attr, []string{ - "uri", - "path", - "query", - "proto", - "ip", - "method", - "host", - "elapsed", - "user", - }) + if config.Server.Gzip { + rootHandler = handlers.CompressHandler(rootHandler) } - attr = slices.Concat(attr, cfg.Logging.Main.Headers) - - logHandler = slogContextHandler{logHandler, attr} + rootHandler = httpContextHandler{rootHandler, config.Error} + rootHandler = http.TimeoutHandler(rootHandler, time.Duration(config.Server.Timeout)*time.Second, config.Error.Messages.Timeout) + rootHandler, err = configureAccessLogging(config.Logging.Access, config.Error.Messages, rootHandler) - if cfg.Telemetry.Enabled { - otelHandler := otelslog.NewHandler(static.GetPackage()) - logHandler = MultiHandler{[]slog.Handler{logHandler, otelHandler}} + if err != nil { + return nil, err } - slog.SetDefault(slog.New(logHandler)) - - return nil + return rootHandler, nil } -func configureAccessLogging(cfg config.AccessConfig, errorMessages config.ErrorMessages, rootHandler http.Handler) (http.Handler, error) { - if cfg.Console || len(cfg.Path) > 0 { - var out io.Writer - var err error - if len(cfg.Path) > 0 { - out, err = makeLogFileWriter(cfg.Path, cfg.Console) +func listenAndServeTLS(config *config.Config, srvErr chan error, srv *http.Server) { + httpPort := config.Server.Encrypt.HTTPPort + httpHostPort := net.JoinHostPort(config.Server.BindHost, strconv.Itoa(httpPort)) - if err != nil { - return nil, err - } - } else { - out = os.Stdout + if config.Server.Encrypt.Certificate != "" && config.Server.Encrypt.KeyFile != "" { + if httpPort != 0 { + go func() { + srvErr <- http.ListenAndServe(httpHostPort, httpRedirectHandler{protoAndHost: "https://" + config.Server.Encrypt.Domain}) + }() } - switch cfg.Format { - case config.AccessFormatCommon: - rootHandler = handlers.LoggingHandler(out, rootHandler) - case config.AccessFormatCombined: - rootHandler = handlers.CombinedLoggingHandler(out, rootHandler) - default: - return nil, fmt.Errorf(errorMessages.InvalidParam, "logging.access.format", cfg.Format) + srvErr <- srv.ListenAndServeTLS(config.Server.Encrypt.Certificate, config.Server.Encrypt.KeyFile) + } else { + // Let's Encrypt workflow + + cacheDir := "certs" + if config.Server.Encrypt.Cache != "" { + cacheDir = config.Server.Encrypt.Cache } - } - return rootHandler, nil -} -type httpContextHandler struct { - http.Handler - errCfg config.ErrorConfig -} + certManager := autocert.Manager{ + Prompt: autocert.AcceptTOS, + HostPolicy: autocert.HostWhitelist(config.Server.Encrypt.Domain), + Cache: autocert.DirCache(cacheDir), + } -func (h httpContextHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { - reqContext := pkg.NewRequestContext(req) - defer func() { - if err := recover(); err != nil { - writeErrorMessage(reqContext, w, &h.errCfg, pkg.TypeOfErrorOther, fmt.Sprint(err), "Unexpected Internal Server Error", debug.Stack()) + if httpPort != 0 { + go func() { srvErr <- http.ListenAndServe(httpHostPort, certManager.HTTPHandler(nil)) }() } - }() - h.Handler.ServeHTTP(w, req.WithContext(reqContext)) -} + srv.TLSConfig = certManager.TLSConfig() -type httpRedirectHandler struct { - protoAndHost string + srvErr <- srv.ListenAndServeTLS("", "") + } } -func (h httpRedirectHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { - http.Redirect(w, req, h.protoAndHost+req.RequestURI, http.StatusMovedPermanently) -} + func ListenAndServe(config *config.Config, layerGroup *layer.LayerGroup, auth authentication.Authentication) error { if config.Server.Encrypt != nil && config.Server.Encrypt.Domain == "" { @@ -358,103 +215,4 @@ func ListenAndServe(config *config.Config, layerGroup *layer.LayerGroup, auth au err = errors.Join(err, otelShutdown(context.Background())) } return err -} - -func setupHandlers(config *config.Config, layerGroup *layer.LayerGroup, auth authentication.Authentication) (http.Handler, error) { - r := http.ServeMux{} - - var myRootHandler http.Handler - var myTileHandler http.Handler - var myDocumentationHandler http.Handler - defaultHandler := defaultHandler{config: config, auth: auth, layerGroup: layerGroup} - - if config.Server.Production { - myRootHandler = http.HandlerFunc(handleNoContent) - } else { - myRootHandler = &defaultHandler - - if config.Server.DocsPath != "" { - myDocumentationHandler = &documentationHandler{defaultHandler} - } - } - - tilePath := config.Server.RootPath + config.Server.TilePath + "/{layer}/{z}/{x}/{y}" - docsPath := config.Server.RootPath + config.Server.DocsPath + "/{path...}" - handler, err := newTileHandler(defaultHandler) - if err != nil { - return nil, err - } - - myTileHandler = &handler - - if config.Telemetry.Enabled { - myRootHandler = otelhttp.NewHandler(myRootHandler, config.Server.RootPath, otelhttp.WithMessageEvents(otelhttp.WriteEvents)) - myTileHandler = otelhttp.NewHandler(myTileHandler, tilePath, otelhttp.WithMessageEvents(otelhttp.WriteEvents)) - - if myDocumentationHandler != nil { - myDocumentationHandler = otelhttp.NewHandler(myDocumentationHandler, docsPath, otelhttp.WithMessageEvents(otelhttp.WriteEvents)) - } - } - - r.Handle(config.Server.RootPath, myRootHandler) - r.Handle(tilePath, myTileHandler) - r.Handle(tilePath+"/", myTileHandler) - - if myDocumentationHandler != nil { - r.Handle(docsPath, myDocumentationHandler) - } - - var rootHandler http.Handler - - rootHandler = &r - - if config.Server.Gzip { - rootHandler = handlers.CompressHandler(rootHandler) - } - - rootHandler = httpContextHandler{rootHandler, config.Error} - rootHandler = http.TimeoutHandler(rootHandler, time.Duration(config.Server.Timeout)*time.Second, config.Error.Messages.Timeout) - rootHandler, err = configureAccessLogging(config.Logging.Access, config.Error.Messages, rootHandler) - - if err != nil { - return nil, err - } - - return rootHandler, nil -} - -func listenAndServeTLS(config *config.Config, srvErr chan error, srv *http.Server) { - httpPort := config.Server.Encrypt.HTTPPort - httpHostPort := net.JoinHostPort(config.Server.BindHost, strconv.Itoa(httpPort)) - - if config.Server.Encrypt.Certificate != "" && config.Server.Encrypt.KeyFile != "" { - if httpPort != 0 { - go func() { - srvErr <- http.ListenAndServe(httpHostPort, httpRedirectHandler{protoAndHost: "https://" + config.Server.Encrypt.Domain}) - }() - } - - srvErr <- srv.ListenAndServeTLS(config.Server.Encrypt.Certificate, config.Server.Encrypt.KeyFile) - } else { - // Let's Encrypt workflow - - cacheDir := "certs" - if config.Server.Encrypt.Cache != "" { - cacheDir = config.Server.Encrypt.Cache - } - - certManager := autocert.Manager{ - Prompt: autocert.AcceptTOS, - HostPolicy: autocert.HostWhitelist(config.Server.Encrypt.Domain), - Cache: autocert.DirCache(cacheDir), - } - - if httpPort != 0 { - go func() { srvErr <- http.ListenAndServe(httpHostPort, certManager.HTTPHandler(nil)) }() - } - - srv.TLSConfig = certManager.TLSConfig() - - srvErr <- srv.ListenAndServeTLS("", "") - } -} +} \ No newline at end of file From f80690fdcfd516750a41dae20d6758f30b3a9074 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Sun, 8 Sep 2024 22:58:50 -0400 Subject: [PATCH 05/11] refactor: move fail provider out of test scope so it can be used in other package's tests --- internal/providers/{fail_test.go => fail.go} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename internal/providers/{fail_test.go => fail.go} (100%) diff --git a/internal/providers/fail_test.go b/internal/providers/fail.go similarity index 100% rename from internal/providers/fail_test.go rename to internal/providers/fail.go From 02937cfe47d9064acd19e92d7b84c9a3d3b5b111 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Sun, 8 Sep 2024 23:00:02 -0400 Subject: [PATCH 06/11] feat: add new entity type for health checks and implement initial 2 checks --- internal/checks/cache.go | 109 +++++++++++++++++ internal/checks/cache_test.go | 66 +++++++++++ internal/checks/tile.go | 185 +++++++++++++++++++++++++++++ internal/checks/tile_test.go | 213 ++++++++++++++++++++++++++++++++++ pkg/entities/health/check.go | 81 +++++++++++++ 5 files changed, 654 insertions(+) create mode 100644 internal/checks/cache.go create mode 100644 internal/checks/cache_test.go create mode 100644 internal/checks/tile.go create mode 100644 internal/checks/tile_test.go create mode 100644 pkg/entities/health/check.go diff --git a/internal/checks/cache.go b/internal/checks/cache.go new file mode 100644 index 00000000..b7364e69 --- /dev/null +++ b/internal/checks/cache.go @@ -0,0 +1,109 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package checks + +import ( + "context" + "errors" + "math/rand/v2" + "slices" + "strconv" + + "github.com/Michad/tilegroxy/internal/images" + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/Michad/tilegroxy/pkg/entities/cache" + "github.com/Michad/tilegroxy/pkg/entities/health" + "github.com/Michad/tilegroxy/pkg/entities/layer" +) + +var cacheReq = pkg.TileRequest{LayerName: "___hc___", Z: 0, X: 0, Y: 0} + +type CacheCheckConfig struct { + Delay uint +} + +func (s CacheCheckConfig) GetDelay() uint { + return s.Delay +} + +type CacheCheck struct { + CacheCheckConfig + cache cache.Cache + errorMessages config.ErrorMessages +} + +func init() { + health.RegisterHealthCheck(CacheCheckRegistration{}) +} + +type CacheCheckRegistration struct { +} + +func (s CacheCheckRegistration) InitializeConfig() health.HealthCheckConfig { + return CacheCheckConfig{} +} + +func (s CacheCheckRegistration) Name() string { + return "tile" +} + +func (s CacheCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, lg *layer.LayerGroup, cache cache.Cache, allCfg *config.Config) (health.HealthCheck, error) { + cfg := checkConfig.(CacheCheckConfig) + + if cfg.Delay == 0 { + cfg.Delay = 600 + } + + return &CacheCheck{cfg, cache, allCfg.Error.Messages}, nil +} + +func makeImage() (pkg.Image, error) { + col := strconv.FormatUint(rand.Uint64N(0xFFFFFF), 16) + + img, err := images.GetStaticImage("color:" + col) + + if err != nil { + return pkg.Image{}, err + } + + return pkg.Image{Content: *img}, nil +} + +func (h CacheCheck) Check(ctx context.Context) error { + img, err := makeImage() + + if err != nil { + return err + } + + err = h.cache.Save(ctx, cacheReq, &img) + + if err != nil { + return err + } + + img2, err := h.cache.Lookup(ctx, cacheReq) + + if err != nil { + return err + } + + if img2 == nil || !slices.Equal(img.Content, img2.Content) { + return errors.New("cache returned wrong result") + } + + return nil +} diff --git a/internal/checks/cache_test.go b/internal/checks/cache_test.go new file mode 100644 index 00000000..5add9028 --- /dev/null +++ b/internal/checks/cache_test.go @@ -0,0 +1,66 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package checks + +import ( + "context" + "testing" + + "github.com/Michad/tilegroxy/internal/caches" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func Test_Fail(t *testing.T) { + cfgAll := config.DefaultConfig() + msg := cfgAll.Error.Messages + + cacheReg := caches.NoopRegistration{} + cacheCfg := cacheReg.InitializeConfig() + cache, err := cacheReg.Initialize(cacheCfg, msg) + require.NoError(t, err) + + reg := CacheCheckRegistration{} + cfgAny := reg.InitializeConfig() + hc, err := reg.Initialize(cfgAny, nil, cache, &cfgAll) + require.NoError(t, err) + + err = hc.Check(context.Background()) + assert.Error(t, err) +} + +func Test_Works(t *testing.T) { + cfgAll := config.DefaultConfig() + msg := cfgAll.Error.Messages + + cacheReg := caches.MemoryRegistration{} + cacheCfg := cacheReg.InitializeConfig() + cache, err := cacheReg.Initialize(cacheCfg, msg) + require.NoError(t, err) + + reg := CacheCheckRegistration{} + cfgAny := reg.InitializeConfig() + hc, err := reg.Initialize(cfgAny, nil, cache, &cfgAll) + require.NoError(t, err) + + require.IsType(t, &CacheCheck{}, hc) + cc := hc.(*CacheCheck) + assert.Equal(t, uint(600), cc.Delay) + assert.Equal(t, cc.Delay, cc.GetDelay()) + + err = hc.Check(context.Background()) + assert.NoError(t, err) +} diff --git a/internal/checks/tile.go b/internal/checks/tile.go new file mode 100644 index 00000000..98375d59 --- /dev/null +++ b/internal/checks/tile.go @@ -0,0 +1,185 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package checks + +import ( + "context" + "encoding/base64" + "errors" + "fmt" + "os" + "slices" + + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/Michad/tilegroxy/pkg/entities/cache" + "github.com/Michad/tilegroxy/pkg/entities/health" + "github.com/Michad/tilegroxy/pkg/entities/layer" +) + +const ( + ValidationSame = "same" + ValidationContentType = "content-tye" + ValidationBase64 = "base-64" + ValidationFile = "file" + ValidationSuccess = "success" + DefaultX = 123 + DefaultY = 534 + DefaultZ = 10 +) + +var AllValidationModes = []string{ValidationSame, ValidationContentType, ValidationBase64, ValidationFile, ValidationSuccess} + +type TileCheckConfig struct { + Delay uint + Layer string + Z int + Y int + X int + Validation string + Result string +} + +func (s TileCheckConfig) GetDelay() uint { + return s.Delay +} + +type TileCheck struct { + TileCheckConfig + lg *layer.LayerGroup + errorMessages config.ErrorMessages + req pkg.TileRequest + img *pkg.Image +} + +func init() { + health.RegisterHealthCheck(TileCheckRegistration{}) +} + +type TileCheckRegistration struct { +} + +func (s TileCheckRegistration) InitializeConfig() health.HealthCheckConfig { + return TileCheckConfig{} +} + +func (s TileCheckRegistration) Name() string { + return "tile" +} + +func (s TileCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, lg *layer.LayerGroup, cache cache.Cache, allCfg *config.Config) (health.HealthCheck, error) { + cfg := checkConfig.(TileCheckConfig) + + if cfg.Delay == 0 { + cfg.Delay = 60 + } + if cfg.Z == 0 { + cfg.Z = DefaultZ + } + if cfg.X == 0 { + cfg.X = DefaultX + } + if cfg.Y == 0 { + cfg.Y = DefaultY + } + + if cfg.Validation == "" { + cfg.Validation = ValidationSame + } + + if !slices.Contains(AllValidationModes, cfg.Validation) { + return nil, fmt.Errorf(allCfg.Error.Messages.EnumError, "check.validation", cfg.Validation, AllValidationModes) + } + + if lg.FindLayer(pkg.BackgroundContext(), cfg.Layer) == nil { + return nil, fmt.Errorf(allCfg.Error.Messages.EnumError, "check.layer", cfg.Layer, lg.ListLayerIDs()) + } + + req := pkg.TileRequest{LayerName: cfg.Layer, Z: cfg.Z, X: cfg.X, Y: cfg.Y} + + _, err := req.GetBounds() + + if err != nil { + return nil, err + } + + return &TileCheck{cfg, lg, allCfg.Error.Messages, req, nil}, nil +} + +func (h *TileCheck) Check(ctx context.Context) error { + img, err := h.lg.RenderTileNoCache(ctx, h.req) + + if err != nil { + return err + } + + switch h.Validation { + case ValidationSame: + return h.ValidateSame(ctx, img) + case ValidationContentType: + return h.ValidateContentType(ctx, img) + case ValidationBase64: + return h.ValidateBase64(ctx, img) + case ValidationFile: + return h.ValidateFile(ctx, img) + } + + return nil +} + +func (h *TileCheck) ValidateSame(ctx context.Context, img *pkg.Image) error { + if h.img == nil { + h.img = img + return nil + } + + if img.ContentType != h.img.ContentType || !slices.Equal(img.Content, h.img.Content) { + h.img = img + return errors.New("result changed") + } + + return nil +} + +func (h *TileCheck) ValidateContentType(ctx context.Context, img *pkg.Image) error { + if img.ContentType != h.Result { + return fmt.Errorf(h.errorMessages.InvalidParam, "content type", img.ContentType) + } + + return nil +} + +func (h *TileCheck) ValidateBase64(ctx context.Context, img *pkg.Image) error { + imgEncode := base64.StdEncoding.EncodeToString(img.Content) + if imgEncode != h.Result { + return fmt.Errorf(h.errorMessages.InvalidParam, "content", imgEncode) + } + + return nil +} + +func (h *TileCheck) ValidateFile(ctx context.Context, img *pkg.Image) error { + expected, err := os.ReadFile(h.Result) + + if err != nil { + return err + } + + if !slices.Equal(expected, img.Content) { + return errors.New("result changed") + } + + return nil +} diff --git a/internal/checks/tile_test.go b/internal/checks/tile_test.go new file mode 100644 index 00000000..6821eec1 --- /dev/null +++ b/internal/checks/tile_test.go @@ -0,0 +1,213 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package checks + +import ( + "os" + "testing" + + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/Michad/tilegroxy/pkg/entities/layer" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/Michad/tilegroxy/internal/images" + _ "github.com/Michad/tilegroxy/internal/providers" +) + +func Test_Validate(t *testing.T) { + cfgAll, lg, reg, cfg := initialize(t, false) + cfg.Layer = "fake" + + _, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.Error(t, err) + + cfg.Layer = "test" + + hc, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.NoError(t, err) + require.IsType(t, &TileCheck{}, hc) + + tc := hc.(*TileCheck) + assert.Equal(t, uint(60), tc.Delay) + assert.Equal(t, tc.Delay, tc.GetDelay()) + assert.Equal(t, DefaultZ, tc.Z) + assert.Equal(t, DefaultX, tc.X) + assert.Equal(t, DefaultY, tc.Y) + + cfg.Validation = "fake" + + _, err = reg.Initialize(cfg, lg, nil, &cfgAll) + require.Error(t, err) +} + +func Test_Base64(t *testing.T) { + ctx := pkg.BackgroundContext() + cfgAll, lg, reg, cfg := initialize(t, false) + cfg.Layer = "test" + cfg.Validation = ValidationBase64 + cfg.Result = "iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAIAAAB7GkOtAAAHIklEQVR4nOzVMREAIAzAQI7Dv+Uio0P+FWTLm5kDQM/dDgBghwEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBAlAEARBkAQJQBAEQZAECUAQBEGQBA1A8AAP//gX0HANL7JAoAAAAASUVORK5CYII=" + + hc, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.NoError(t, err) + require.IsType(t, &TileCheck{}, hc) + tc := hc.(*TileCheck) + + err = hc.Check(ctx) + assert.NoError(t, err) + + tc.Result = "test" + err = hc.Check(ctx) + assert.Error(t, err) +} + +func Test_File(t *testing.T) { + ctx := pkg.BackgroundContext() + dir := t.TempDir() + file := dir + "/" + "tile_test_result.png" + fileVal1, err := images.GetStaticImage("color:FFFFFF") + require.NoError(t, err) + err = os.WriteFile(file, *fileVal1, 0600) + require.NoError(t, err) + + cfgAll, lg, reg, cfg := initialize(t, false) + cfg.Layer = "test" + cfg.Validation = ValidationFile + cfg.Result = file + + hc, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.NoError(t, err) + + err = hc.Check(ctx) + assert.NoError(t, err) + + fileVal2, err := images.GetStaticImage("color:000") + require.NoError(t, err) + err = os.WriteFile(file, *fileVal2, 0600) + require.NoError(t, err) + + err = hc.Check(ctx) + assert.Error(t, err) +} + +func Test_Success(t *testing.T) { + ctx := pkg.BackgroundContext() + cfgAll, lg, reg, cfg := initialize(t, false) + cfg.Layer = "test" + cfg.Validation = ValidationSuccess + + hc, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.NoError(t, err) + require.IsType(t, &TileCheck{}, hc) + + err = hc.Check(ctx) + assert.NoError(t, err) +} + +func Test_SuccessFail(t *testing.T) { + ctx := pkg.BackgroundContext() + cfgAll, lg, reg, cfg := initialize(t, true) + cfg.Layer = "test" + cfg.Validation = ValidationSuccess + + hc, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.NoError(t, err) + require.IsType(t, &TileCheck{}, hc) + + err = hc.Check(ctx) + assert.Error(t, err) +} + +func Test_ContentType(t *testing.T) { + ctx := pkg.BackgroundContext() + cfgAll, lg, reg, cfg := initialize(t, false) + cfg.Layer = "test" + cfg.Validation = ValidationContentType + cfg.Result = "image/png" + + hc, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.NoError(t, err) + require.IsType(t, &TileCheck{}, hc) + tc := hc.(*TileCheck) + + err = hc.Check(ctx) + assert.NoError(t, err) + + tc.Result = "test" + err = hc.Check(ctx) + assert.Error(t, err) +} + +func Test_Same(t *testing.T) { + ctx := pkg.BackgroundContext() + cfgAll, lg, reg, cfg := initialize(t, false) + cfg.Layer = "test" + cfg.Validation = ValidationSame + + hc, err := reg.Initialize(cfg, lg, nil, &cfgAll) + require.NoError(t, err) + require.IsType(t, &TileCheck{}, hc) + tc := hc.(*TileCheck) + + err = hc.Check(ctx) + assert.NoError(t, err) + + err = hc.Check(ctx) + assert.NoError(t, err) + + tc.img = &pkg.Image{} + + err = hc.Check(ctx) + assert.Error(t, err) + + err = hc.Check(ctx) + assert.NoError(t, err) +} + +func initialize(t *testing.T, fail bool) (config.Config, *layer.LayerGroup, TileCheckRegistration, TileCheckConfig) { + cfgAll := config.DefaultConfig() + + var layerCfg config.LayerConfig + + if fail { + layerCfg = config.LayerConfig{ + ID: "test", + Provider: map[string]any{ + "name": "fail", + "onauth": true, + }, + } + } else { + layerCfg = config.LayerConfig{ + ID: "test", + Provider: map[string]any{ + "name": "static", + "color": "FFFFFF", + }, + } + } + + cfgAll.Layers = append(cfgAll.Layers, layerCfg) + lg, err := layer.ConstructLayerGroup(cfgAll, nil, nil) + require.NoError(t, err) + + reg := TileCheckRegistration{} + cfgAny := reg.InitializeConfig() + + require.IsType(t, TileCheckConfig{}, cfgAny) + cfg := cfgAny.(TileCheckConfig) + return cfgAll, lg, reg, cfg +} diff --git a/pkg/entities/health/check.go b/pkg/entities/health/check.go new file mode 100644 index 00000000..642cb4d4 --- /dev/null +++ b/pkg/entities/health/check.go @@ -0,0 +1,81 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package health + +import ( + "context" + "fmt" + + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/Michad/tilegroxy/pkg/entities/cache" + "github.com/Michad/tilegroxy/pkg/entities/layer" + "github.com/mitchellh/mapstructure" +) + +type HealthCheck interface { + Check(ctx context.Context) error + GetDelay() uint +} + +type HealthCheckConfig interface { + GetDelay() uint +} + +type HealthCheckRegistration interface { + Name() string + Initialize(checkConfig HealthCheckConfig, lg *layer.LayerGroup, cache cache.Cache, allCfg *config.Config) (HealthCheck, error) + InitializeConfig() HealthCheckConfig +} + +var registrations = make(map[string]HealthCheckRegistration) + +func RegisterHealthCheck(reg HealthCheckRegistration) { + registrations[reg.Name()] = reg +} + +func RegisteredHealthCheck(name string) (HealthCheckRegistration, bool) { + o, ok := registrations[name] + return o, ok +} + +func RegisteredHealthCheckNames() []string { + names := make([]string, 0, len(registrations)) + for n := range registrations { + names = append(names, n) + } + return names +} + +func ConstructHealthCheck(rawConfig map[string]interface{}, lg *layer.LayerGroup, allCfg *config.Config) (HealthCheck, error) { + rawConfig = pkg.ReplaceEnv(rawConfig) + + name, ok := rawConfig["name"].(string) + + if ok { + reg, ok := RegisteredHealthCheck(name) + if ok { + cfg := reg.InitializeConfig() + err := mapstructure.Decode(rawConfig, &cfg) + if err != nil { + return nil, err + } + return reg.Initialize(cfg, lg, lg.DefaultCache, allCfg) + } + } + + nameCoerce := fmt.Sprintf("%#v", rawConfig["name"]) + return nil, fmt.Errorf(allCfg.Error.Messages.EnumError, "check.name", nameCoerce, RegisteredHealthCheckNames()) +} From 21b566c140ab0f20e2e6b59577b7e58e1938b5d7 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Mon, 9 Sep 2024 01:06:39 -0400 Subject: [PATCH 07/11] feat: add endpoint for retrieving health information --- internal/checks/cache.go | 6 +- internal/server/health.go | 258 +++++++++++++++++++++++++++++++ internal/server/server.go | 22 ++- pkg/config/config.go | 16 +- pkg/entities/layer/layer.go | 4 +- pkg/entities/layer/layergroup.go | 2 + 6 files changed, 300 insertions(+), 8 deletions(-) create mode 100644 internal/server/health.go diff --git a/internal/checks/cache.go b/internal/checks/cache.go index b7364e69..47201f0c 100644 --- a/internal/checks/cache.go +++ b/internal/checks/cache.go @@ -20,6 +20,7 @@ import ( "math/rand/v2" "slices" "strconv" + "strings" "github.com/Michad/tilegroxy/internal/images" "github.com/Michad/tilegroxy/pkg" @@ -57,7 +58,7 @@ func (s CacheCheckRegistration) InitializeConfig() health.HealthCheckConfig { } func (s CacheCheckRegistration) Name() string { - return "tile" + return "cache" } func (s CacheCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, lg *layer.LayerGroup, cache cache.Cache, allCfg *config.Config) (health.HealthCheck, error) { @@ -72,6 +73,9 @@ func (s CacheCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, func makeImage() (pkg.Image, error) { col := strconv.FormatUint(rand.Uint64N(0xFFFFFF), 16) + if len(col) < 6 { + col = strings.Repeat("0", 6-len(col)) + col + } img, err := images.GetStaticImage("color:" + col) diff --git a/internal/server/health.go b/internal/server/health.go new file mode 100644 index 00000000..eaba056f --- /dev/null +++ b/internal/server/health.go @@ -0,0 +1,258 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package server + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "log/slog" + "net" + "net/http" + "reflect" + "runtime/debug" + "strconv" + "sync" + "time" + + _ "github.com/Michad/tilegroxy/internal/checks" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/Michad/tilegroxy/pkg/entities/health" + "github.com/Michad/tilegroxy/pkg/entities/layer" + "github.com/Michad/tilegroxy/pkg/static" +) + +type healthHandler struct { + checks []health.HealthCheck + checkResultCache *sync.Map +} + +func (h healthHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { + ctx := req.Context() + slog.DebugContext(ctx, "server: health handler started") + defer slog.DebugContext(ctx, "server: health handler ended") + + if req.Method != http.MethodGet { + w.WriteHeader(http.StatusNoContent) + return + } + + body := make(map[string]any) + checks := make(map[string]any) + details := make([]map[string]any, 0, len(h.checks)) + body["checks"] = checks + isOk := true + + if h.checkResultCache != nil { + for i, check := range h.checks { + detail := make(map[string]any) + details = append(details, detail) + + detail["componentId"] = strconv.Itoa(i) + + var checkType string + if t := reflect.TypeOf(check); t.Kind() == reflect.Ptr { + checkType = t.Elem().Name() + } else { + checkType = t.Name() + } + + detail["componentType"] = checkType + + result, ok := h.checkResultCache.Load(i) + resultCheck, ok2 := result.(CheckResult) + + if !ok || !ok2 { + isOk = false + detail["status"] = "error" + detail["output"] = "Check has not updated stored health value" + } else { + if resultCheck.err != nil { + isOk = false + detail["status"] = "error" + detail["output"] = resultCheck.err.Error() + } else { + // Include 5 second leeway for check not being performed instantly + if resultCheck.timestamp.Add(resultCheck.ttl).Add(5 * time.Second).Before(time.Now()) { + isOk = false + detail["status"] = "error" + detail["output"] = "stale" + } else { + detail["status"] = "ok" + } + } + detail["time"] = resultCheck.timestamp.Format(time.RFC3339) + detail["ttl"] = resultCheck.ttl / time.Second + } + } + } else { + isOk = false + body["output"] = "missing check cache" + } + + checks["tilegroxy:checks"] = details + + if isOk { + body["status"] = "ok" + } else { + body["status"] = "error" + } + + version, gitRef, _ := static.GetVersionInformation() + body["version"] = version + body["releaseId"] = gitRef + + data, err := json.Marshal(body) + + if err != nil { + slog.ErrorContext(ctx, "Unable to write health", "error", err, "stack", string(debug.Stack())) + isOk = false + } + + w.Header().Add("Content-Type", "application/json+health") + + if isOk { + w.WriteHeader(http.StatusOK) + } else { + w.WriteHeader(http.StatusInternalServerError) + } + _, err = w.Write(data) + + if err != nil { + slog.WarnContext(ctx, fmt.Sprintf("Unable to write to documentation request due to %v", err)) + } +} + +type CheckResult struct { + err error + timestamp time.Time + ttl time.Duration +} + +func setupHealth(ctx context.Context, cfg *config.Config, layerGroup *layer.LayerGroup) (func(context.Context) error, error) { + h := cfg.Server.Health + + slog.InfoContext(ctx, fmt.Sprintf("Initializing health subsystem with %v checks on %v:%v", len(h.Checks), h.Host, h.Port)) + + var err error + var callback func(context.Context) error + checkResultCache := sync.Map{} + var checks []health.HealthCheck + + if len(h.Checks) > 0 { + checks, callback, err = setupCheckRoutines(ctx, h, layerGroup, cfg, &checkResultCache) + if err != nil { + return callback, err + } + } + + callback2, err := setupHealthEndpoints(ctx, h, checks, &checkResultCache) + + if callback2 != nil { + if callback != nil { + return func(ctx context.Context) error { + return errors.Join(callback(ctx), callback2(ctx)) + }, err + } else { + return callback2, err + } + } + + return callback, err +} + +func setupHealthEndpoints(ctx context.Context, h config.HealthConfig, checks []health.HealthCheck, checkResultCache *sync.Map) (func(context.Context) error, error) { + srvErr := make(chan error, 1) + httpHostPort := net.JoinHostPort(h.Host, strconv.Itoa(h.Port)) + + r := http.ServeMux{} + r.HandleFunc("/", handleNoContent) + r.Handle("/health", healthHandler{checks, checkResultCache}) + + srv := &http.Server{ + Addr: httpHostPort, + BaseContext: func(_ net.Listener) context.Context { return ctx }, + Handler: &r, + } + + go func() { srvErr <- srv.ListenAndServe() }() + + var err error + + // Give srv a little breathing room to try to start up + select { + case err = <-srvErr: + case <-time.After(100 * time.Millisecond): + } + + return srv.Shutdown, err +} + +func setupCheckRoutines(ctx context.Context, h config.HealthConfig, layerGroup *layer.LayerGroup, cfg *config.Config, checkResultCache *sync.Map) ([]health.HealthCheck, func(context.Context) error, error) { + checks := make([]health.HealthCheck, 0, len(h.Checks)) + var callback func(context.Context) error + tickers := make([]*time.Ticker, 0, len(h.Checks)) + exitChannels := make([]chan bool, 0, len(h.Checks)) + + for _, checkCfg := range h.Checks { + hc, err := health.ConstructHealthCheck(checkCfg, layerGroup, cfg) + if err != nil { + return nil, nil, err + } + checks = append(checks, hc) + } + + for i, check := range checks { + ttl := time.Second * time.Duration(check.GetDelay()) + + ticker := time.NewTicker(ttl) + done := make(chan bool) + tickers = append(tickers, ticker) + exitChannels = append(exitChannels, done) + + go func() { + for { + select { + case <-done: + return + case t := <-ticker.C: + slog.Log(ctx, config.LevelTrace, fmt.Sprintf("Ticking check %v after %v", i, t)) + + err := check.Check(ctx) + + result := CheckResult{err: err, timestamp: time.Now(), ttl: ttl} + + checkResultCache.Store(i, result) + } + } + }() + } + + callback = func(ctx context.Context) error { + slog.InfoContext(ctx, "Terminating health subsystem") + + for _, ticker := range tickers { + ticker.Stop() + } + for _, channel := range exitChannels { + channel <- true + } + + return nil + } + + return checks, callback, nil +} diff --git a/internal/server/server.go b/internal/server/server.go index 92c41904..e05bcefd 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -46,7 +46,6 @@ func handleNoContent(w http.ResponseWriter, _ *http.Request) { // not useful in practice due to OS-specific nature of signals var InterruptFlags = []os.Signal{os.Interrupt} - func setupHandlers(config *config.Config, layerGroup *layer.LayerGroup, auth authentication.Authentication) (http.Handler, error) { r := http.ServeMux{} @@ -146,8 +145,6 @@ func listenAndServeTLS(config *config.Config, srvErr chan error, srv *http.Serve } } - - func ListenAndServe(config *config.Config, layerGroup *layer.LayerGroup, auth authentication.Authentication) error { if config.Server.Encrypt != nil && config.Server.Encrypt.Domain == "" { return fmt.Errorf(config.Error.Messages.ParamRequired, "server.encrypt.domain") @@ -168,6 +165,16 @@ func ListenAndServe(config *config.Config, layerGroup *layer.LayerGroup, auth au ctx, stop := signal.NotifyContext(pkg.BackgroundContext(), InterruptFlags...) defer stop() + var healthShutdown func(context.Context) error + + if config.Server.Health.Enabled { + healthShutdown, err = setupHealth(ctx, config, layerGroup) + + if err != nil { + return err + } + } + var otelShutdown func(context.Context) error if config.Telemetry.Enabled { @@ -211,8 +218,13 @@ func ListenAndServe(config *config.Config, layerGroup *layer.LayerGroup, auth au err = srv.Shutdown(context.Background()) - if config.Telemetry.Enabled { + if otelShutdown != nil { err = errors.Join(err, otelShutdown(context.Background())) } + + if healthShutdown != nil { + err = errors.Join(err, healthShutdown(context.Background())) + } + return err -} \ No newline at end of file +} diff --git a/pkg/config/config.go b/pkg/config/config.go index fcc6c319..06820346 100644 --- a/pkg/config/config.go +++ b/pkg/config/config.go @@ -35,13 +35,22 @@ type EncryptionConfig struct { HTTPPort int // The port used for non-encrypted traffic. Required if using Let's Encrypt for ACME challenge and needs to indirectly be 80 (that is, it could be 8080 if something else redirects 80 to 8080). Everything except .well-known will be redirected to the main port when set. } +// Configuration for health checks +type HealthConfig struct { + Enabled bool // If set to false the port isn't bound to. Defaults false + Port int // The port to serve health on. Defaults to 3000 + Host string // The host to bind to. Defaults to 0.0.0.0 + Checks []map[string]any // An array defining the specific checks to perform. +} + type ServerConfig struct { Encrypt *EncryptionConfig // Whether and how to use TLS. Defaults to none AKA no encryption. + Health HealthConfig // Whether to enable health endpoints on a secondary port. BindHost string // IP address to bind HTTP server to Port int // Port to bind HTTP server to RootPath string // Root HTTP Path to apply to all endpoints. Defaults to / TilePath string // HTTP Path to serve tiles under (in addition to RootPath). Defaults to tiles which means /tiles/{layer}/{z}/{x}/{y}. - DocsPath string /// HTTP Path for accessing the documentation website. Defaults to /docs + DocsPath string // HTTP Path for accessing the documentation website. Defaults to docs Headers map[string]string // Include these headers in all response from server Production bool // Controls serving splash page, documentation, x-powered-by header. Defaults to false, set true to harden for prod Timeout uint // How long (in seconds) a request can be in flight before we cancel it and return an error @@ -206,6 +215,11 @@ func DefaultConfig() Config { Production: false, Timeout: 60, Gzip: false, + Health: HealthConfig{ + Enabled: false, + Port: 3000, + Host: "0.0.0.0", + }, }, Client: ClientConfig{ UserAgent: "tilegroxy/" + version, diff --git a/pkg/entities/layer/layer.go b/pkg/entities/layer/layer.go index ccebfa6d..bd365ea9 100644 --- a/pkg/entities/layer/layer.go +++ b/pkg/entities/layer/layer.go @@ -249,7 +249,9 @@ func (l *Layer) MatchesName(ctx context.Context, layerName string) bool { if doesMatch, matches := match(l.Pattern, layerName); doesMatch { if validateParamMatches(matches, l.ParamValidator) { layerPatternMatches, _ := pkg.LayerPatternMatchesFromContext(ctx) - *layerPatternMatches = matches + if layerPatternMatches != nil { + *layerPatternMatches = matches + } return true } } diff --git a/pkg/entities/layer/layergroup.go b/pkg/entities/layer/layergroup.go index bc896464..0eb90faa 100644 --- a/pkg/entities/layer/layergroup.go +++ b/pkg/entities/layer/layergroup.go @@ -32,6 +32,7 @@ import ( type LayerGroup struct { layers []*Layer + DefaultCache cache.Cache cacheHitCounter metric.Int64Counter cacheMissCounter metric.Int64Counter } @@ -55,6 +56,7 @@ func ConstructLayerGroup(cfg config.Config, cache cache.Cache, secreter secret.S layerGroup.cacheMissCounter, err2 = meter.Int64Counter("tilegroxy.cache.total.miss", metric.WithDescription("Number of requests that missed the cache (ignoring skips)")) layerGroup.layers = layerObjects + layerGroup.DefaultCache = cache return &layerGroup, errors.Join(err1, err2) } From 1c28c98274026f59507ad9382fbfb3d3d96c6a2f Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Mon, 9 Sep 2024 21:33:34 -0400 Subject: [PATCH 08/11] test: add tests for new health endpoint --- internal/server/error_test.go | 70 +++++++++++++++ internal/server/health.go | 16 ++-- internal/server/health_test.go | 153 +++++++++++++++++++++++++++++++++ internal/server/server.go | 2 +- internal/server/server_test.go | 48 ----------- 5 files changed, 232 insertions(+), 57 deletions(-) create mode 100644 internal/server/error_test.go create mode 100644 internal/server/health_test.go diff --git a/internal/server/error_test.go b/internal/server/error_test.go new file mode 100644 index 00000000..d319ed06 --- /dev/null +++ b/internal/server/error_test.go @@ -0,0 +1,70 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package server + +import ( + "io" + "net/http" + "net/http/httptest" + "testing" + + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func Test_ErrorVals_Execute(t *testing.T) { + cfg := config.DefaultConfig() + + cfg.Error.AlwaysOK = false + + for i := pkg.TypeOfErrorBounds; i <= pkg.TypeOfErrorOther; i++ { + cfg.Error.AlwaysOK = false + status, level, imgPath := errorVars(&cfg.Error, pkg.TypeOfError(i)) + assert.Greater(t, status, 300) + assert.NotEmpty(t, imgPath) + cfg.Error.AlwaysOK = true + status2, level2, imgPath2 := errorVars(&cfg.Error, pkg.TypeOfError(i)) + assert.Equal(t, http.StatusOK, status2) + assert.Equal(t, level2, level) + assert.Equal(t, imgPath, imgPath2) + } +} + +func Test_WriteErrorMessage_Execute(t *testing.T) { + cfg := config.DefaultConfig() + ctx := pkg.BackgroundContext() + + rw := httptest.NewRecorder() + + cfg.Error.Mode = config.ModeErrorNoError + writeErrorMessage(ctx, rw, &cfg.Error, pkg.TypeOfErrorOther, "test", "test", nil) + r := rw.Result() + defer func() { require.NoError(t, r.Body.Close()) }() + assert.Equal(t, 500, r.StatusCode) + b, _ := io.ReadAll(r.Body) + assert.Empty(t, b) + + cfg.Error.Mode = config.ModeErrorImage + cfg.Error.Images.Other = "safjakslfjaslkfj" // Invalid + writeErrorMessage(ctx, rw, &cfg.Error, pkg.TypeOfErrorOther, "test", "test", nil) + r = rw.Result() + defer func() { require.NoError(t, r.Body.Close()) }() + assert.Equal(t, 500, r.StatusCode) + b, _ = io.ReadAll(r.Body) + assert.Empty(t, b) + +} diff --git a/internal/server/health.go b/internal/server/health.go index eaba056f..f0f304cc 100644 --- a/internal/server/health.go +++ b/internal/server/health.go @@ -35,6 +35,12 @@ import ( "github.com/Michad/tilegroxy/pkg/static" ) +type CheckResult struct { + err error + timestamp time.Time + ttl time.Duration +} + type healthHandler struct { checks []health.HealthCheck checkResultCache *sync.Map @@ -132,17 +138,11 @@ func (h healthHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { _, err = w.Write(data) if err != nil { - slog.WarnContext(ctx, fmt.Sprintf("Unable to write to documentation request due to %v", err)) + slog.WarnContext(ctx, fmt.Sprintf("Unable to write to health request due to %v", err)) } } -type CheckResult struct { - err error - timestamp time.Time - ttl time.Duration -} - -func setupHealth(ctx context.Context, cfg *config.Config, layerGroup *layer.LayerGroup) (func(context.Context) error, error) { +func SetupHealth(ctx context.Context, cfg *config.Config, layerGroup *layer.LayerGroup) (func(context.Context) error, error) { h := cfg.Server.Health slog.InfoContext(ctx, fmt.Sprintf("Initializing health subsystem with %v checks on %v:%v", len(h.Checks), h.Host, h.Port)) diff --git a/internal/server/health_test.go b/internal/server/health_test.go new file mode 100644 index 00000000..5ac7803a --- /dev/null +++ b/internal/server/health_test.go @@ -0,0 +1,153 @@ +// Copyright 2024 Michael Davis +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package server + +import ( + "encoding/json" + "io" + "net/http" + "strconv" + "testing" + "time" + + "github.com/Michad/tilegroxy/pkg" + "github.com/Michad/tilegroxy/pkg/config" + "github.com/Michad/tilegroxy/pkg/entities/layer" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const port = 3456 + +func initialize(t *testing.T, fail bool) (config.Config, *layer.LayerGroup) { + cfgAll := config.DefaultConfig() + + var layerCfg config.LayerConfig + + if fail { + layerCfg = config.LayerConfig{ + ID: "test", + Provider: map[string]any{ + "name": "fail", + "onauth": true, + }, + } + } else { + layerCfg = config.LayerConfig{ + ID: "test", + Provider: map[string]any{ + "name": "static", + "color": "FFFFFF", + }, + } + } + + cfgAll.Server.Health.Enabled = true + cfgAll.Server.Health.Port = port + cfgAll.Server.Health.Checks = []map[string]any{ + { + "name": "tile", + "layer": "test", + "delay": 1, + }, + } + + cfgAll.Layers = append(cfgAll.Layers, layerCfg) + lg, err := layer.ConstructLayerGroup(cfgAll, nil, nil) + require.NoError(t, err) + + return cfgAll, lg +} + +// Make sure setup works and we get a callback that kills the server by ensuring we can do it twice +func Test_Health_Setup(t *testing.T) { + ctx := pkg.BackgroundContext() + cfg, lg := initialize(t, false) + + callback, err := SetupHealth(ctx, &cfg, lg) + require.NoError(t, err) + err = callback(ctx) + require.NoError(t, err) + + callback, err = SetupHealth(ctx, &cfg, lg) + require.NoError(t, err) + err = callback(ctx) + require.NoError(t, err) +} + +func Test_Health_Success(t *testing.T) { + ctx := pkg.BackgroundContext() + cfg, lg := initialize(t, false) + + callback, err := SetupHealth(ctx, &cfg, lg) + require.NoError(t, err) + time.Sleep(1 * time.Second) + + baseUrl := "http://127.0.0.1:" + strconv.Itoa(port) + resp, err := http.DefaultClient.Get(baseUrl) + require.NoError(t, err) + assert.Equal(t, http.StatusNoContent, resp.StatusCode) + require.NoError(t, resp.Body.Close()) + + resp, err = http.DefaultClient.Head(baseUrl + "/health") + require.NoError(t, err) + assert.Equal(t, http.StatusNoContent, resp.StatusCode) + require.NoError(t, resp.Body.Close()) + + resp, err = http.DefaultClient.Get(baseUrl + "/health") + require.NoError(t, err) + assert.Equal(t, http.StatusOK, resp.StatusCode) + + jsonByte, err := io.ReadAll(resp.Body) + require.NoError(t, err) + require.NoError(t, resp.Body.Close()) + + jsonMap := make(map[string]any) + err = json.Unmarshal(jsonByte, &jsonMap) + require.NoError(t, err) + + assert.Equal(t, "ok", jsonMap["status"], jsonMap) + + err = callback(ctx) + require.NoError(t, err) +} + +func Test_Health_Fail(t *testing.T) { + ctx := pkg.BackgroundContext() + cfg, lg := initialize(t, true) + + callback, err := SetupHealth(ctx, &cfg, lg) + require.NoError(t, err) + + baseUrl := "http://127.0.0.1:" + strconv.Itoa(port) + + resp, err := http.DefaultClient.Get(baseUrl + "/health") + require.NoError(t, err) + assert.Equal(t, http.StatusInternalServerError, resp.StatusCode) + + jsonByte, err := io.ReadAll(resp.Body) + require.NoError(t, err) + require.NoError(t, resp.Body.Close()) + + jsonMap := make(map[string]any) + err = json.Unmarshal(jsonByte, &jsonMap) + require.NoError(t, err) + + assert.Equal(t, "error", jsonMap["status"], jsonMap) + assert.Equal(t, "error", jsonMap["checks"].(map[string]any)["tilegroxy:checks"].([]any)[0].(map[string]any)["status"], jsonMap) + + err = callback(ctx) + require.NoError(t, err) +} diff --git a/internal/server/server.go b/internal/server/server.go index e05bcefd..a7700f7b 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -168,7 +168,7 @@ func ListenAndServe(config *config.Config, layerGroup *layer.LayerGroup, auth au var healthShutdown func(context.Context) error if config.Server.Health.Enabled { - healthShutdown, err = setupHealth(ctx, config, layerGroup) + healthShutdown, err = SetupHealth(ctx, config, layerGroup) if err != nil { return err diff --git a/internal/server/server_test.go b/internal/server/server_test.go index 0387fc1b..04ea59b0 100644 --- a/internal/server/server_test.go +++ b/internal/server/server_test.go @@ -15,60 +15,12 @@ package server import ( - "io" - "net/http" - "net/http/httptest" "testing" - "github.com/Michad/tilegroxy/pkg" "github.com/Michad/tilegroxy/pkg/config" - "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" ) -func Test_ErrorVals_Execute(t *testing.T) { - cfg := config.DefaultConfig() - - cfg.Error.AlwaysOK = false - - for i := pkg.TypeOfErrorBounds; i <= pkg.TypeOfErrorOther; i++ { - cfg.Error.AlwaysOK = false - status, level, imgPath := errorVars(&cfg.Error, pkg.TypeOfError(i)) - assert.Greater(t, status, 300) - assert.NotEmpty(t, imgPath) - cfg.Error.AlwaysOK = true - status2, level2, imgPath2 := errorVars(&cfg.Error, pkg.TypeOfError(i)) - assert.Equal(t, http.StatusOK, status2) - assert.Equal(t, level2, level) - assert.Equal(t, imgPath, imgPath2) - } -} - -func Test_WriteErrorMessage_Execute(t *testing.T) { - cfg := config.DefaultConfig() - ctx := pkg.BackgroundContext() - - rw := httptest.NewRecorder() - - cfg.Error.Mode = config.ModeErrorNoError - writeErrorMessage(ctx, rw, &cfg.Error, pkg.TypeOfErrorOther, "test", "test", nil) - r := rw.Result() - defer func() { require.NoError(t, r.Body.Close()) }() - assert.Equal(t, 500, r.StatusCode) - b, _ := io.ReadAll(r.Body) - assert.Empty(t, b) - - cfg.Error.Mode = config.ModeErrorImage - cfg.Error.Images.Other = "safjakslfjaslkfj" // Invalid - writeErrorMessage(ctx, rw, &cfg.Error, pkg.TypeOfErrorOther, "test", "test", nil) - r = rw.Result() - defer func() { require.NoError(t, r.Body.Close()) }() - assert.Equal(t, 500, r.StatusCode) - b, _ = io.ReadAll(r.Body) - assert.Empty(t, b) - -} - func Test_ListenAndServe_Validate(t *testing.T) { cfg := config.DefaultConfig() cfg.Server.Encrypt = &config.EncryptionConfig{Certificate: "asfjaslkf", Domain: ""} From 8a9558b57de5cb8debcbc952574b65d555732b39 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Mon, 9 Sep 2024 21:41:38 -0400 Subject: [PATCH 09/11] style: lint fixes --- internal/checks/cache.go | 12 ++++++++---- internal/checks/tile.go | 10 +++++----- internal/checks/tile_test.go | 14 +++++++------- internal/server/health.go | 11 +++++++---- internal/server/health_test.go | 12 ++++++------ 5 files changed, 33 insertions(+), 26 deletions(-) diff --git a/internal/checks/cache.go b/internal/checks/cache.go index 47201f0c..834f4d52 100644 --- a/internal/checks/cache.go +++ b/internal/checks/cache.go @@ -61,7 +61,7 @@ func (s CacheCheckRegistration) Name() string { return "cache" } -func (s CacheCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, lg *layer.LayerGroup, cache cache.Cache, allCfg *config.Config) (health.HealthCheck, error) { +func (s CacheCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, _ *layer.LayerGroup, cache cache.Cache, allCfg *config.Config) (health.HealthCheck, error) { cfg := checkConfig.(CacheCheckConfig) if cfg.Delay == 0 { @@ -71,10 +71,14 @@ func (s CacheCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, return &CacheCheck{cfg, cache, allCfg.Error.Messages}, nil } +const numColorDigits = 6 +const hexBase = 16 +const maxColorInt = 0xFFFFFF + func makeImage() (pkg.Image, error) { - col := strconv.FormatUint(rand.Uint64N(0xFFFFFF), 16) - if len(col) < 6 { - col = strings.Repeat("0", 6-len(col)) + col + col := strconv.FormatUint(rand.Uint64N(maxColorInt), hexBase) + if len(col) < numColorDigits { + col = strings.Repeat("0", numColorDigits-len(col)) + col } img, err := images.GetStaticImage("color:" + col) diff --git a/internal/checks/tile.go b/internal/checks/tile.go index 98375d59..c5e9eb8a 100644 --- a/internal/checks/tile.go +++ b/internal/checks/tile.go @@ -79,7 +79,7 @@ func (s TileCheckRegistration) Name() string { return "tile" } -func (s TileCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, lg *layer.LayerGroup, cache cache.Cache, allCfg *config.Config) (health.HealthCheck, error) { +func (s TileCheckRegistration) Initialize(checkConfig health.HealthCheckConfig, lg *layer.LayerGroup, _ cache.Cache, allCfg *config.Config) (health.HealthCheck, error) { cfg := checkConfig.(TileCheckConfig) if cfg.Delay == 0 { @@ -139,7 +139,7 @@ func (h *TileCheck) Check(ctx context.Context) error { return nil } -func (h *TileCheck) ValidateSame(ctx context.Context, img *pkg.Image) error { +func (h *TileCheck) ValidateSame(_ context.Context, img *pkg.Image) error { if h.img == nil { h.img = img return nil @@ -153,7 +153,7 @@ func (h *TileCheck) ValidateSame(ctx context.Context, img *pkg.Image) error { return nil } -func (h *TileCheck) ValidateContentType(ctx context.Context, img *pkg.Image) error { +func (h *TileCheck) ValidateContentType(_ context.Context, img *pkg.Image) error { if img.ContentType != h.Result { return fmt.Errorf(h.errorMessages.InvalidParam, "content type", img.ContentType) } @@ -161,7 +161,7 @@ func (h *TileCheck) ValidateContentType(ctx context.Context, img *pkg.Image) err return nil } -func (h *TileCheck) ValidateBase64(ctx context.Context, img *pkg.Image) error { +func (h *TileCheck) ValidateBase64(_ context.Context, img *pkg.Image) error { imgEncode := base64.StdEncoding.EncodeToString(img.Content) if imgEncode != h.Result { return fmt.Errorf(h.errorMessages.InvalidParam, "content", imgEncode) @@ -170,7 +170,7 @@ func (h *TileCheck) ValidateBase64(ctx context.Context, img *pkg.Image) error { return nil } -func (h *TileCheck) ValidateFile(ctx context.Context, img *pkg.Image) error { +func (h *TileCheck) ValidateFile(_ context.Context, img *pkg.Image) error { expected, err := os.ReadFile(h.Result) if err != nil { diff --git a/internal/checks/tile_test.go b/internal/checks/tile_test.go index 6821eec1..1f037721 100644 --- a/internal/checks/tile_test.go +++ b/internal/checks/tile_test.go @@ -67,7 +67,7 @@ func Test_Base64(t *testing.T) { tc := hc.(*TileCheck) err = hc.Check(ctx) - assert.NoError(t, err) + require.NoError(t, err) tc.Result = "test" err = hc.Check(ctx) @@ -92,7 +92,7 @@ func Test_File(t *testing.T) { require.NoError(t, err) err = hc.Check(ctx) - assert.NoError(t, err) + require.NoError(t, err) fileVal2, err := images.GetStaticImage("color:000") require.NoError(t, err) @@ -144,7 +144,7 @@ func Test_ContentType(t *testing.T) { tc := hc.(*TileCheck) err = hc.Check(ctx) - assert.NoError(t, err) + require.NoError(t, err) tc.Result = "test" err = hc.Check(ctx) @@ -163,18 +163,18 @@ func Test_Same(t *testing.T) { tc := hc.(*TileCheck) err = hc.Check(ctx) - assert.NoError(t, err) + require.NoError(t, err) err = hc.Check(ctx) - assert.NoError(t, err) + require.NoError(t, err) tc.img = &pkg.Image{} err = hc.Check(ctx) - assert.Error(t, err) + require.Error(t, err) err = hc.Check(ctx) - assert.NoError(t, err) + require.NoError(t, err) } func initialize(t *testing.T, fail bool) (config.Config, *layer.LayerGroup, TileCheckRegistration, TileCheckConfig) { diff --git a/internal/server/health.go b/internal/server/health.go index f0f304cc..89a9891f 100644 --- a/internal/server/health.go +++ b/internal/server/health.go @@ -35,6 +35,9 @@ import ( "github.com/Michad/tilegroxy/pkg/static" ) +var startupWaitTime = 100 * time.Millisecond +var checkLeeway = 5 * time.Second + type CheckResult struct { err error timestamp time.Time @@ -92,7 +95,7 @@ func (h healthHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) { detail["output"] = resultCheck.err.Error() } else { // Include 5 second leeway for check not being performed instantly - if resultCheck.timestamp.Add(resultCheck.ttl).Add(5 * time.Second).Before(time.Now()) { + if resultCheck.timestamp.Add(resultCheck.ttl).Add(checkLeeway).Before(time.Now()) { isOk = false detail["status"] = "error" detail["output"] = "stale" @@ -166,9 +169,9 @@ func SetupHealth(ctx context.Context, cfg *config.Config, layerGroup *layer.Laye return func(ctx context.Context) error { return errors.Join(callback(ctx), callback2(ctx)) }, err - } else { - return callback2, err } + + return callback2, err } return callback, err @@ -195,7 +198,7 @@ func setupHealthEndpoints(ctx context.Context, h config.HealthConfig, checks []h // Give srv a little breathing room to try to start up select { case err = <-srvErr: - case <-time.After(100 * time.Millisecond): + case <-time.After(startupWaitTime): } return srv.Shutdown, err diff --git a/internal/server/health_test.go b/internal/server/health_test.go index 5ac7803a..5aef652d 100644 --- a/internal/server/health_test.go +++ b/internal/server/health_test.go @@ -95,18 +95,18 @@ func Test_Health_Success(t *testing.T) { require.NoError(t, err) time.Sleep(1 * time.Second) - baseUrl := "http://127.0.0.1:" + strconv.Itoa(port) - resp, err := http.DefaultClient.Get(baseUrl) + baseURL := "http://127.0.0.1:" + strconv.Itoa(port) + resp, err := http.DefaultClient.Get(baseURL) require.NoError(t, err) assert.Equal(t, http.StatusNoContent, resp.StatusCode) require.NoError(t, resp.Body.Close()) - resp, err = http.DefaultClient.Head(baseUrl + "/health") + resp, err = http.DefaultClient.Head(baseURL + "/health") require.NoError(t, err) assert.Equal(t, http.StatusNoContent, resp.StatusCode) require.NoError(t, resp.Body.Close()) - resp, err = http.DefaultClient.Get(baseUrl + "/health") + resp, err = http.DefaultClient.Get(baseURL + "/health") require.NoError(t, err) assert.Equal(t, http.StatusOK, resp.StatusCode) @@ -131,9 +131,9 @@ func Test_Health_Fail(t *testing.T) { callback, err := SetupHealth(ctx, &cfg, lg) require.NoError(t, err) - baseUrl := "http://127.0.0.1:" + strconv.Itoa(port) + baseURL := "http://127.0.0.1:" + strconv.Itoa(port) - resp, err := http.DefaultClient.Get(baseUrl + "/health") + resp, err := http.DefaultClient.Get(baseURL + "/health") require.NoError(t, err) assert.Equal(t, http.StatusInternalServerError, resp.StatusCode) From 7e40f6a48ebcacb2a9bd41e9c86d94e43faff4f6 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Mon, 9 Sep 2024 21:51:06 -0400 Subject: [PATCH 10/11] docs: add details of HC format --- .../ROOT/pages/configuration/health.adoc | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/operation/modules/ROOT/pages/configuration/health.adoc b/docs/operation/modules/ROOT/pages/configuration/health.adoc index d0448b0d..1b3f59aa 100644 --- a/docs/operation/modules/ROOT/pages/configuration/health.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/health.adoc @@ -10,6 +10,8 @@ Hitting `/health` will invoke the full health check. The full healthcheck consists of invoking a series of individual checks. The list of specific checks is configurable. Actual execution of these checks is fully asynchronous, which means calling the health endpoint returns the cached results of the most recent check rather than triggering the checks immediately. +The healthcheck follows the format documented link:https://datatracker.ietf.org/doc/html/draft-inadarei-api-health-check-06[here]. + Configuration options: [cols="1,3,1,1,1"] @@ -195,3 +197,33 @@ server: delay: 600 ---- + + +Produces an output like: + + +---- +{ + "checks": { + "tilegroxy:checks": [ + { + "componentId": "0", + "componentType": "TileCheck", + "status": "ok", + "time": "2024-09-09T01:05:56-04:00", + "ttl": 60 + }, + { + "componentId": "1", + "componentType": "CacheCheck", + "status": "ok", + "time": "2024-09-09T01:05:56-04:00", + "ttl": 600 + } + ] + }, + "releaseId": "ab123", + "status": "ok", + "version": "v0.7.0" +} +---- \ No newline at end of file From d8a8d8af2b5d22cdd69bba68c204b3f9badc3f25 Mon Sep 17 00:00:00 2001 From: Michael Davis Date: Mon, 9 Sep 2024 22:17:13 -0400 Subject: [PATCH 11/11] docs: update order of config table to match new health docs --- .../configuration/authentication/custom.adoc | 9 ++++--- .../configuration/authentication/jwt.adoc | 24 +++++++++-------- .../authentication/static_key.adoc | 5 ++-- .../ROOT/pages/configuration/cache/disk.adoc | 7 ++--- .../pages/configuration/cache/memcache.adoc | 13 +++++----- .../pages/configuration/cache/memory.adoc | 7 ++--- .../ROOT/pages/configuration/cache/multi.adoc | 5 ++-- .../ROOT/pages/configuration/cache/redis.adoc | 21 ++++++++------- .../ROOT/pages/configuration/cache/s3.adoc | 19 +++++++------- .../ROOT/pages/configuration/client.adoc | 18 +++++++------ .../ROOT/pages/configuration/encryption.adoc | 14 +++++----- .../ROOT/pages/configuration/error.adoc | 24 ++++++++++------- .../ROOT/pages/configuration/layer.adoc | 19 +++++++------- .../modules/ROOT/pages/configuration/log.adoc | 26 +++++++++++-------- .../pages/configuration/provider/blend.adoc | 15 ++++++----- .../pages/configuration/provider/cgi.adoc | 19 +++++++------- .../pages/configuration/provider/custom.adoc | 7 ++--- .../pages/configuration/provider/effect.adoc | 9 ++++--- .../configuration/provider/fallback.adoc | 13 +++++----- .../pages/configuration/provider/proxy.adoc | 10 ++++--- .../pages/configuration/provider/ref.adoc | 5 ++-- .../pages/configuration/provider/static.adoc | 7 ++--- .../configuration/provider/transform.adoc | 11 ++++---- .../configuration/provider/url_template.adoc | 11 ++++---- .../secret/aws_secrets_manager.adoc | 15 ++++++----- .../ROOT/pages/configuration/server.adoc | 24 +++++++++-------- .../ROOT/pages/configuration/telemetry.adoc | 6 +++-- 27 files changed, 201 insertions(+), 162 deletions(-) diff --git a/docs/operation/modules/ROOT/pages/configuration/authentication/custom.adoc b/docs/operation/modules/ROOT/pages/configuration/authentication/custom.adoc index 4f414b12..64a2afab 100644 --- a/docs/operation/modules/ROOT/pages/configuration/authentication/custom.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/authentication/custom.adoc @@ -10,26 +10,27 @@ Name should be "custom" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | token +| How to extract the auth token from the request. Each Key/Value should be one of the options in the table above | map[string]string | Yes | None -| How to extract the auth token from the request. Each Key/Value should be one of the options in the table above | cachesize +| Configures the size of the cache of already verified tokens used to avoid re-verifying every request. Set to -1 to disable | int | No | 100 -| Configures the size of the cache of already verified tokens used to avoid re-verifying every request. Set to -1 to disable | file +| Contains the path to the file containing the go code to perform validation of the auth token as a file | string | No | None -| Contains the path to the file containing the go code to perform validation of the auth token as a file |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/authentication/jwt.adoc b/docs/operation/modules/ROOT/pages/configuration/authentication/jwt.adoc index 089d1092..1c23df55 100644 --- a/docs/operation/modules/ROOT/pages/configuration/authentication/jwt.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/authentication/jwt.adoc @@ -6,6 +6,7 @@ Currently this implementation only supports a single key specified against a sin The following claims are supported/enforced: +[cols="1,4"] |=== | Claim | Implementation @@ -32,74 +33,75 @@ Name should be "jwt" Configuration options: +[cols="1,3,1,1,1"] |=== | Parameter | Type | Required | Default | Description | Key +| The key for verifying the signature. The public key if using asymmetric signing. If the value starts with "env." the remainder is interpreted as the name of the Environment Variable to use to retrieve the verification key. | string | Yes | None -| The key for verifying the signature. The public key if using asymmetric signing. If the value starts with "env." the remainder is interpreted as the name of the Environment Variable to use to retrieve the verification key. | Algorithm +| Algorithm to allow for JWT signature. One of: "HS256", "HS384", "HS512", "RS256", "RS384", "RS512", "ES256", "ES384", "ES512", "PS256", "PS384", "PS512", "EdDSA" | string | Yes | None -| Algorithm to allow for JWT signature. One of: "HS256", "HS384", "HS512", "RS256", "RS384", "RS512", "ES256", "ES384", "ES512", "PS256", "PS384", "PS512", "EdDSA" | HeaderName +| The header to extract the JWT from. If this is "Authorization" it removes "Bearer " from the start. Make sure this is in "canonical case" e.g. X-Header - auth will always fail otherwise | string | No | Authorization -| The header to extract the JWT from. If this is "Authorization" it removes "Bearer " from the start. Make sure this is in "canonical case" e.g. X-Header - auth will always fail otherwise | MaxExpiration +| How many seconds from now can the expiration be. JWTs more than X seconds from now will result in a 401 | uint32 | No | 1 day -| How many seconds from now can the expiration be. JWTs more than X seconds from now will result in a 401 | ExpectedAudience +| Require the "aud" grant to be this string | string | No | None -| Require the "aud" grant to be this string | ExpectedSubject +| Require the "sub" grant to be this string | string | No | None -| Require the "sub" grant to be this string | ExpectedIssuer +| Require the "iss" grant to be this string | string | No | None -| Require the "iss" grant to be this string | ExpectedScope +| Require the "scope" grant to contain this string | string | No | None -| Require the "scope" grant to contain this string | LayerScope +| If true the "scope" grant is used to whitelist access to layers | bool | No | false -| If true the "scope" grant is used to whitelist access to layers | ScopePrefix +| If true this prefix indicates scopes to use. For example a prefix of "tile/" will mean a scope of "tile/test" grants access to "test". Doesn't impact ExpectedScope | string | No | Empty string -| If true this prefix indicates scopes to use. For example a prefix of "tile/" will mean a scope of "tile/test" grants access to "test". Doesn't impact ExpectedScope | UserId +| Use the specified grant as the user identifier. This is just used for logging by default but it's made available to custom providers | string | No | sub -| Use the specified grant as the user identifier. This is just used for logging by default but it's made available to custom providers |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/authentication/static_key.adoc b/docs/operation/modules/ROOT/pages/configuration/authentication/static_key.adoc index 2f099449..256c4180 100644 --- a/docs/operation/modules/ROOT/pages/configuration/authentication/static_key.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/authentication/static_key.adoc @@ -8,12 +8,13 @@ Name should be "static key" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | key +| The bearer token to require be supplied. If not specified `tilegroxy` will generate a random token at startup and output it in logs | string | No | Auto -| The bearer token to require be supplied. If not specified `tilegroxy` will generate a random token at startup and output it in logs |=== \ No newline at end of file diff --git a/docs/operation/modules/ROOT/pages/configuration/cache/disk.adoc b/docs/operation/modules/ROOT/pages/configuration/cache/disk.adoc index 80cbf67e..fe1601f4 100644 --- a/docs/operation/modules/ROOT/pages/configuration/cache/disk.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/cache/disk.adoc @@ -10,20 +10,21 @@ Name should be "disk" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | path +| The absolute path to the directory to store cache entries within. Directory (and tree) will be created if it does not already exist | string | Yes | None -| The absolute path to the directory to store cache entries within. Directory (and tree) will be created if it does not already exist | filemode +| A https://pkg.go.dev/io/fs#FileMode[Go filemode] as an integer to use for all created files/directories. This might change in the future to support a more conventional unix permission notation | uint32 | No | 0777 -| A https://pkg.go.dev/io/fs#FileMode[Go filemode] as an integer to use for all created files/directories. This might change in the future to support a more conventional unix permission notation |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/cache/memcache.adoc b/docs/operation/modules/ROOT/pages/configuration/cache/memcache.adoc index e760519a..7db213a2 100644 --- a/docs/operation/modules/ROOT/pages/configuration/cache/memcache.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/cache/memcache.adoc @@ -6,38 +6,39 @@ Name should be "memcache" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | host +| The host of the memcache server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | String | No | 127.0.0.1 -| The host of the memcache server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | port +| The port of the memcache server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | int | No | 6379 -| The port of the memcache server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | keyprefix +| A prefix to use for keys stored in cache. Helps avoid collisions when multiple applications use the same memcache | string | No | None -| A prefix to use for keys stored in cache. Helps avoid collisions when multiple applications use the same memcache | ttl +| How long cache entries should persist for in seconds. Cannot be disabled. | uint32 | No | 1 day -| How long cache entries should persist for in seconds. Cannot be disabled. | servers +| The list of servers to connect to supplied as an array of objects, each with a host and key parameter. This should only have a single entry when operating in standalone mode. If this is unspecified it uses the standalone `host` and `port` parameters as a default, therefore this shouldn't be specified at the same time as those | Array of `host` and `port` | No | host and port -| The list of servers to connect to supplied as an array of objects, each with a host and key parameter. This should only have a single entry when operating in standalone mode. If this is unspecified it uses the standalone `host` and `port` parameters as a default, therefore this shouldn't be specified at the same time as those |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/cache/memory.adoc b/docs/operation/modules/ROOT/pages/configuration/cache/memory.adoc index 5a17c6c7..9dc78a5f 100644 --- a/docs/operation/modules/ROOT/pages/configuration/cache/memory.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/cache/memory.adoc @@ -8,20 +8,21 @@ Name should be "memory" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | maxsize +| Maximum number of tiles to hold in the cache. Must be at least 10. Setting this too high can cause out-of-memory panics. This is not a guaranteed setting, which entry is evicted when exceeding this size is an implementation detail and the size can temporarily grow somewhat larger. | uint16 | No | 100 -| Maximum number of tiles to hold in the cache. Must be at least 10. Setting this too high can cause out-of-memory panics. This is not a guaranteed setting, which entry is evicted when exceeding this size is an implementation detail and the size can temporarily grow somewhat larger. | ttl +| Maximum time to live for cache entries in seconds | uint32 | No | 3600 -| Maximum time to live for cache entries in seconds |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/cache/multi.adoc b/docs/operation/modules/ROOT/pages/configuration/cache/multi.adoc index 4c52e51d..cc5958f5 100644 --- a/docs/operation/modules/ROOT/pages/configuration/cache/multi.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/cache/multi.adoc @@ -8,14 +8,15 @@ Name should be "multi" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | tiers +| An array of Cache configurations. Multi should not be nested inside a Multi | Cache[] | Yes | None -| An array of Cache configurations. Multi should not be nested inside a Multi |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/cache/redis.adoc b/docs/operation/modules/ROOT/pages/configuration/cache/redis.adoc index 3810cf98..13773232 100644 --- a/docs/operation/modules/ROOT/pages/configuration/cache/redis.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/cache/redis.adoc @@ -6,62 +6,63 @@ Name should be "redis" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | host +| The host of the redis server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | String | No | 127.0.0.1 -| The host of the redis server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | port +| The port of the redis server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | int | No | 6379 -| The port of the redis server. A convenience equivalent to supplying `servers` with a single entry. Do not supply both this and `servers` | db +| Database number, defaults to 0. Unused in cluster mode | int | No | 0 -| Database number, defaults to 0. Unused in cluster mode | keyprefix +| A prefix to use for keys stored in cache. Serves a similar purpose as `db` in avoiding collisions when multiple applications use the same redis | string | No | None -| A prefix to use for keys stored in cache. Serves a similar purpose as `db` in avoiding collisions when multiple applications use the same redis | username +| Username to use to authenticate with redis | string | No | None -| Username to use to authenticate with redis | password +| Password to use to authenticate with redis | string | No | None -| Password to use to authenticate with redis | mode +| Controls operating mode of redis. Can be `standalone`, `ring` or `cluster`. Standalone is a single redis server. Ring distributes entries to multiple servers without any replication https://redis.uptrace.dev/guide/ring.html[(more details)]. Cluster is a proper redis cluster. | string | No | standalone -| Controls operating mode of redis. Can be `standalone`, `ring` or `cluster`. Standalone is a single redis server. Ring distributes entries to multiple servers without any replication https://redis.uptrace.dev/guide/ring.html[(more details)]. Cluster is a proper redis cluster. | ttl +| How long cache entries should persist for in seconds. Cannot be disabled. | uint32 | No | 1 day -| How long cache entries should persist for in seconds. Cannot be disabled. | servers +| The list of servers to connect to supplied as an array of objects, each with a host and key parameter. This should only have a single entry when operating in standalone mode. If this is unspecified it uses the standalone `host` and `port` parameters as a default, therefore this shouldn't be specified at the same time as those | Array of `host` and `port` | No | host and port -| The list of servers to connect to supplied as an array of objects, each with a host and key parameter. This should only have a single entry when operating in standalone mode. If this is unspecified it uses the standalone `host` and `port` parameters as a default, therefore this shouldn't be specified at the same time as those |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/cache/s3.adoc b/docs/operation/modules/ROOT/pages/configuration/cache/s3.adoc index 23420e06..6bd9d5f5 100644 --- a/docs/operation/modules/ROOT/pages/configuration/cache/s3.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/cache/s3.adoc @@ -14,56 +14,57 @@ Name should be "s3" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | bucket +| The name of the bucket to use | string | Yes | None -| The name of the bucket to use | path +| The path prefix to use for storing tiles | string | No | / -| The path prefix to use for storing tiles | region +| The AWS region containing the bucket. Required if region is not specified via other means. Consult https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints[AWS documentation] for possible values | string | No | None -| The AWS region containing the bucket. Required if region is not specified via other means. Consult https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints[AWS documentation] for possible values | access +| The AWS Access Key ID to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | string | No | None -| The AWS Access Key ID to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | secret +| The AWS Secret Key to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | string | No | None -| The AWS Secret Key to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | profile +| The profile to use to authenticate against the AWS API. Consult https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html#file-format-profile[AWS documentation for specifics] | string | No | None -| The profile to use to authenticate against the AWS API. Consult https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html#file-format-profile[AWS documentation for specifics] | storageclass +| The storage class to use for the object. You probably can leave this blank and use the bucket default. Consult https://aws.amazon.com/s3/storage-classes/[AWS documentation] for an overview of options. The following are currently valid: STANDARD REDUCED_REDUNDANCY STANDARD_IA ONEZONE_IA INTELLIGENT_TIERING GLACIER DEEP_ARCHIVE OUTPOSTS GLACIER_IR SNOW EXPRESS_ONEZONE | string | No | STANDARD -| The storage class to use for the object. You probably can leave this blank and use the bucket default. Consult https://aws.amazon.com/s3/storage-classes/[AWS documentation] for an overview of options. The following are currently valid: STANDARD REDUCED_REDUNDANCY STANDARD_IA ONEZONE_IA INTELLIGENT_TIERING GLACIER DEEP_ARCHIVE OUTPOSTS GLACIER_IR SNOW EXPRESS_ONEZONE | endpoint +| Override the S3 API Endpoint we talk to. Useful if you're using S3 outside AWS or using a directory bucket | string | No | AWS Auto -| Override the S3 API Endpoint we talk to. Useful if you're using S3 outside AWS or using a directory bucket |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/client.adoc b/docs/operation/modules/ROOT/pages/configuration/client.adoc index e5ab496f..a9f77fd5 100644 --- a/docs/operation/modules/ROOT/pages/configuration/client.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/client.adoc @@ -4,54 +4,56 @@ Configures how the HTTP client should operate for tile requests that require cal Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | UserAgent +| The user agent to include in outgoing http requests. | string | No | tilegroxy/VERSION -| The user agent to include in outgoing http requests. | MaxLength +| The maximum Content-Length to allow incoming responses | int | No | 10 MiB -| The maximum Content-Length to allow incoming responses | UnknownLength +| Allow responses that are missing a Content-Length header, this could lead to excessive memory usage | bool | No | false -| Allow responses that are missing a Content-Length header, this could lead to excessive memory usage | ContentTypes +| The content-types to allow remote servers to return. Anything else will be interpreted as an error | string[] | No | image/png, image/jpg -| The content-types to allow remote servers to return. Anything else will be interpreted as an error | StatusCodes +| The status codes from the remote server to consider successful | int[] | No | 200 -| The status codes from the remote server to consider successful | Headers +| Include these headers in requests | map[string]string | No | None -| Include these headers in requests | RewriteContentTypes +| Replaces ``Content-Type``s that match the key with the value. This is to handle servers returning a generic content type. Mapping to an empty string that will cause tilegroxy to intuit the Content-Type by inspecting the contents - this may be inaccurate for MVT. This only applies after the check that Content-Type is valid according to the `ContentTypes` parameter meaning your original Content-Type will need to be in both parameters to be used | map[string]string | No | {"application/octet-stream": ""} -| Replaces ``Content-Type``s that match the key with the value. This is to handle servers returning a generic content type. Mapping to an empty string that will cause tilegroxy to intuit the Content-Type by inspecting the contents - this may be inaccurate for MVT. This only applies after the check that Content-Type is valid according to the `ContentTypes` parameter meaning your original Content-Type will need to be in both parameters to be used |=== The following can be supplied as environment variables: +[cols="1,2"] |=== | Configuration Parameter | Environment Variable diff --git a/docs/operation/modules/ROOT/pages/configuration/encryption.adoc b/docs/operation/modules/ROOT/pages/configuration/encryption.adoc index e46486a7..24508dac 100644 --- a/docs/operation/modules/ROOT/pages/configuration/encryption.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/encryption.adoc @@ -11,42 +11,44 @@ If a certificate and keyfile are supplied the server will utilize option 1, othe Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | Domain +| The domain name you're operating with (the domain end-users use) | string | Yes | None -| The domain name you're operating with (the domain end-users use) | Cache +| The path to a directory to cache certificates in if using let's encrypt. | string | No | ./certs -| The path to a directory to cache certificates in if using let's encrypt. | Certificate +| The file path to the TLS certificate | string | None -| The file path to the TLS certificate | | KeyFile +| The file path to the keyfile | string | None -| The file path to the keyfile | | HttpPort +| The port used for non-encrypted traffic. Required if using Let's Encrypt to provide for the ACME challenge, in which case this needs to indirectly be 80 (that is, this can be set to e.g. 8080 if something ahead of this redirects 80 to 8080). Everything except .well-known will be redirected to the main port when set. | int | No | None -| The port used for non-encrypted traffic. Required if using Let's Encrypt to provide for the ACME challenge, in which case this needs to indirectly be 80 (that is, this can be set to e.g. 8080 if something ahead of this redirects 80 to 8080). Everything except .well-known will be redirected to the main port when set. |=== The following can be supplied as environment variables: +[cols="1,2"] |=== | Configuration Parameter | Environment Variable diff --git a/docs/operation/modules/ROOT/pages/configuration/error.adoc b/docs/operation/modules/ROOT/pages/configuration/error.adoc index 947e1c0e..2f327e25 100644 --- a/docs/operation/modules/ROOT/pages/configuration/error.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/error.adoc @@ -16,36 +16,38 @@ It is highly recommended you use the Image mode for production usage. Returning Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | Mode +| The error mode as described above. One of: text none image image+header | string | No | image -| The error mode as described above. One of: text none image image+header | Messages +| Controls the error messages returned as described below | ErrorMessages | No | Various -| Controls the error messages returned as described below | Images +| Controls the images returned for errors as described below | ErrorImages | No | Various -| Controls the images returned for errors as described below | AlwaysOk +| If set we always return 200 regardless of what happens | bool | No | false -| If set we always return 200 regardless of what happens |=== The following can be supplied as environment variables: +[cols="1,2"] |=== | Configuration Parameter | Environment Variable @@ -62,36 +64,38 @@ When using the image or image+header modes you can configure the images you want Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | OutOfBounds +| The image to display for requests outside the extent of the layer | string | No | embedded:transparent.png -| The image to display for requests outside the extent of the layer | Authentication +| The image to display for auth errors | string | No | embedded:unauthorized.png -| The image to display for auth errors | Provider +| The image to display for errors returned by the layer's provider | string | No | embedded:error.png -| The image to display for errors returned by the layer's provider | Other +| The image to display for all other errors | string | No | embedded:error.png -| The image to display for all other errors |=== The following can be supplied as environment variables: +[cols="1,2"] |=== | Configuration Parameter | Environment Variable diff --git a/docs/operation/modules/ROOT/pages/configuration/layer.adoc b/docs/operation/modules/ROOT/pages/configuration/layer.adoc index 506efb91..a1b758f7 100644 --- a/docs/operation/modules/ROOT/pages/configuration/layer.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/layer.adoc @@ -8,44 +8,45 @@ When using a pattern you can include https://regex101.com/[Regular Expressions] Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | id +| A url-safe identifier of the layer. Primarily used as the default path for incoming tile web requests | string | Yes | None -| A url-safe identifier of the layer. Primarily used as the default path for incoming tile web requests | pattern +| A url-safe pattern with non-subsequent placeholders | string | No | id -| A url-safe pattern with non-subsequent placeholders | paramValidator +| A mapping of regular expressions to use to validate the values that match against the placeholders. The regular expressions must match the full value. Specify a key of "*" to apply it to all values | map[string]string | No | None -| A mapping of regular expressions to use to validate the values that match against the placeholders. The regular expressions must match the full value. Specify a key of "*" to apply it to all values | provider -| Provider +| The configuration that drives how tiles are generated +| xref:configuration/provider/index.adoc[Provider] | Yes | None -| See below | client -| Client +| A Client configuration to use for this layer specifically that overrides the Client from the top-level of the configuration. See below for Client schema +| xref:configuration/client.adoc[Client] | No | None -| A Client configuration to use for this layer specifically that overrides the Client from the top-level of the configuration. See below for Client schema | skipcache +| If true, skip reading and writing to cache | bool | No | false -| If true, skip reading and writing to cache |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/log.adoc b/docs/operation/modules/ROOT/pages/configuration/log.adoc index 06f6afc5..96416e7a 100644 --- a/docs/operation/modules/ROOT/pages/configuration/log.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/log.adoc @@ -36,48 +36,50 @@ Level controls the verbosity of logs. There is no guarantee as to the specific l Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | Console +| Whether to write application logs to standard out | bool | No | true -| Whether to write application logs to standard out | Path +| The file location to write logs to. Log rotation is not built-in, use an external tool to avoid excessive growth | string | No | None -| The file location to write logs to. Log rotation is not built-in, use an external tool to avoid excessive growth | Format +| The format to output application logs in. Applies to both standard out and file out. Possible values: plain, json | string | No | plain -| The format to output application logs in. Applies to both standard out and file out. Possible values: plain, json | Level +| The most-detailed log level that should be included. Possible values: debug, info, warn, error, trace, absurd | string | No | info -| The most-detailed log level that should be included. Possible values: debug, info, warn, error, trace, absurd | Request +| Whether to include any extra attributes based on request parameters (excluding explicitly requested). If auto (default) it defaults true if format is json, false otherwise | bool | No | auto -| Whether to include any extra attributes based on request parameters (excluding explicitly requested). If auto (default) it defaults true if format is json, false otherwise | Headers +| Headers to include as attributes in structured log messages. Attribute key will be in all lowercase. | string[] | No | None -| Headers to include as attributes in structured log messages. Attribute key will be in all lowercase. |=== The following can be supplied as environment variables: +[cols="1,3"] |=== | Configuration Parameter | Environment Variable @@ -106,30 +108,32 @@ Configures logs for incoming HTTP requests. Primarily outputs in standard Apache Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | Console +| Whether to write access logs to standard out | bool | No | true -| Whether to write access logs to standard out | Path +| The file location to write logs to. Log rotation is not built-in, use an external tool to avoid excessive growth | string | No | None -| The file location to write logs to. Log rotation is not built-in, use an external tool to avoid excessive growth | Format +| The format to output access logs in. Applies to both standard out and file out. Possible values: common, combined | string | No | common -| The format to output access logs in. Applies to both standard out and file out. Possible values: common, combined |=== The following can be supplied as environment variables: +[cols="1,3"] |=== | Configuration Parameter | Environment Variable diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/blend.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/blend.adoc index e92d521b..9390a5f6 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/blend.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/blend.adoc @@ -11,44 +11,45 @@ image:diagram-blend.png[Blend request flow] Name should be "blend" +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | providers +| The providers to blend together. Order matters | Provider[] | Yes | None -| The providers to blend together. Order matters | mode +| How to blend the images. https://github.com/anthonynsimon/bild#blend-modes[Examples of the modes]. Possible values: "add", "color burn", "color dodge", "darken", "difference", "divide", "exclusion", "lighten", "linear burn", "linear light", "multiply", "normal", "opacity", "overlay", "screen", "soft light", "subtract" | String | No | normal -| How to blend the images. https://github.com/anthonynsimon/bild#blend-modes[Examples of the modes]. Possible values: "add", "color burn", "color dodge", "darken", "difference", "divide", "exclusion", "lighten", "linear burn", "linear light", "multiply", "normal", "opacity", "overlay", "screen", "soft light", "subtract" | opacity +| Only applicable if mode is "opacity". A value between 0 and 1 controlling the amount of opacity | Float | No | 0 -| Only applicable if mode is "opacity". A value between 0 and 1 controlling the amount of opacity | layer +| An alternative to the `providers` parameter for specifying references to other layers that utilize patterns. Equivalent to specifying a number of <> providers in `providers` | Object - See next rows | No | None -| An alternative to the `providers` parameter for specifying references to other layers that utilize patterns. Equivalent to specifying a number of <> providers in `providers` | layer.pattern +| A string with one or more placeholders present wrapped in curly brackets that match the layer placeholder you want to refer towards | String | Yes | None -| A string with one or more placeholders present wrapped in curly brackets that match the layer placeholder you want to refer towards | layer.values +| An entry per instantiation of the layer, each entry should have a value for each placeholder in the pattern with the key being the placeholder and the value being the replacement value | {"k":"v"}[] | Yes | None -| An entry per instantiation of the layer, each entry should have a value for each placeholder in the pattern with the key being the placeholder and the value being the replacement value |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/cgi.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/cgi.adoc index 86f634ba..aa7db4f0 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/cgi.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/cgi.adoc @@ -6,54 +6,55 @@ Name should be "cgi" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | Exec +| The path to the CGI executable | string | Yes | None -| The path to the CGI executable | Args +| Arguments to pass into the executable in standard "split on spaces" format | []string | No | None -| Arguments to pass into the executable in standard "split on spaces" format | Uri +| The URI (path + query) to pass into the CGI for the fake request - think mod_rewrite style invocation of the CGI | string | Yes | None -| The URI (path + query) to pass into the CGI for the fake request - think mod_rewrite style invocation of the CGI | Domain +| The host to pass into the CGI for the fake request | string | No | localhost -| The host to pass into the CGI for the fake request | Headers +| Extra headers to pass into the CGI with the request | map[string][]string | No | None -| Extra headers to pass into the CGI with the request | Env +| Extra environment variables to supply to the CGI invocations. If the value is an empty string it passes along the value from the main tilegroxy invocation | map[string]string | No | None -| Extra environment variables to supply to the CGI invocations. If the value is an empty string it passes along the value from the main tilegroxy invocation | WorkingDir +| Working directory for the CGI invocation | string | No | Base dir of exec -| Working directory for the CGI invocation | InvalidAsError +| If true, if the CGI response includes a content type that isn't in the <>'s list of acceptable content types then it treats the response body as an error message | bool | No | false -| If true, if the CGI response includes a content type that isn't in the <>'s list of acceptable content types then it treats the response body as an error message |=== \ No newline at end of file diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/custom.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/custom.adoc index bce02942..58872c94 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/custom.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/custom.adoc @@ -6,18 +6,19 @@ Name should be "custom" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | file +| An absolute file path to find the Go code implementing the provider | string | Yes | None -| An absolute file path to find the Go code implementing the provider | Any +| Any additional parameter you include will be automatically supplied to your custom provider as-is | Any | No | None -| Any additional parameter you include will be automatically supplied to your custom provider as-is |=== \ No newline at end of file diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/effect.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/effect.adoc index fda144c2..7e73440f 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/effect.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/effect.adoc @@ -6,26 +6,27 @@ This can only be used with layers that return JPEG or PNG images. The result alw Name should be "effect" +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | provider +| The provider to get the imagery to apply the effect to | Provider | Yes | None -| The provider to get the imagery to apply the effect to | mode +| The effect to apply. https://github.com/anthonynsimon/bild[Examples of the modes]. Possible values: "blur", "gaussian", "brightness", "contrast", "gamma", "hue", "saturation", "dilate", "edge detection", "erode", "median", "threshold", "emboss", "grayscale", "invert", "sepia", "sharpen", or "sobel" | String | No | normal -| The effect to apply. https://github.com/anthonynsimon/bild[Examples of the modes]. Possible values: "blur", "gaussian", "brightness", "contrast", "gamma", "hue", "saturation", "dilate", "edge detection", "erode", "median", "threshold", "emboss", "grayscale", "invert", "sepia", "sharpen", or "sobel" | intensity +| The intensity of the effect, exact meaning/value range depends on mode. Only applicable if mode is one of: "blur", "gaussian", "brightness", "contrast", "gamma", "hue", "saturation", "dilate", "edge detection", "erode", "median", or "threshold" | Float | No | 0 -| The intensity of the effect, exact meaning/value range depends on mode. Only applicable if mode is one of: "blur", "gaussian", "brightness", "contrast", "gamma", "hue", "saturation", "dilate", "edge detection", "erode", "median", or "threshold" |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/fallback.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/fallback.adoc index cc08e101..b5953b77 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/fallback.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/fallback.adoc @@ -10,38 +10,39 @@ Name should be "fallback" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | primary +| The provider to delegate to first | Provider | Yes | None -| The provider to delegate to first | secondary +| The provider to delegate to if primary returns an error | Provider | Yes | None -| The provider to delegate to if primary returns an error | zoom +| Zooming below or above this range will activate the fallback. Can be a single number, a range with a dash between start and end, or a comma separated list of the first two options. For example "4" "2-3" or "2,3-4" | String | No | 0-21 -| Zooming below or above this range will activate the fallback. Can be a single number, a range with a dash between start and end, or a comma separated list of the first two options. For example "4" "2-3" or "2,3-4" | bounds +| Any tiles that don't intersect with this bounds will activate the fallback | Object with north, south, east, west | No | Whole world -| Any tiles that don't intersect with this bounds will activate the fallback | cache +| When to save the resulting tile to the cache. Options: always, unless-error, unless-fallback. | string | No | unless-error -| When to save the resulting tile to the cache. Options: always, unless-error, unless-fallback. |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/proxy.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/proxy.adoc index 322b6e6f..19196c30 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/proxy.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/proxy.adoc @@ -10,30 +10,32 @@ Name should be "proxy" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | url +| A URL pointing to the tile server. Should contain placeholders surrounded by "{}" that are replaced on-the-fly | string | Yes | None -| A URL pointing to the tile server. Should contain placeholders surrounded by "{}" that are replaced on-the-fly | inverty +| Changes Y tile numbering to be South-to-North instead of North-to-South. Only impacts Y/y placeholder | bool | No | false -| Changes Y tile numbering to be South-to-North instead of North-to-South. Only impacts Y/y placeholder | srid +| What projection bounds should be in. Can only be 4326 or 3857 | uint | No | 4326 -| What projection bounds should be in. Can only be 4326 or 3857 |=== The following placeholders are available in the URL: +[cols="1,4"] |=== | Placeholder | Description diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/ref.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/ref.adoc index 85ca90e4..5c8e0f55 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/ref.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/ref.adoc @@ -8,14 +8,15 @@ Name should be "ref" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | layer +| The layername to refer towards, treated the same if it were supplied in an incoming request. | string | Yes | None -| The layername to refer towards, treated the same if it were supplied in an incoming request. |=== Example diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/static.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/static.adoc index 2d33fbbb..b2209af2 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/static.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/static.adoc @@ -6,18 +6,19 @@ Name should be "static" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | image +| Either a filepath to an image on the local filesystem or one of the xref:configuration/error.adoc[built-in images] | string | Yes | None -| Either a filepath to an image on the local filesystem or one of the <> | color +| A hexcode (RGB or RGBA) of a color to return. Equivalent to specifying `image` with this value with a prefix of "color:" | string | No | None -| A hexcode (RGB or RGBA) of a color to return. Equivalent to specifying `image` with this value with a prefix of "color:" |=== diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/transform.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/transform.adoc index 756d4320..55bd0dfc 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/transform.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/transform.adoc @@ -14,32 +14,33 @@ Name should be "transform" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | file +| An absolute file path to find the Go code implementing the transformation | string | No | None -| An absolute file path to find the Go code implementing the transformation | formula +| The go code implementing the transformation. Required if file isn't included | string | No | None -| The go code implementing the transformation. Required if file isn't included | provider +| The provider to get the imagery to transform | Provider | Yes | None -| The provider to get the imagery to transform | threads +| How many threads (goroutines) to use per tile. The typical tile has 65,536 pixels, setting this to 8 for instance means each thread has to process 8,192 pixels in parallel. This helps avoid latency becoming problematically slow. | int | No | 1 -| How many threads (goroutines) to use per tile. The typical tile has 65,536 pixels, setting this to 8 for instance means each thread has to process 8,192 pixels in parallel. This helps avoid latency becoming problematically slow. |=== Example: diff --git a/docs/operation/modules/ROOT/pages/configuration/provider/url_template.adoc b/docs/operation/modules/ROOT/pages/configuration/provider/url_template.adoc index 65c07b7f..f9c3736a 100644 --- a/docs/operation/modules/ROOT/pages/configuration/provider/url_template.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/provider/url_template.adoc @@ -8,30 +8,31 @@ Name should be "url template" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | template +| A URL pointing to the tile server. Should contain placeholders `$xmin` `$xmax` `$ymin` and `$ymax` for tile bounds and can also contains `$srs` `$width` and `$height` | string | Yes | None -| A URL pointing to the tile server. Should contain placeholders `$xmin` `$xmax` `$ymin` and `$ymax` for tile bounds and can also contains `$srs` `$width` and `$height` | width +| What to use for $width placeholder | uint | No | 256 -| What to use for $width placeholder | height +| What to use for $height placeholder | uint | No | 256 -| What to use for $height placeholder | srid +| What projection the bounds should be in and what to use for $srs placeholder. Can only be 4326 or 3857 | uint | No | 4326 -| What projection the bounds should be in and what to use for $srs placeholder. Can only be 4326 or 3857 |=== diff --git a/docs/operation/modules/ROOT/pages/configuration/secret/aws_secrets_manager.adoc b/docs/operation/modules/ROOT/pages/configuration/secret/aws_secrets_manager.adoc index d319317a..6264eed2 100644 --- a/docs/operation/modules/ROOT/pages/configuration/secret/aws_secrets_manager.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/secret/aws_secrets_manager.adoc @@ -12,42 +12,43 @@ Name should be "awssecretsmanager" Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | separator +| How to separate the Id of the secret from the JSON key in the secret name as a whole | string | No | : -| How to separate the Id of the secret from the JSON key in the secret name as a whole | ttl +| How long to cache secrets in seconds. Cache disabled if less than 0. | int | No | 1 hour -| How long to cache secrets in seconds. Cache disabled if less than 0. | region +| The AWS region containing the bucket. Required if region is not specified via other means. Consult https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints[AWS documentation] for possible values | string | No | None -| The AWS region containing the bucket. Required if region is not specified via other means. Consult https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints[AWS documentation] for possible values | access +| The AWS Access Key ID to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | string | No | None -| The AWS Access Key ID to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | secret +| The AWS Secret Key to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | string | No | None -| The AWS Secret Key to authenticate with. This is not recommended; it is offered as a fallback authentication method only. Consult https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-authentication.html[AWS documentation] for better options | profile +| The profile to use to authenticate against the AWS API. Consult https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html#file-format-profile[AWS documentation for specifics] | string | No | None -| The profile to use to authenticate against the AWS API. Consult https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html#file-format-profile[AWS documentation for specifics] |=== \ No newline at end of file diff --git a/docs/operation/modules/ROOT/pages/configuration/server.adoc b/docs/operation/modules/ROOT/pages/configuration/server.adoc index 5e15058e..c86c4e50 100644 --- a/docs/operation/modules/ROOT/pages/configuration/server.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/server.adoc @@ -5,72 +5,74 @@ Configures how the HTTP server should operate Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | BindHost +| IP address to bind HTTP server to | string | No | 127.0.0.1 -| IP address to bind HTTP server to | Port +| Port to bind HTTP server to | int | No | 8080 -| Port to bind HTTP server to | RootPath +| The root HTTP Path to serve all requests under. | string | No | / -| The root HTTP Path to serve all requests under. | TilePath +| The HTTP Path to serve tiles under in addition to RootPath. The defaults will result in a path that looks like /tiles/\{layer}/\{z}/\{x}/\{y} | string | No | tiles -| The HTTP Path to serve tiles under in addition to RootPath. The defaults will result in a path that looks like /tiles/\{layer}/\{z}/\{x}/\{y} | Headers +| Include these headers in all response from server | map[string]string | No | None -| Include these headers in all response from server | Production +| Hardens operation for usage in production. For instance, controls serving splash page, documentation, x-powered-by header. | bool | No | false -| Hardens operation for usage in production. For instance, controls serving splash page, documentation, x-powered-by header. | Timeout +| How long (in seconds) a request can be in flight before we cancel it and return an error | uint | No | 60 -| How long (in seconds) a request can be in flight before we cancel it and return an error | Gzip +| Whether to gzip compress HTTP responses | bool | No | false -| Whether to gzip compress HTTP responses | Encrypt +| Configuration for enabling TLS (HTTPS). Don't specify to operate without encryption (the default) | xref:configuration/encryption.adoc[Encryption] | No | None -| Configuration for enabling TLS (HTTPS). Don't specify to operate without encryption (the default) | Health +| Configuration to turn on endpoints for validating the health of the server on a secondary port | xref:configuration/health.adoc[Health] | No | None -| Configuration to turn on endpoints for validating the health of the server on a secondary port |=== The following can be supplied as environment variables: +[cols="1,3"] |=== | Configuration Parameter | Environment Variable diff --git a/docs/operation/modules/ROOT/pages/configuration/telemetry.adoc b/docs/operation/modules/ROOT/pages/configuration/telemetry.adoc index 31f942d0..e3307206 100644 --- a/docs/operation/modules/ROOT/pages/configuration/telemetry.adoc +++ b/docs/operation/modules/ROOT/pages/configuration/telemetry.adoc @@ -7,18 +7,20 @@ More details on Telemetry capabilities can be found in xref:./telemetry.adoc[Tel Configuration options: +[cols="1,3,1,1,1"] |=== -| Parameter | Type | Required | Default | Description +| Parameter | Description | Type | Required | Default | Enabled +| Turns on/off telemetry | bool | No | false -| Turns on/off telemetry |=== The following can be supplied as environment variables: +[cols="1,3"] |=== | Configuration Parameter | Environment Variable