Update qryn+gigapipe docs #1594

lmangani · 2024-10-16T18:51:22Z

Updates for the qryn opensource and qryn/gigapipe integrations

lmangani · 2024-10-16T20:23:26Z

@blumamir we should be nearly ready!
Could you help us review the changes and troubleshoot the source of the Docs test failures?

lmangani · 2024-10-17T08:23:19Z

@blumamir the blocker seems to be related to secrets. Could you review our changes and suggest what's wrong?

blumamir

Thanks @lmangani

This looks great, thank you for improving it so nicely and making sure everything works.

I've added few comments, most of them are nits or suggestions, so feel free to address what you like and ignore what you don't want to do.

destinations/data/qryn.yaml

destinations/data/qrynoss.yaml

blumamir · 2024-10-17T08:59:11Z

docs/backends/qryn.mdx

+- **API Username**: The HTTP Basic Auth username for your qryn instance
+- **API Password**: The HTTP Basic Auth password for your qryn instance


If I got it right, those are the value in the QRYN_LOGIN and QRYN_PASSWORD environment variables described in the docs?

WDYT about making it explicit, so users know where they can find the content of these values to populate here quickly.

blumamir · 2024-10-17T09:02:21Z

docs/backends/qryn.mdx

+- **Name**: Choose a name for this backend _(e.g. qryn)_
+- **API Username**: The HTTP Basic Auth username for your qryn instance
+- **API Password**: The HTTP Basic Auth password for your qryn instance
+- **API URL**: The API Endpoint for for your qryn instance _(e.g. https://qryn.local:3100)_


The example here shows https. I wonder is http supported as well? and if so, which one is more common?

With the OSS version http is the default indeed. we can set the example to plain http

amazing thanks, I think it can avoid confusion for some users 🚀

blumamir · 2024-10-17T09:41:36Z

docs/backends/gigapipe.mdx

+- **API Secret**: The API Secret for your Gigapipe Account
+- **API Key**: The API Token for your Gigapipe Account


We experienced that users sometimes struggle to find out where in the managed backend ui console these values can be found.

If you find it helpful, you can browse other destinations and consider adding some instructions or screenshot on how to populate these values. for example

We included a Frame with a screenshot but we'll sure update it with a more specific one as we proceed.

blumamir · 2024-10-17T09:47:45Z

common/config/qryn.go

+	return fmt.Sprintf("%s://%s:%s@%s", u.Scheme, apiKey, apiSecret, u.Host), nil
+}
+
+func (g *Qryn) maybeAddExporterName(conf *qrynConf, currentConfig *Config, processorName string, name string,


Can you elaborate on the usage for this exporter name?

I see it adds an attribute "qryn_exporter" to all telemetry signals with values odigos-qryn-logs, odigos-qryn-traces, odigos-qryn-metrics.

Is this added to trace in qryn where the data came from?

Is qryn_exporter a common attribute in qryn ecosystem?

blumamir · 2024-10-17T09:57:33Z

common/config/qryn.go

@@ -101,5 +149,22 @@ func parseURL(rawURL, apiKey, apiSecret string) (string, error) {
 		return parseURL(fmt.Sprintf("https://%s", rawURL), apiKey, apiSecret)
 	}

-	return fmt.Sprintf("https://%s:%s@%s", apiKey, apiSecret, u.Host), nil
+	return fmt.Sprintf("%s://%s:%s@%s", u.Scheme, apiKey, apiSecret, u.Host), nil


important: if we encode the secret into the url this way, it will show up as plain text in the config map which can be undesirable for some users due to security concerns.

You can use the basicauthextension to implement this authentication, while keeping the password as env variable. see otlphttp for example code on how to use it

Update: after the latest commit, the value is no longer plain text in the configmap which is great.

If you like, you can still consider using the basic auto extension, or resolve this thread.

blumamir · 2024-10-17T10:22:13Z

common/config/qryn.go

@@ -101,5 +149,22 @@ func parseURL(rawURL, apiKey, apiSecret string) (string, error) {
 		return parseURL(fmt.Sprintf("https://%s", rawURL), apiKey, apiSecret)
 	}



Consider trimming rawURL white spaces, as it will make url.Parse(rawURL) return with error.

noWhiteSpaces := strings.TrimSpace(rawUrl) parsedUrl, err := url.Parse(noWhiteSpaces)

blumamir · 2024-10-17T10:31:52Z

common/config/qryn.go

@@ -101,5 +149,22 @@ func parseURL(rawURL, apiKey, apiSecret string) (string, error) {
 		return parseURL(fmt.Sprintf("https://%s", rawURL), apiKey, apiSecret)
 	}

-	return fmt.Sprintf("https://%s:%s@%s", apiKey, apiSecret, u.Host), nil
+	return fmt.Sprintf("%s://%s:%s@%s", u.Scheme, apiKey, apiSecret, u.Host), nil


if we just take the u.Host, we will silently drop any u.Path that the user might have set. I guess in most cases this is fine, but in clusters with internal routing this might break the export.

I am also fine with keeping it as is if you prefer and address this issue if someone bumps into it.

An alternative can be to modify the u object itself, and then use u.String() to make it a safe url string in idomatic way

akvlad · 2024-10-17T10:36:15Z

@lmangani @blumamir "secrets" issue seems done. Docs examples are updated as well. Please approve, let's see if all the CI tests are successful.

blumamir · 2024-10-17T10:37:18Z

@lmangani @blumamir "secrets" issue seems done. Docs examples are updated as well. Please approve, let's see if all the CI tests are successful.

Thanks qryn team, I just reviewed the previous version. taking a look at the new commit to update the review 🥇

Co-authored-by: Amir Blum <amirgiraffe@gmail.com>

blumamir

Thanks again qryn team, you are awesome.

Added few more minor comments, but they are all optional.
Feel free to adddress what makes sense to you and resolve the rest and then we can merge.

common/config/qryn.go

blumamir · 2024-10-17T10:58:51Z

destinations/data/qrynoss.yaml

+      displayName: Basic auth password
+      componentType: input
+      componentProps:
+        type: password


you can consider adding:

tooltip: 'some short text describing this option'

It will then show up in the UI as a question mark popup next to the option, giving more context on how to fecth it or what it does.

destinations/data/qrynoss.yaml

blumamir · 2024-10-17T11:08:44Z

docs/backends/qryn.mdx

+- **Name**: Choose a name for this backend _(e.g. qryn)_
+- **API Username**: The HTTP Basic Auth username for your qryn instance
+- **API Password**: The HTTP Basic Auth password for your qryn instance
+- **API URL**: The API Endpoint for for your qryn instance _(e.g. https://qryn.local:3100)_


amazing thanks, I think it can avoid confusion for some users 🚀

common/config/qryn.go

blumamir · 2024-10-17T11:13:42Z

common/config/qryn.go

@@ -101,5 +149,22 @@ func parseURL(rawURL, apiKey, apiSecret string) (string, error) {
 		return parseURL(fmt.Sprintf("https://%s", rawURL), apiKey, apiSecret)
 	}

-	return fmt.Sprintf("https://%s:%s@%s", apiKey, apiSecret, u.Host), nil
+	return fmt.Sprintf("%s://%s:%s@%s", u.Scheme, apiKey, apiSecret, u.Host), nil


Update: after the latest commit, the value is no longer plain text in the configmap which is great.

If you like, you can still consider using the basic auto extension, or resolve this thread.

Co-authored-by: Amir Blum <amirgiraffe@gmail.com>

akvlad · 2024-10-17T11:44:53Z

@lmangani @blumamir Sorry guys. I have renamed the yamls as well as you did. Now they can fit the docs. Fixed the "urlparse" issue as well

lmangani · 2024-10-17T11:55:27Z

@blumamir I think we're ready! Please ping us on the side to provide you some testing tokens for validation 😉

blumamir · 2024-10-17T14:08:26Z

Thanks, merged the PR.
Will test the integration soon

akvlad and others added 5 commits October 16, 2024 16:57

qryn dest debug

960e0f2

Merge branch 'main' of github.com:metrico/odigos

8684e42

logo

5304e3b

Add Gigapipe backend docs

fb4437f

Update qryn docs

ea65fd5

lmangani marked this pull request as draft October 16, 2024 18:51

lmangani added 2 commits October 16, 2024 22:20

Update ENVs for qryn oss

4a86938

Update ENVs for gigapipe

8e4079d

akvlad added 3 commits October 17, 2024 12:48

take secrets back

98901a4

Merge branch 'main' into qryn-gigapipe-docs

3339409

docs examples update

11170d1

blumamir reviewed Oct 17, 2024

View reviewed changes

lmangani marked this pull request as ready for review October 17, 2024 10:51

Update destinations/data/qrynoss.yaml

7f6b3ab

Co-authored-by: Amir Blum <amirgiraffe@gmail.com>

blumamir approved these changes Oct 17, 2024

View reviewed changes

lmangani and others added 3 commits October 17, 2024 13:25

Update common/config/qryn.go

6bfea8b

Co-authored-by: Amir Blum <amirgiraffe@gmail.com>

Use http in example for qryn-oss

a64b679

yaml rename; url parse update; docs fix

1e62751

blumamir merged commit 41db3fd into odigos-io:main Oct 17, 2024
26 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update qryn+gigapipe docs #1594

Update qryn+gigapipe docs #1594

lmangani commented Oct 16, 2024 •

edited

Loading

lmangani commented Oct 16, 2024

lmangani commented Oct 17, 2024

blumamir left a comment

blumamir Oct 17, 2024

blumamir Oct 17, 2024

lmangani Oct 17, 2024

blumamir Oct 17, 2024

blumamir Oct 17, 2024 •

edited

Loading

lmangani Oct 17, 2024

blumamir Oct 17, 2024 •

edited

Loading

blumamir Oct 17, 2024

blumamir Oct 17, 2024

blumamir Oct 17, 2024

blumamir Oct 17, 2024

akvlad commented Oct 17, 2024

blumamir commented Oct 17, 2024

blumamir left a comment

blumamir Oct 17, 2024

blumamir Oct 17, 2024

blumamir Oct 17, 2024

akvlad commented Oct 17, 2024 •

edited

Loading

lmangani commented Oct 17, 2024

blumamir commented Oct 17, 2024

		- API Username: The HTTP Basic Auth username for your qryn instance
		- API Password: The HTTP Basic Auth password for your qryn instance

		- API Secret: The API Secret for your Gigapipe Account
		- API Key: The API Token for your Gigapipe Account

		@@ -101,5 +149,22 @@ func parseURL(rawURL, apiKey, apiSecret string) (string, error) {
		return parseURL(fmt.Sprintf("https://%s", rawURL), apiKey, apiSecret)
		}

Update qryn+gigapipe docs #1594

Update qryn+gigapipe docs #1594

Conversation

lmangani commented Oct 16, 2024 • edited Loading

lmangani commented Oct 16, 2024

lmangani commented Oct 17, 2024

blumamir left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blumamir Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blumamir Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akvlad commented Oct 17, 2024

blumamir commented Oct 17, 2024

blumamir left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akvlad commented Oct 17, 2024 • edited Loading

lmangani commented Oct 17, 2024

blumamir commented Oct 17, 2024

lmangani commented Oct 16, 2024 •

edited

Loading

blumamir Oct 17, 2024 •

edited

Loading

blumamir Oct 17, 2024 •

edited

Loading

akvlad commented Oct 17, 2024 •

edited

Loading