Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DD_TRACE_GLOBAL_TAGS Configuration Option #533

Merged
merged 13 commits into from
Oct 26, 2019
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
using System;
using System.Collections;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;

Expand Down Expand Up @@ -98,5 +99,12 @@ IEnumerator IEnumerable.GetEnumerator()
{
return _sources.GetEnumerator();
}

/// <inheritdoc />
public IDictionary<string, string> GetDictionary(string key)
{
return _sources.Select(source => source.GetDictionary(key))
.FirstOrDefault(value => value != null);
}
}
}
5 changes: 5 additions & 0 deletions src/Datadog.Trace/Configuration/ConfigurationKeys.cs
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,11 @@ public static class ConfigurationKeys
/// <seealso cref="TracerSettings.AnalyticsEnabled"/>
public const string GlobalAnalyticsEnabled = "DD_TRACE_ANALYTICS_ENABLED";

/// <summary>
/// Configuration key for a list of tags to be applied globally to spans.
/// </summary>
public const string GlobalTags = "DD_TRACE_GLOBAL_TAGS";

/// <summary>
/// Configuration key for enabling or disabling the automatic injection
/// of correlation identifiers into the logging context.
Expand Down
12 changes: 11 additions & 1 deletion src/Datadog.Trace/Configuration/IConfigurationSource.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
using System.Collections.Generic;

namespace Datadog.Trace.Configuration
{
/// <summary>
Expand Down Expand Up @@ -36,5 +38,13 @@ public interface IConfigurationSource
/// <param name="key">The key that identifies the setting.</param>
/// <returns>The value of the setting, or <c>null</c> if not found.</returns>
bool? GetBool(string key);
}

/// <summary>
/// Gets the <see cref="IDictionary{TKey, TValue}"/> value of
/// the setting with the specified key.
/// </summary>
/// <param name="key">The key that identifies the setting.</param>
/// <returns>The value of the setting, or <c>null</c> if not found.</returns>
IDictionary<string, string> GetDictionary(string key);
}
}
37 changes: 37 additions & 0 deletions src/Datadog.Trace/Configuration/JsonConfigurationSource.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
Expand Down Expand Up @@ -97,5 +99,40 @@ public T GetValue<T>(string key)
? default
: token.Value<T>();
}

/// <summary>
/// Gets a <see cref="ConcurrentDictionary{TKey, TValue}"/> containing all of the values.
/// </summary>
/// <remarks>
/// Example JSON where `globalTags` is the configuration key.
/// {
/// "globalTags": {
/// "name1": "value1",
/// "name2": "value2"
/// }
/// }
/// </remarks>
/// <param name="key">The key that identifies the setting.</param>
/// <returns><see cref="IDictionary{TKey, TValue}"/> containing all of the key-value pairs.</returns>
/// <exception cref="JsonReaderException">Thrown if the configuration value is not a valid JSON string.</exception>"
public IDictionary<string, string> GetDictionary(string key)
{
var token = _configuration.SelectToken(key, errorWhenNoMatch: false);
if (token == null)
{
return null;
}

// Presence of a curly brace indicates that this is likely a JSON string. An exception will be thrown
// if it fails to parse or convert to a dictionary.
if (token.ToString().Contains("{"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point the json library already deserialized the json string, so I don't think we should serialize it back to a json string by calling JToken.ToString(). Consider checking the JToken.Type instead.

Suggested change
if (token.ToString().Contains("{"))
if (token.Type == JTokenType.Object)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh that's interesting. I was looking for a JToken method or property to validate, but that looks good! Thanks I'll test it out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucaspimentel if (token.Type == JTokenType.Object) won't work. The token.Type is String even in the case where the config string contains valid JSON, so it is never converted. What might work best is to not convert it using a ToObject but instead always treat it as a string and use StringConfigurationSource.ParseCustomKeyValues all of the time. This has the side-benefit of being able to handle the case of an extra curly brace in the config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After more thought, I'm inclined to continue to try to verify that the string is JSON by checking for the { character. Since this is considered to be a JSON-formatted config string, that appears to be the only way to verify it, at least that I can find. What I said above about always treating it like a string is not a good idea because bad JSON, e.g., including extra braces, will result in the keys and/or values including the braces, which would be a bad thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this has finally been improved (thanks for the insight, @lucaspimentel ). Now the tests for JsonConfigurationSource do not start by putting the key/value data into a dictionary. Instead, the value is passed directly to the JsonConfigurationSource constructor, as it should.

I also added a second bad data test.

{
var dictionary = JToken.Parse(token.ToString())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to above, token.ToString() serializes the token back into a json string, only to deserialize it a second time with JToken.Parse(). We should be able drop these two calls and use token.ToObject(...) directly.

Suggested change
var dictionary = JToken.Parse(token.ToString())
var dictionary = token

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

?.ToObject(typeof(ConcurrentDictionary<string, string>)) as ConcurrentDictionary<string, string>;
Copy link
Member

@lucaspimentel lucaspimentel Oct 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one's more of a nit, but there's a generic version of ToObject<T> which doesn't require the cast, making this more concise.

Suggested change
?.ToObject(typeof(ConcurrentDictionary<string, string>)) as ConcurrentDictionary<string, string>;
?.ToObject<ConcurrentDictionary<string, string>>();

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

return dictionary;
}

return StringConfigurationSource.ParseCustomKeyValues(token.ToString());
}
}
}
46 changes: 46 additions & 0 deletions src/Datadog.Trace/Configuration/StringConfigurationSource.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
using System.Collections.Concurrent;
using System.Collections.Generic;

namespace Datadog.Trace.Configuration
{
/// <summary>
Expand All @@ -6,6 +9,39 @@ namespace Datadog.Trace.Configuration
/// </summary>
public abstract class StringConfigurationSource : IConfigurationSource
{
/// <summary>
/// Returns a <see cref="IDictionary{TKey, TValue}"/> from parsing
/// <paramref name="data"/>.
/// </summary>
/// <param name="data">A string containing key-value pairs which are comma-separated, and for which the key and value are colon-separated.</param>
/// <returns><see cref="IDictionary{TKey, TValue}"/> of key value pairs.</returns>
public static IDictionary<string, string> ParseCustomKeyValues(string data)
{
var dictionary = new ConcurrentDictionary<string, string>();

if (data == null)
zacharycmontoya marked this conversation as resolved.
Show resolved Hide resolved
{
return dictionary;
lucaspimentel marked this conversation as resolved.
Show resolved Hide resolved
}

var entries = data.Split(',');

foreach (var e in entries)
{
var kv = e.Split(':');
if (kv.Length != 2)
{
continue;
}

var key = kv[0];
var value = kv[1];
dictionary[key] = value;
}

return dictionary;
}

/// <inheritdoc />
public abstract string GetString(string key);

Expand Down Expand Up @@ -38,5 +74,15 @@ public abstract class StringConfigurationSource : IConfigurationSource
? result
: (bool?)null;
}

/// <summary>
/// Gets a <see cref="ConcurrentDictionary{TKey, TValue}"/> from parsing
/// </summary>
/// <param name="key">The key</param>
/// <returns><see cref="ConcurrentDictionary{TKey, TValue}"/> containing all of the key-value pairs.</returns>
public IDictionary<string, string> GetDictionary(string key)
{
return ParseCustomKeyValues(GetString(key));
}
}
}
10 changes: 10 additions & 0 deletions src/Datadog.Trace/Configuration/TracerSettings.cs
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Newtonsoft.Json;

namespace Datadog.Trace.Configuration
{
Expand Down Expand Up @@ -76,6 +78,9 @@ public TracerSettings(IConfigurationSource source)
false;

Integrations = new IntegrationSettingsCollection(source);

GlobalTags = source?.GetDictionary(ConfigurationKeys.GlobalTags) as ConcurrentDictionary<string, string> ??
new ConcurrentDictionary<string, string>();
}

/// <summary>
Expand Down Expand Up @@ -141,6 +146,11 @@ public TracerSettings(IConfigurationSource source)
/// </summary>
public IntegrationSettingsCollection Integrations { get; }

/// <summary>
/// Gets or sets the global tags, which are applied to all <see cref="Span"/>s.
/// </summary>
public ConcurrentDictionary<string, string> GlobalTags { get; set; }
bobuva marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// Create a <see cref="TracerSettings"/> populated from the default sources
/// returned by <see cref="CreateDefaultConfigurationSource"/>.
Expand Down
3 changes: 3 additions & 0 deletions src/Datadog.Trace/IDatadogTracer.cs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
using System;
using System.Collections.Generic;
using Datadog.Trace.Configuration;
using Datadog.Trace.Sampling;

namespace Datadog.Trace
Expand All @@ -14,6 +15,8 @@ internal interface IDatadogTracer

ISampler Sampler { get; }

TracerSettings Settings { get; }

Span StartSpan(string operationName, ISpanContext parent, string serviceName, DateTimeOffset? startTime, bool ignoreActiveScope);

void Write(List<Span> span);
Expand Down
9 changes: 9 additions & 0 deletions src/Datadog.Trace/Tracer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,15 @@ public Span StartSpan(string operationName, ISpanContext parent = null, string s
span.SetTag(Tags.Env, env);
}

// Apply any global tags
if (Settings.GlobalTags.Count > 0)
{
foreach (var entry in Settings.GlobalTags)
{
span.SetTag(entry.Key, entry.Value);
}
}

traceContext.AddSpan(span);
return span;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,8 @@
</PropertyGroup>

<ItemGroup>
<None Include="..\..\src\Datadog.Trace.ClrProfiler.Native\bin\$(Configuration)\$(Platform)\**"
CopyToOutputDirectory="Always"
CopyToPublishDirectory="Always"
Link="profiler-lib\%(RecursiveDir)\%(Filename)%(Extension)" />
<Content Include="..\..\integrations.json"
CopyToOutputDirectory="Always"
CopyToPublishDirectory="Always"
Link="profiler-lib\integrations.json" />
<None Include="..\..\src\Datadog.Trace.ClrProfiler.Native\bin\$(Configuration)\$(Platform)\**" CopyToOutputDirectory="Always" CopyToPublishDirectory="Always" Link="profiler-lib\%(RecursiveDir)\%(Filename)%(Extension)" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we revert this entire files? Looks like VS added some json schema stuff at the end and also reformatted this line. I assume this happened while you were testing things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

<Content Include="..\..\integrations.json" CopyToOutputDirectory="Always" CopyToPublishDirectory="Always" Link="profiler-lib\integrations.json" />
</ItemGroup>

<ItemGroup>
Expand All @@ -24,4 +18,6 @@
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="12.0.1" />
</ItemGroup>

<ProjectExtensions><VisualStudio><UserProperties _1_1_4_1_1_4integrations_1json__JsonSchema="http://json.schemastore.org/ansible-stable-2.7" /></VisualStudio></ProjectExtensions>
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ public static IEnumerable<object[]> GetDefaultTestData()
yield return new object[] { CreateFunc(s => s.ServiceName), null };
yield return new object[] { CreateFunc(s => s.DisabledIntegrationNames.Count), 0 };
yield return new object[] { CreateFunc(s => s.LogsInjectionEnabled), false };
yield return new object[] { CreateFunc(s => s.GlobalTags.Count), 0 };
}

public static IEnumerable<object[]> GetTestData()
Expand All @@ -36,6 +37,20 @@ public static IEnumerable<object[]> GetTestData()
yield return new object[] { ConfigurationKeys.ServiceName, "web-service", CreateFunc(s => s.ServiceName), "web-service" };

yield return new object[] { ConfigurationKeys.DisabledIntegrations, "integration1;integration2", CreateFunc(s => s.DisabledIntegrationNames.Count), 2 };

yield return new object[] { ConfigurationKeys.GlobalTags, "k1:v1, k2:v2", CreateFunc(s => s.GlobalTags.Count), 2 };
}

// JsonConfigurationSource needs to be tested with JSON data, which cannot be used with the other IConfigurationSource implementations.
public static IEnumerable<object[]> GetJsonTestData()
{
yield return new object[] { ConfigurationKeys.GlobalTags, @"{ ""name1"":""value1"", ""name2"": ""value2""}", CreateFunc(s => s.GlobalTags.Count), 2 };
}

public static IEnumerable<object[]> GetBadJsonTestData()
{
// Extra opening brace
yield return new object[] { ConfigurationKeys.GlobalTags, @"{ {""name1"":""value1"", ""name2"": ""value2""}" };
}

public static Func<TracerSettings, object> CreateFunc(Func<TracerSettings, object> settingGetter)
Expand Down Expand Up @@ -90,7 +105,8 @@ public void EnvironmentConfigurationSource(

[Theory]
[MemberData(nameof(GetTestData))]
public void NameValueConfigurationSource1(
[MemberData(nameof(GetJsonTestData))]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I didn't know we could use [MemberData] multiple times. TIL.

public void JsonConfigurationSource(
string key,
string value,
Func<TracerSettings, object> settingGetter,
Expand All @@ -104,5 +120,18 @@ public void NameValueConfigurationSource1(
object actualValue = settingGetter(settings);
Assert.Equal(expectedValue, actualValue);
}

[Theory]
[MemberData(nameof(GetBadJsonTestData))]
public void JsonConfigurationSource_BadData(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice test! 👍

string key,
string value)
{
var config = new Dictionary<string, string> { [key] = value };
string json = JsonConvert.SerializeObject(config);
IConfigurationSource source = new JsonConfigurationSource(json);

Assert.Throws<JsonReaderException>(() => { new TracerSettings(source); });
}
}
}
4 changes: 4 additions & 0 deletions test/Datadog.Trace.Tests/Datadog.Trace.Tests.csproj
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<RunAnalyzersDuringBuild>true</RunAnalyzersDuringBuild>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="log4net" Version="2.0.8" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.1" />
Expand Down
1 change: 1 addition & 0 deletions test/Datadog.Trace.Tests/TracerTests.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Datadog.Trace.Agent;
Expand Down