Merge branch 'master' into wip-spinlock-improvements

CHANGELOG.md

@@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## Guideline to update the version:
+Increment the:
+- MAJOR version when you make incompatible API/ABI changes,
+- MINOR version when you add functionality in a backwards compatible manner, and
+- PATCH version when you make backwards compatible bug fixes.
+
+
 ## [Unreleased]
 ### Added
     * Trace API and SDK

CONTRIBUTING.md

@@ -93,14 +93,16 @@ To run tests locally, please read the [CI instructions](ci/README.md).
   as `work-in-progress`, or mark it as [`draft`](https://github.blog/2019-02-14-introducing-draft-pull-requests/).
 * Make sure [CLA](https://identity.linuxfoundation.org/projects/cncf) is
   signed and CI is clear.
+* For non-trivial changes, please update the [CHANGELOG](./CHANGELOG.md).
 
 ### How to Get PRs Merged
 
 A PR is considered to be **ready to merge** when:
 
-* It has received two approvals from [Approvers](https://github.com/open-telemetry/community/blob/master/community-membership.md#approver)
-  / [Maintainers](https://github.com/open-telemetry/community/blob/master/community-membership.md#maintainer)
-  (at different companies).
+* It has received two approvals with at least one approval from [Approver](https://github.com/open-telemetry/community/blob/master/community-membership.md#approver)
+  / [Maintainer](https://github.com/open-telemetry/community/blob/master/community-membership.md#maintainer)
+  (at different company).
+* A pull reqeust opened by an Approver / Maintainer can be merged with only one approval from  Approver / Maintainer (at different company).
 * Major feedback items/points are resolved.
 * It has been open for review for at least one working day. This gives people
   reasonable time to review.
@@ -109,6 +111,13 @@ A PR is considered to be **ready to merge** when:
 
 Any Approver / Maintainer can merge the PR once it is **ready to merge**.
 
+If a PR has been stuck (e.g. there are lots of debates and people couldn't agree on each other), the owner should try to get people aligned by:
+
+* Consolidating the perspectives and putting a summary in the PR. It is recommended to add a link into the PR description, which points to a comment with a summary in the PR conversation
+* Stepping back to see if it makes sense to narrow down the scope of the PR or split it up.
+
+If none of the above worked and the PR has been stuck for more than 2 weeks, the owner should bring it to the (OpenTelemetry C++ SIG meeting)[https://zoom.us/j/8203130519].
+
 ## Useful Resources
 
 Hi! If you’re looking at this document, these resources will provide you the

RELEASING.md

@@ -5,8 +5,8 @@
 1. Make sure all relevant changes for this release are included under `Unreleased` section in `CHANGELOG.md` and are in language that non-contributors to the project can understand.
 
 2. Run the pre-release script. It creates a branch `pre_release_<new-tag>` and updates `CHANGELOG.md` with the `<new-tag>`:
-    ```
-    ./pre_release.sh -t <new-tag>
+    ```console
+    ./buildscripts/pre_release.sh -t <new-tag>
     ```
 3. Verify that CHANGELOG.md is updated properly:
     ```
@@ -35,5 +35,30 @@ Failure to do so will leave things in a broken state.
     git push upstream
     ```
 
+## Versioning:
+
+Once tag is created, it's time to use that tag for Runtime Versioning
+
+1. Create a new brach for updating version information in `./sdk/src/version.cc`.
+    ```
+        git checkout -b update_version_${tag} master
+    ```
+2. Run the pre-commit script to update the version:
+    ```console
+        ./buildscripts/pre-commit
+    ```
+
+3. Check if any changes made since last release broke ABI compatibility. If yes, update `OPENTELEMETRY_ABI_VERSION_NO` in [version.h](api/include/opentelemetry/version.h).
+
+4. Push the changes to upstream and create a Pull Request on GitHub.
+
+5. Once changes are merged, move the tag created earlier to the new commit hash from step 4.
+
+    ```
+    git tag -f <previous-tag> <new-commit-hash>
+    git push --tags --force
+
+    ```
+
 ## Release
 Finally create a Release for the new <new-tag> on GitHub. The release body should include all the release notes from the Changelog for this release.

buildscripts/pre-commit

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+
+# This script is supposed to be executed to recreate version.cc. It can be executed as:
+#    git pre-commit hook
+#    git per-merge-commit hook
+#    manually as part of release process ( refer RELEASING.md )
+
+set -eo pipefail
+
+if [[ ! -w "$(pwd)/sdk/src/version/version.cc"  && ! -w "$(pwd)/api/include/opentelemetry/version.h" ]]; then
+    echo "Error: Version file(s) are not writable. Check permissions and try again."
+    exit 1
+fi
+
+# format: "v<MAJOR>.<MINOR>.<PATCH>-<PRERELEASE>+<BUILDMETADATA>-<NUMBER_OF_NEW_COMMITS>-g<LAST_COMMIT_HASH>""
+semver_regex="^v(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)(\\-([0-9A-Za-z-]+(\\.[0-9A-Za-z-]+)*))?(\\+[0-9A-Za-z-]+(\\.[0-9A-Za-z-]+)*)?-([0-9]+)-g([0-9|a-z]+)$"
+git_tag=$(git describe --tags --long 2>/dev/null) || true
+if [[ ! -z $git_tag ]] && [[ $git_tag =~ $semver_regex ]]; then
+    major="${BASH_REMATCH[1]}"
+    minor="${BASH_REMATCH[2]}"
+    patch="${BASH_REMATCH[3]}"
+    pre_release="${BASH_REMATCH[5]}" #optional
+    build_metadata="${BASH_REMATCH[7]}" #optional
+    count_new_commits="${BASH_REMATCH[9]}"
+    latest_commit_hash="${BASH_REMATCH[10]}"
+    if [[ -z ${major} ]] || [[ -z ${minor} ]] || [[ -z ${patch} ]] || [[ -z ${count_new_commits} ]] || [[ -z ${latest_commit_hash} ]]; then
+        echo "Error: Incorrect tag format recevived. Exiting.."
+        exit 1
+    fi
+else
+    major=0 && minor=0 && patch=0 && pre_release="" && build_metadata="" && count_new_commits=0
+    latest_commit_hash="$(git rev-parse --short HEAD)"
+fi
+: ${pre_release:="NONE"} # use default string if not defined
+: ${build_metadata:="NONE"} # use default string if not defined
+latest_commit_hash=$(git rev-parse ${latest_commit_hash}) # get full hash from short
+
+if [[ -z ${latest_commit_hash} ]]; then
+    echo "Error: Incorrect short hash received. Exiting.."
+    exit 1
+fi
+
+branch="$(git rev-parse --abbrev-ref HEAD)"
+short_version="${major}.${minor}.${patch}"
+full_version="${short_version}-${pre_release}-${build_metadata}-${count_new_commits}-${branch}-${latest_commit_hash}"
+
+#update api version.h
+sed -i "/^\#define OPENTELEMETRY_VERSION/c\#define OPENTELEMETRY_VERSION \"${short_version}\""  "$(pwd)/api/include/opentelemetry/version.h"
+#update sdk version.cc
+cat > "$(pwd)/sdk/src/version/version.cc" <<END
+// Please DONOT touch this file.
+// Any changes done here would be overwritten by pre-commit git hook
+
+#include "opentelemetry/sdk/version/version.h"
+
+OPENTELEMETRY_BEGIN_NAMESPACE
+namespace sdk
+{
+namespace version
+{
+    const int MAJOR_VERSION = ${major};
+    const int MINOR_VERSION = ${minor};
+    const int PATCH_VERSION = ${patch};
+    const char* PRE_RELEASE = "${pre_release}";
+    const char* BUILD_METADATA = "${build_metadata}";
+    const int COUNT_NEW_COMMITS = ${count_new_commits};
+    const char* BRANCH = "${branch}";
+    const char* COMMIT_HASH = "${latest_commit_hash}";
+    const char* SHORT_VERSION = "${short_version}";
+    const char* FULL_VERSION = "${full_version}";
+    const char* BUILD_DATE = "$(date -u)";
+}
+}
+OPENTELEMETRY_END_NAMESPACE
+END
+git add "$(pwd)/sdk/src/version/version.cc"

buildscripts/pre-merge-commit

@@ -0,0 +1 @@
+./pre-commit

ci/docfx.json

@@ -11,7 +11,10 @@
     "resource": [
       {
         "files": [
-
+          "api/**.h",
+          "api/**.cc",
+          "sdk/**.h",
+          "sdk/**.cc"
         ]
       }
     ],

sdk/include/opentelemetry/sdk/logs/exporter.h

@@ -0,0 +1,72 @@
+/*
+ * Copyright The OpenTelemetry Authors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include <memory>
+#include <vector>
+#include "opentelemetry/logs/log_record.h"
+#include "opentelemetry/nostd/span.h"
+#include "opentelemetry/sdk/logs/processor.h"
+
+OPENTELEMETRY_BEGIN_NAMESPACE
+namespace sdk
+{
+namespace logs
+{
+/**
+ * ExportResult is returned as result of exporting a batch of Log Records.
+ */
+enum class ExportResult
+{
+  // The batch was exported successfully
+  kSuccess = 0,
+  // The batch was exported unsuccessfully and was dropped, but can not be retried
+  kFailure
+};
+
+/**
+ * LogExporter defines the interface that log exporters must implement.
+ */
+class LogExporter
+{
+public:
+  virtual ~LogExporter() = default;
+
+  /**
+   * Exports the batch of log records to their export destination.
+   * This method must not be called concurrently for the same exporter instance.
+   * The exporter may attempt to retry sending the batch, but should drop
+   * and return kFailure after a certain timeout.
+   * @param records a span of unique pointers to log records
+   * @returns an ExportResult code (whether export was success or failure)
+   */
+  virtual ExportResult Export(
+      const nostd::span<std::unique_ptr<opentelemetry::logs::LogRecord>> &records) noexcept = 0;
+
+  /**
+   * Marks the exporter as ShutDown and cleans up any resources as required.
+   * Shutdown should be called only once for each Exporter instance.
+   * @param timeout minimum amount of microseconds to wait for shutdown before giving up and
+   * returning failure.
+   * @return true if the exporter shutdown succeeded, false otherwise
+   */
+  virtual bool Shutdown(
+      std::chrono::microseconds timeout = std::chrono::microseconds::max()) noexcept = 0;
+};
+}  // namespace logs
+}  // namespace sdk
+OPENTELEMETRY_END_NAMESPACE

sdk/include/opentelemetry/sdk/logs/processor.h

@@ -26,21 +26,37 @@ namespace sdk
 namespace logs
 {
 /**
- * This Log Processor is responsible for conversion of logs to exportable
- * representation and passing them to exporters.
+ * The Log Processor is responsible for passing log records
+ * to the configured exporter.
  */
 class LogProcessor
 {
 public:
   virtual ~LogProcessor() = default;
 
+  /**
+   * OnReceive is called by the SDK once a log record has been successfully created.
+   * @param record the log record
+   */
   virtual void OnReceive(std::unique_ptr<opentelemetry::logs::LogRecord> &&record) noexcept = 0;
 
-  virtual void ForceFlush(
-      std::chrono::microseconds timeout = std::chrono::microseconds(0)) noexcept = 0;
+  /**
+   * Exports all log records that have not yet been exported to the configured Exporter.
+   * @param timeout that the forceflush is required to finish within.
+   * @return a result code indicating whether it succeeded, failed or timed out
+   */
+  virtual bool ForceFlush(
+      std::chrono::microseconds timeout = std::chrono::microseconds::max()) noexcept = 0;
 
-  virtual void Shutdown(
-      std::chrono::microseconds timeout = std::chrono::microseconds(0)) noexcept = 0;
+  /**
+   * Shuts down the processor and does any cleanup required.
+   * ShutDown should only be called once for each processor.
+   * @param timeout minimum amount of microseconds to wait for
+   * shutdown before giving up and returning failure.
+   * @return true if the shutdown succeeded, false otherwise
+   */
+  virtual bool Shutdown(
+      std::chrono::microseconds timeout = std::chrono::microseconds::max()) noexcept = 0;
 };
 }  // namespace logs
 }  // namespace sdk

sdk/include/opentelemetry/sdk/logs/simple_log_processor.h

@@ -0,0 +1,64 @@
+/*
+ * Copyright The OpenTelemetry Authors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include <atomic>
+#include <mutex>
+
+#include "opentelemetry/common/spin_lock_mutex.h"
+#include "opentelemetry/sdk/logs/exporter.h"
+#include "opentelemetry/sdk/logs/processor.h"
+
+OPENTELEMETRY_BEGIN_NAMESPACE
+namespace sdk
+{
+namespace logs
+{
+/**
+ * The simple log processor passes all log records
+ * in a batch of 1 to the configured
+ * LogExporter.
+ *
+ * All calls to the configured LogExporter are synchronized using a
+ * spin-lock on an atomic_flag.
+ */
+class SimpleLogProcessor : public LogProcessor
+{
+
+public:
+  explicit SimpleLogProcessor(std::unique_ptr<LogExporter> &&exporter);
+  virtual ~SimpleLogProcessor() = default;
+
+  void OnReceive(std::unique_ptr<opentelemetry::logs::LogRecord> &&record) noexcept override;
+
+  bool ForceFlush(
+      std::chrono::microseconds timeout = std::chrono::microseconds::max()) noexcept override;
+
+  bool Shutdown(
+      std::chrono::microseconds timeout = std::chrono::microseconds::max()) noexcept override;
+
+private:
+  // The configured exporter
+  std::unique_ptr<LogExporter> exporter_;
+  // The lock used to ensure the exporter is not called concurrently
+  opentelemetry::common::SpinLockMutex lock_;
+  // The atomic boolean flag to ensure the ShutDown() function is only called once
+  std::atomic_flag shutdown_latch_{ATOMIC_FLAG_INIT};
+};
+}  // namespace logs
+}  // namespace sdk
+OPENTELEMETRY_END_NAMESPACE

sdk/include/opentelemetry/sdk/version/version.h

@@ -0,0 +1,23 @@
+#pragma once
+
+#include "opentelemetry/version.h"
+
+OPENTELEMETRY_BEGIN_NAMESPACE
+namespace sdk
+{
+namespace version
+{
+extern const int MAJOR_VERSION;
+extern const int MINOR_VERSION;
+extern const int PATCH_VERSION;
+extern const char *PRE_RELEASE;
+extern const char *BUILD_METADATA;
+extern const int COUNT_NEW_COMMITS;
+extern const char *BRANCH;
+extern const char *COMMIT_HASH;
+extern const char *FULL_VERSION;
+extern const char *FULL_VERSION_WITH_BRANCH_COMMITHASH;
+extern const char *BUILD_DATE;
+}  // namespace version
+}  // namespace sdk
+OPENTELEMETRY_END_NAMESPACE

sdk/src/CMakeLists.txt

@@ -2,3 +2,4 @@ add_subdirectory(common)
 add_subdirectory(trace)
 add_subdirectory(metrics)
 add_subdirectory(logs)
+add_subdirectory(version)

sdk/src/logs/CMakeLists.txt

@@ -1,3 +1,4 @@
-add_library(opentelemetry_logs logger_provider.cc logger.cc)
+add_library(opentelemetry_logs logger_provider.cc logger.cc
+                               simple_log_processor.cc)
 
 target_link_libraries(opentelemetry_logs opentelemetry_common)