[chore] Make comparetest and pdatautil packages exported under pkg

[dmitryax]

Move internal/coreinternal/pdatautil and internal/comparetest module in pkg and export them as pkg/pdatautil and pkg/pdatatest so they can be used outside of the contrib repository after pcommon.Map.Sort() method is deprecated.

Once stabilized, they can be moved to the core pdata module.

Updates open-telemetry/opentelemetry-collector#6688

[runforesight]

Foresight Summary

    
Major Impacts

build-and-test-windows duration(5 seconds) has decreased 38 minutes 56 seconds compared to main branch avg(39 minutes 1 second).

View More Details

⭕  changelog workflow has finished in 4 seconds (2 minutes less than main branch avg.) and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
changelog	-     🔗	N/A	See Details

---

⭕  build-and-test-windows workflow has finished in 5 seconds (38 minutes 56 seconds less than main branch avg.) and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
windows-unittest-matrix	-     🔗	N/A	See Details
windows-unittest	-     🔗	N/A	See Details

---

✅  telemetrygen workflow has finished in 1 minute 5 seconds (1 minute 17 seconds less than main branch avg.) and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
publish-latest	-     🔗	N/A	See Details
build-dev	-     🔗	N/A	See Details
publish-stable	-     🔗	N/A	See Details

---

✅  tracegen workflow has finished in 1 minute 7 seconds (1 minute 12 seconds less than main branch avg.) and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
build-dev	-     🔗	N/A	See Details
publish-latest	-     🔗	N/A	See Details
publish-stable	-     🔗	N/A	See Details

---

✅  check-links workflow has finished in 1 minute 20 seconds and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
changed files	-     🔗	N/A	See Details
check-links	-     🔗	N/A	See Details

---

✅  prometheus-compliance-tests workflow has finished in 3 minutes 30 seconds (3 minutes 49 seconds less than main branch avg.) and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
prometheus-compliance-tests	-     🔗	✅ 21  ❌ 0  ⏭ 0    🔗	See Details

---

✅  build-and-test workflow has finished in 40 minutes 32 seconds (8 minutes 17 seconds less than main branch avg.) and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
unittest-matrix (1.19, internal)	-     🔗	✅ 561  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.18, internal)	-     🔗	✅ 561  ❌ 0  ⏭ 0    🔗	See Details
correctness-metrics	-     🔗	✅ 2  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, processor)	-     🔗	✅ 1487  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.18, extension)	-     🔗	✅ 528  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, extension)	-     🔗	✅ 528  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.18, processor)	-     🔗	✅ 1487  ❌ 0  ⏭ 0    🔗	See Details
correctness-traces	-     🔗	✅ 17  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, receiver-0)	-     🔗	✅ 2565  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.18, receiver-0)	-     🔗	✅ 2565  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, exporter)	-     🔗	✅ 2500  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.18, exporter)	-     🔗	✅ 2500  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, receiver-1)	-     🔗	✅ 1895  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.18, receiver-1)	-     🔗	✅ 1895  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.19, other)	-     🔗	✅ 4590  ❌ 0  ⏭ 0    🔗	See Details
unittest-matrix (1.18, other)	-     🔗	✅ 4590  ❌ 0  ⏭ 0    🔗	See Details
integration-tests	-     🔗	✅ 55  ❌ 0  ⏭ 0    🔗	See Details
setup-environment	-     🔗	N/A	See Details
check-codeowners	-     🔗	N/A	See Details
lint-matrix (receiver-0)	-     🔗	N/A	See Details
lint-matrix (receiver-1)	-     🔗	N/A	See Details
lint-matrix (processor)	-     🔗	N/A	See Details
lint-matrix (exporter)	-     🔗	N/A	See Details
lint-matrix (extension)	-     🔗	N/A	See Details
lint-matrix (internal)	-     🔗	N/A	See Details
lint-matrix (other)	-     🔗	N/A	See Details
check-collector-module-version	-     🔗	N/A	See Details
build-examples	-     🔗	N/A	See Details
checks	-     🔗	N/A	See Details
unittest (1.19)	-     🔗	N/A	See Details
unittest (1.18)	-     🔗	N/A	See Details
lint	-     🔗	N/A	See Details
cross-compile (darwin, amd64)	-     🔗	N/A	See Details
cross-compile (darwin, arm64)	-     🔗	N/A	See Details
cross-compile (linux, 386)	-     🔗	N/A	See Details
cross-compile (linux, amd64)	-     🔗	N/A	See Details
cross-compile (linux, arm)	-     🔗	N/A	See Details
cross-compile (linux, arm64)	-     🔗	N/A	See Details
cross-compile (linux, ppc64le)	-     🔗	N/A	See Details
cross-compile (windows, 386)	-     🔗	N/A	See Details
cross-compile (windows, amd64)	-     🔗	N/A	See Details
build-package (deb)	-     🔗	N/A	See Details
build-package (rpm)	-     🔗	N/A	See Details
windows-msi	-     🔗	N/A	See Details
publish-check	-     🔗	N/A	See Details
publish-dev	-     🔗	N/A	See Details
publish-stable	-     🔗	N/A	See Details

---

✅  load-tests workflow has finished in 7 minutes 51 seconds (6 minutes 3 seconds less than main branch avg.) and finished at 21st Jan, 2023.

---

Job	Failed Steps	Tests
loadtest (TestTraceAttributesProcessor)	-     🔗	✅ 3  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestIdleMode)	-     🔗	✅ 1  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs)	-     🔗	✅ 8  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile)	-     🔗	✅ 6  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS)	-     🔗	✅ 12  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs)	-     🔗	✅ 10  ❌ 0  ⏭ 0    🔗	See Details
loadtest (TestBallastMemory|TestLog10kDPS)	-     🔗	✅ 19  ❌ 0  ⏭ 0    🔗	See Details
setup-environment	-     🔗	N/A	See Details

---

🔎 See details on Foresight

^{*You can configure Foresight comments in your organization settings page.}

[mx-psi]

What modules would use this outside of contrib?

[dmitryax]

What modules would use this outside of contrib?

Any other distros that use Map.Sort() to do things like Logs == Logs in tests or build signatures from the attributes map

[dmitryax]

Two examples where the pdatatest is needed after Map.Sort() is removed:

• https://github.com/influxdata/influxdb-observability/blob/4e30db9141147d5b37ec2967baaf17053cd86baf/influx2otel/metrics_helper_test.go#L13
• https://github.com/SumoLogic/sumologic-otel-collector/blob/547af6af647bf09b0dd652fa2f6a95ae4be56fc1/pkg/processor/sourceprocessor/source_processor_test.go#L136

[bogdandrutu]

1. Should the pdatatest follow the same pdata structure and have a ptracetest, etc.?
2. Where is the golden used? Do we need to expose that now?
3. Should pdatautil actually be pcommonutil?

[dmitryax]

Where is the golden used? Do we need to expose that now?

Probably not needed to be exposed now. I'll move it aside.

Should the pdatatest follow the same pdata structure and have a ptracetest, etc.?

That makes sense to keep the package name the same as it'll eventually land in core/pdata, just to reduce the amount of work for users when we will be moving it to core, so they need to rename the module path, but not the package name. I'll update it.

Should pdatautil actually be pcommonutil?

I don't think we will need pcommonutil in core for the current functionality. I think it'll be pcommon.[Map|Value].Hash() methods, so it doesn't matter what's the module name here. I think pdatautil is a good one to start with

[dmitryax]

@bogdandrutu I've applied the changes discussed above

[bogdandrutu]

not sure if the module should be pdatatest or just pdata.

[dmitryax]

I thought pdatatest is better here to not confuse with actual core/pdata module