Error with ESM support when using the openai package

[dyarleniber]

When running a Node.js (v20) server with ESM modules using dd-trace and openai packages, the process crashes with the following error:

➜  dd-trace-openai-nodejs-example git:(main) node --env-file=.env --import dd-trace/register.js index.js
file:///Users/admin/workspace/personal/dd-trace-openai-nodejs-example/node_modules/openai/resources/index.mjs?iitm=true:72
    let $Completions = namespace.Completions
        ^

SyntaxError: Identifier '$Completions' has already been declared
    at ModuleLoader.moduleStrategy (node:internal/modules/esm/translators:167:18)
    at callTranslator (node:internal/modules/esm/loader:285:14)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:291:30)

Node.js v20.11.0

Expected Behavior

Running node --import dd-trace/register.js index.js should start the server without crashing.

Steps to Reproduce

The following repository contains the code to reproduce the issue: dd-trace-openai-nodejs-example.

Using Node.js v20.11.0:

1. Run npm init -y
2. Run npm i dd-trace@5.12.0 openai@4.40.0
3. Add to the package.json file:

{
  "type": "module"
}

4. Import dd-trace and openai in the index.js file:

import dataDogTracer from "dd-trace";
import OpenAI from "openai";

dataDogTracer.init({
  logInjection: true,
  runtimeMetrics: true,
});

const openAIClient = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

...

5. Run node --import dd-trace/register.js index.js

The following repository contains the code to reproduce the issue: dd-trace-openai-nodejs-example.

Your Environment

• Node.js: 20.11.0
• dd-trace: 5.12.0
• openai: 4.40.0
• OS: macOS Sonoma 14.4.1

[rattrayalex]

This may be similar to getsentry/sentry-javascript#12414 and thus openai/openai-node#1010 and openai/openai-node#903.

If that is the case, this problem would likely now surface like this:

TypeError: getDefaultAgent is not a function
    at OpenAI.buildRequest (file:///my-project/node_modules/openai/core.mjs:208:66)
    at OpenAI.makeRequest (file:///my-project/node_modules/openai/core.mjs:279:44)

and the workaround is to do this:

// loader.mjs
import { register } from "node:module";

register("import-in-the-middle/hook.mjs", import.meta.url, {
  parentURL: import.meta.url,
  data: { include: ["openai"]}, // this is the important bit here
});

which I believe may need to be done on the Datadog side.

[rattrayalex]

According to @guidev, this bug (or perhaps another one?) makes Datadog unusable in any node projects that use the openai node library, which gets almost as many weekly downloads as dd-trace.

@sabrenner, worth a look?

[adam-on-tailwind]

FYI the loader.mjs workaround prevented the error in my project, but also stopped traces from being sent. The way I was able to fix this was by importing the openai web shims using the --import flag, enabling experimental fetch --experimental-fetch and supplying an http Agent to the openai client at initialization.

Start node like this:

node --import openai/shims/web --import dd-trace/register.js --require dd-trace/init --experimental-fetch src/main/index.js

When initializing the openai client:

import 'http'
import OpenAI from 'openai'

const openai = new OpenAI({
  httpAgent: new http.Agent()
})

And it all works! Same steps apply if you're using Anthropic btw.

[sabrenner]

Thanks for the ping @rattrayalex! I'm wondering if folks here have tried @adam-on-tailwind's solution, and how it worked for them.

In any case, I try out including the { data: { include: ['openai'] } } workaround in dd-trace's register.js, and verify that using OpenAI doesn't break when using dd-trace. From what I've seen, just doing this won't fix tracing of OpenAI when using ESM, just resolve the TypeError: getDefaultAgent is not a function error. But as noted here and on other threads, I think that is an underlying issue in import-in-the-middle itself, and I'll work with some other folks to see what we can do about that.

[rattrayalex]

Awesome, thanks Sam! Let me know if there's anything I can do to help.