Sourcemapping

[josephjclark]

Short Description

This PR uses sourcemapping to report accurate positions on error messages

Fixes #230

Implementation Details

Early stages here. I've worked out how to get the sourcemap from recast when compiling, and how to use it in the runtime to map positions. There's a couple of basic unit tests around this.

Plenty to do:

• Generate a text string representing the source line and mark where the error occured
• Work out how and where to log this. Probably in the runtime's error reporter, which I don't really like. Maybe this is a good time to refactor this?
• Make sure positions are logged appropriately for all the right error types.
• Get stacktrace showing the callstack within the VM, but not the runtime. I've had a go at this but it doesn't seem to work as I expect. I might branch this out as a new issue.
• Graft sourcemaps to the Workflow
• Fix the CLI to use new compiler ouputs
• Fix the worker to use new compiler outputs
• Look closely at worker outputs and make sure they're OK
• Test from the CLI and worker and show outputs here

QA Notes

List any considerations/cases/advice for testing/QA here.

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to know!):

• Code generation (copilot but not intellisense)
• Learning or fact checking
• Strategy / design
• Optimisation / refactoring
• Translation / spellchecking / doc gen
• Other
• I have not used AI

You can read more details in our Responsible AI Policy

[josephjclark]

TODO: I'm confusing myself here about the error reporter and the role it plays. I really think I'm gonna rip it out as part of this work.

What we really need is:

1. for the runtime to serialize errors in a nice way
2. CLI and the Worker to print those errors in a nice format for their platforms

That means CLI logs some nice pretty messages, and the worker probably just logs a JSON object (but not stringified)

Not sure. Needs more thought.

[josephjclark]

It might be a distraction but I want to go back and look at the example in #582

It still doesn't serialize well (in prod or this branch), and just shows the error as [object Object]

But when I look at the stack trace, there's no stack in the job code, so we can't extract any error information.

I think it's something to do with the axios callback - the error occurs in an async callstack, hence there's nothing useful in the stack to report. Is there anything we can everdo about this?

Btw to repro the error (and get a real error message out), do this:

import { CookieJar } from 'tough-cookie';

const cookiejar = new CookieJar();
cookiejar.getCookies({});

This goes away in later http versions, but I'm still worried about / interested in this async error case.

[josephjclark]

Errors which occur within the job code directly are currently working well.

But I'm struggling for errors within adaptor code, and that's a real problem.

Basically when an error is thrown from inside an adaptor, the stack trace does not include the vm sandbox. So you get a stack like this:

Error: UNEXPECTED_RELATIVE_URL
    at assertUrl (/home/joe/repo/openfn/cli-repo/node_modules/@openfn/language-http_6.5.1/dist/index.cjs:133:15)
    at /home/joe/repo/openfn/cli-repo/node_modules/@openfn/language-http_6.5.1/dist/index.cjs:167:5
    at file:///home/joe/repo/openfn/kit/packages/runtime/dist/index.js:630:24
    at async file:///home/joe/repo/openfn/kit/packages/runtime/dist/index.js:597:22 

What I rely on for source mapping is a frame inside the vm, like this:

ReferenceError: x is not defined
    at vm:module(0):3:27
    at /home/joe/repo/openfn/cli-repo/node_modules/@openfn/language-common_2.1.1/dist/index.cjs:333:12
    at file:///home/joe/repo/openfn/kit/packages/runtime/dist/index.js:628:24
    at async file:///home/joe/repo/openfn/kit/packages/runtime/dist/index.js:595:22

Where that vm:module frame is the job code position

I don't really understand why the vm frames are omitted from the stack, but it'll be something to do with how the node vm module works I'm sure. I'll have to get my thinking cap on about this.