Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(api-server): improve tests #9325

Merged
merged 9 commits into from
Oct 24, 2023
Merged

chore(api-server): improve tests #9325

merged 9 commits into from
Oct 24, 2023

Conversation

jtoar
Copy link
Contributor

@jtoar jtoar commented Oct 21, 2023

This PR improves the tests for the @redwoodjs/api-server package. In doing this, I deliberately didn't edit any sources files (save one edit to export an object for testing).

My goal was basically to test everything I could. I'm going to be deduplicating code across the framework (a lot of the code in the api-server package was duplicated to avoid breaking anything while iterating on experimental features), and since users can invoke api-server code in a few different ways (npx @redwoodjs/api-server, yarn rw serve, and importing handlers from @redwoodjs/api-server), I don't want to accidentally break anything anywhere. Improving test coverage also revealed some things that need to be fixed.

A few high-level notes. Overall, I reduced our reliance on mocking (in many tests, we weren't using fastify at all). Most tests covered configureFastify and while a few checked to see that routes were registered correctly, none of them actually started the api-server, or used any of fastify's testing facilities like inject to check that the right file was sent back, etc. (Now they do!) Lastly, while most of the tests I added assume fastify, the dist.test.ts isn't and is more like a black-box test.

Re the "revealed some things that need to be fixed", in no particular order:

  • apiUrl and apiHost can be a bit confusing. Even the previous withApiProxy.test.ts got them wrong

    While this test checked if the options were passed correctly, it mixed up the apiUrl for the apiHost. apiHost is the upstream (and should be a URL).

    await withApiProxy(mockedFastifyInstance as unknown as FastifyInstance, {
    apiUrl: 'http://localhost',
    apiHost: 'my-api-host',
    })

    @Josh-Walker-GM alerted me to this some time ago, but I didn't understand the nuance. Now that I do, these are just poorly named and we should try to fix them.

  • Everything in web dist is served

    // Serve other non-html assets
    fastify.register(fastifyStatic, {
    root: getPaths().web.dist,
    })

    We may want to revisit this. I'm sure most of it needs to be served, but it seems like a poor default when it comes to security. Also, it just results in routes being served that don't work. E.g., all prerendered routes are available at their .html suffix, but they error out as soon as they're rendered.

  • The lambda adapter seems a little too simple

    const lambdaEventForFastifyRequest = (
    request: FastifyRequest
    ): APIGatewayProxyEvent => {
    return {
    httpMethod: request.method,
    headers: request.headers,
    path: request.urlData('path'),
    queryStringParameters: qs.parse(request.url.split(/\?(.+)/)[1]),
    requestContext: {
    requestId: request.id,
    identity: {
    sourceIp: request.ip,
    },
    },
    ...parseBody(request.rawBody || ''), // adds `body` and `isBase64Encoded`
    } as APIGatewayProxyEvent
    }

    There's a core fastify plugin for lambdas, and it's much more complicated than our implementation. Should we just use that? What's the difference between our implementation and theirs?

  • setLambdaFunctions should ignore functions that don't export a handler.

    It probably doesn't do any harm but it registers them as undefined:

    LAMBDA_FUNCTIONS[routeName] = handler
    if (!handler) {
    console.warn(
    routeName,
    'at',
    fnPath,
    'does not have a function called handler defined.'
    )
    }

  • The lambda adapter supports callback syntax

    const handlerCallback =
    (reply: FastifyReply) =>
    (error: Error, lambdaResult: APIGatewayProxyResult) => {
    if (error) {
    fastifyResponseForLambdaError(req, reply, error)
    return
    }
    fastifyResponseForLambdaResult(reply, lambdaResult)
    }
    // Execute the lambda function.
    // https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
    const handlerPromise = handler(
    event,
    // @ts-expect-error - Add support for context: https://github.com/DefinitelyTyped/DefinitelyTyped/blob/0bb210867d16170c4a08d9ce5d132817651a0f80/types/aws-lambda/index.d.ts#L443-L467
    {},
    handlerCallback(reply)
    )

    I don't think we need to support this anymore.

  • The CLI handlers don't have help and don't error out on unknown args

    It's not uncommon to use this CLI standalone so we should polish it.

    apiServerHandler(
    yargs(hideBin(process.argv)).options(apiCliOptions).parseSync()
    )

    $ yarn rw-server --help
    Options:
      --help     Show help                                                 [boolean]
      --version  Show version number                                       [boolean]
    
    $ yarn rw-server --foo --bar --baz
    Starting API and Web Servers...
    
  • You can't import @redwoodjs/api-server outside a Redwood project

    This one isn't relevant to users, but it made testing the built package more complicated than it needed to be. The issue is that getConfig is called as soon as the file is imported. And if you're not in a redwood project at that point, you're out of luck:

    port: { default: getConfig().web?.port || 8910, type: 'number', alias: 'p' },

@jtoar jtoar added the release:chore This PR is a chore (means nothing for users) label Oct 21, 2023
@jtoar jtoar added this to the chore milestone Oct 21, 2023
Copy link
Collaborator

@Josh-Walker-GM Josh-Walker-GM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a couple of really very minor comments that shouldn't hold this up. I hope they don't appear overly nit-picky as that was certainly not my intension.

This work to add these tests to understand better the effects of incoming changes is really great!

Edit: The only other thing I was going to mention was the location of the dist.test.ts file. At first it struck me as a little odd that it was at the root level rather than in src/ somewhere but I don't really have a strong opinion on where it should live so wherever you put it is good with me.

packages/api-server/dist.test.ts Outdated Show resolved Hide resolved
packages/api-server/src/__tests__/withWebServer.test.ts Outdated Show resolved Hide resolved
packages/api-server/src/__tests__/withWebServer.test.ts Outdated Show resolved Hide resolved
jtoar and others added 3 commits October 22, 2023 07:23
Co-authored-by: Josh GM Walker <56300765+Josh-Walker-GM@users.noreply.github.com>
@jtoar
Copy link
Contributor Author

jtoar commented Oct 22, 2023

Forgot to address your comment about the dist.test.ts file. The tests in that file use the built code in dist, not src, so that's why I called them "dist tests". I.e. But the more practical reason that they're run differently and not as part of the "src tests" is that they take a lot longer to run.

@jtoar jtoar merged commit 35c0eeb into main Oct 24, 2023
32 checks passed
@jtoar jtoar deleted the ds-api-server/improve-tests branch October 24, 2023 16:26
jtoar added a commit that referenced this pull request Oct 25, 2023
In #9325, I added a few
fixtures to the `@redwoodjs/api-server` package that include dist files.
`yarn build:clean` deletes all dist directories in all packages (the
glob is `packages/**/dist`), so `yarn rwfw project:sync` ended up
deleting twenty or so files. This PR makes `yarn build:clean` a little
smarter. I couldn't see a way to do it via the `rimraf` CLI, so I just
made a script.

This PR also increases the timeout in the server tests which were a
little flakey.
jtoar added a commit that referenced this pull request Oct 26, 2023
I added a new job to CI in
#9325 that tested all the
different serve entry points. It was a bit flakey so I increased its
timeouts in #9336.
Unfortunately it's still a bit flakey. Right now when these tests fail
they're creating confusion, and most of our changes lately don't affect
these tests. They're still valuable to me locally, and at least right
now I'm the only one making changes to api-server. So I'll just leave
the tasks directory as is and remove them from CI till I do a bit more
testing on their flakiness.
jtoar added a commit that referenced this pull request Nov 2, 2023
This PR improves the tests for the `@redwoodjs/api-server` package. In
doing this, I deliberately didn't edit any sources files (save one edit
to export an object for testing).

My goal was basically to test everything I could. I'm going to be
deduplicating code across the framework (a lot of the code in the
api-server package was duplicated to avoid breaking anything while
iterating on experimental features), and since users can invoke
api-server code in a few different ways (`npx @redwoodjs/api-server`,
`yarn rw serve`, and importing handlers from `@redwoodjs/api-server`), I
don't want to accidentally break anything anywhere. Improving test
coverage also revealed some things that need to be fixed.

A few high-level notes. Overall, I reduced our reliance on mocking (in
many tests, we weren't using fastify at all). Most tests covered
`configureFastify` and while a few checked to see that routes were
registered correctly, none of them actually started the api-server, or
used any of fastify's testing facilities like `inject` to check that the
right file was sent back, etc. (Now they do!) Lastly, while most of the
tests I added assume fastify, the `dist.test.ts` isn't and is more like
a black-box test.

Re the "revealed some things that need to be fixed", in no particular
order:

- `apiUrl` and `apiHost` can be a bit confusing. Even the previous
`withApiProxy.test.ts` got them wrong

While this test checked if the options were passed correctly, it mixed
up the `apiUrl` for the `apiHost`. `apiHost` is the upstream (and should
be a URL).

https://github.com/redwoodjs/redwood/blob/daaa1998837bdb6eaa42d9160292e781fadb3dc8/packages/api-server/src/__tests__/withApiProxy.test.ts#L51-L54

@Josh-Walker-GM alerted me to this some time ago, but I didn't
understand the nuance. Now that I do, these are just poorly named and we
should try to fix them.

- Everything in web dist is served

https://github.com/redwoodjs/redwood/blob/daaa1998837bdb6eaa42d9160292e781fadb3dc8/packages/api-server/src/plugins/withWebServer.ts#L50-L53

We may want to revisit this. I'm sure most of it needs to be served, but
it seems like a poor default when it comes to security. Also, it just
results in routes being served that don't work. E.g., all prerendered
routes are available at their `.html` suffix, but they error out as soon
as they're rendered.

- The lambda adapter seems a little too simple

https://github.com/redwoodjs/redwood/blob/daaa1998837bdb6eaa42d9160292e781fadb3dc8/packages/api-server/src/requestHandlers/awsLambdaFastify.ts#L11-L27

There's a [core fastify plugin for
lambdas](https://github.com/fastify/aws-lambda-fastify), and it's much
more complicated than our implementation. Should we just use that?
What's the difference between our implementation and theirs?

- `setLambdaFunctions` should ignore functions that don't export a
handler.

  It probably doesn't do any harm but it registers them as `undefined`:

https://github.com/redwoodjs/redwood/blob/daaa1998837bdb6eaa42d9160292e781fadb3dc8/packages/api-server/src/plugins/lambdaLoader.ts#L32-L40

- The lambda adapter supports callback syntax

https://github.com/redwoodjs/redwood/blob/daaa1998837bdb6eaa42d9160292e781fadb3dc8/packages/api-server/src/requestHandlers/awsLambdaFastify.ts#L71-L89

  I don't think we need to support this anymore.

- The CLI handlers don't have help and don't error out on unknown args

  It's not uncommon to use this CLI standalone so we should polish it.

https://github.com/redwoodjs/redwood/blob/daaa1998837bdb6eaa42d9160292e781fadb3dc8/packages/api-server/src/index.ts#L26-L28

  ```
  $ yarn rw-server --help
  Options:
--help Show help [boolean]
--version Show version number [boolean]
  ```

  ```
  $ yarn rw-server --foo --bar --baz
  Starting API and Web Servers...
  ```

- You can't import `@redwoodjs/api-server` outside a Redwood project

This one isn't relevant to users, but it made testing the built package
more complicated than it needed to be. The issue is that `getConfig` is
called as soon as the file is imported. And if you're not in a redwood
project at that point, you're out of luck:

https://github.com/redwoodjs/redwood/blob/daaa1998837bdb6eaa42d9160292e781fadb3dc8/packages/api-server/src/cliHandlers.ts#L24

---------

Co-authored-by: Josh GM Walker <56300765+Josh-Walker-GM@users.noreply.github.com>
jtoar added a commit that referenced this pull request Nov 2, 2023
In #9325, I added a few
fixtures to the `@redwoodjs/api-server` package that include dist files.
`yarn build:clean` deletes all dist directories in all packages (the
glob is `packages/**/dist`), so `yarn rwfw project:sync` ended up
deleting twenty or so files. This PR makes `yarn build:clean` a little
smarter. I couldn't see a way to do it via the `rimraf` CLI, so I just
made a script.

This PR also increases the timeout in the server tests which were a
little flakey.
jtoar added a commit that referenced this pull request Nov 2, 2023
I added a new job to CI in
#9325 that tested all the
different serve entry points. It was a bit flakey so I increased its
timeouts in #9336.
Unfortunately it's still a bit flakey. Right now when these tests fail
they're creating confusion, and most of our changes lately don't affect
these tests. They're still valuable to me locally, and at least right
now I'm the only one making changes to api-server. So I'll just leave
the tasks directory as is and remove them from CI till I do a bit more
testing on their flakiness.
@Josh-Walker-GM Josh-Walker-GM modified the milestones: chore, v8.0.0 Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release:chore This PR is a chore (means nothing for users)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants