Skip to content

Commit

Permalink
feat (postgres): support for creating and restoring Snapshots (testco…
Browse files Browse the repository at this point in the history
…ntainers#2199)

* Add tips and examples to the documentation

I recently went through building a complex testing setup for an entire webservice using testcontainers-go, which taught me some important tips and tricks about the library. I have added these in the documentation wherever I thought they could be useful.

I also added a new example for using the postgres module and making each test use a clean database. It's a pretty useful tool, and it's available as an example in the Java version, so I've added it here as well.

* Added the example to the sidebar

* Move everything to the module

* Remove testing stuff I added to debug the integration

* Apply review feedback and rename reset to restore since it makes more sense in context

* Update modules/postgres/postgres.go

Co-authored-by: Manuel de la Peña <social.mdelapenya@gmail.com>

* Add test examples for file copy and update the docs

* Add the wait for hello file to help understanding the example

* chore: use pinned version of the image

* chore: simplify waiting for an expected log

* chore: check for errors in tests

* docs: use dot

* docs: simplify tabs

* chore: simplify variables and paths

* docs: remove misleading comment

* docs: move customise requests to each module

* docs: swap places

* chore: less verbose

* chore: use testify assertions

* Revert "chore: use testify assertions"

This reverts commit b7d6e90.

---------

Co-authored-by: Manuel de la Peña <social.mdelapenya@gmail.com>
Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
  • Loading branch information
3 people authored Feb 14, 2024
1 parent 54b1d8a commit 48fc228
Show file tree
Hide file tree
Showing 12 changed files with 528 additions and 85 deletions.
213 changes: 213 additions & 0 deletions docker_files_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
package testcontainers

import (
"context"
"path/filepath"
"testing"
"time"

"github.com/stretchr/testify/require"

"github.com/testcontainers/testcontainers-go/wait"
)

func TestCopyFileToContainer(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()

// copyFileOnCreate {
absPath, err := filepath.Abs(filepath.Join(".", "testdata", "hello.sh"))
if err != nil {
t.Fatal(err)
}

container, err := GenericContainer(ctx, GenericContainerRequest{
ContainerRequest: ContainerRequest{
Image: "docker.io/bash",
Files: []ContainerFile{
{
HostFilePath: absPath,
ContainerFilePath: "/hello.sh",
FileMode: 0o700,
},
},
Cmd: []string{"bash", "/hello.sh"},
WaitingFor: wait.ForLog("done"),
},
Started: true,
})
// }

require.NoError(t, err)
require.NoError(t, container.Terminate(ctx))
}

func TestCopyFileToRunningContainer(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()

// Not using the assertations here to avoid leaking the library into the example
// copyFileAfterCreate {
waitForPath, err := filepath.Abs(filepath.Join(".", "testdata", "waitForHello.sh"))
if err != nil {
t.Fatal(err)
}
helloPath, err := filepath.Abs(filepath.Join(".", "testdata", "hello.sh"))
if err != nil {
t.Fatal(err)
}

container, err := GenericContainer(ctx, GenericContainerRequest{
ContainerRequest: ContainerRequest{
Image: "docker.io/bash:5.2.26",
Files: []ContainerFile{
{
HostFilePath: waitForPath,
ContainerFilePath: "/waitForHello.sh",
FileMode: 0o700,
},
},
Cmd: []string{"bash", "/waitForHello.sh"},
},
Started: true,
})
if err != nil {
t.Fatal(err)
}

err = container.CopyFileToContainer(ctx, helloPath, "/scripts/hello.sh", 0o700)
// }

require.NoError(t, err)

// Give some time to the wait script to catch the hello script being created
err = wait.ForLog("done").WithStartupTimeout(200*time.Millisecond).WaitUntilReady(ctx, container)
require.NoError(t, err)

require.NoError(t, container.Terminate(ctx))
}

func TestCopyDirectoryToContainer(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()

// Not using the assertations here to avoid leaking the library into the example
// copyDirectoryToContainer {
dataDirectory, err := filepath.Abs(filepath.Join(".", "testdata"))
if err != nil {
t.Fatal(err)
}

container, err := GenericContainer(ctx, GenericContainerRequest{
ContainerRequest: ContainerRequest{
Image: "docker.io/bash",
Files: []ContainerFile{
{
HostFilePath: dataDirectory,
// ContainerFile cannot create the parent directory, so we copy the scripts
// to the root of the container instead. Make sure to create the container directory
// before you copy a host directory on create.
ContainerFilePath: "/",
FileMode: 0o700,
},
},
Cmd: []string{"bash", "/testdata/hello.sh"},
WaitingFor: wait.ForLog("done"),
},
Started: true,
})
// }

require.NoError(t, err)
require.NoError(t, container.Terminate(ctx))
}

func TestCopyDirectoryToRunningContainerAsFile(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()

// copyDirectoryToRunningContainerAsFile {
dataDirectory, err := filepath.Abs(filepath.Join(".", "testdata"))
if err != nil {
t.Fatal(err)
}
waitForPath, err := filepath.Abs(filepath.Join(dataDirectory, "waitForHello.sh"))
if err != nil {
t.Fatal(err)
}

container, err := GenericContainer(ctx, GenericContainerRequest{
ContainerRequest: ContainerRequest{
Image: "docker.io/bash",
Files: []ContainerFile{
{
HostFilePath: waitForPath,
ContainerFilePath: "/waitForHello.sh",
FileMode: 0o700,
},
},
Cmd: []string{"bash", "/waitForHello.sh"},
},
Started: true,
})
require.NoError(t, err)

// as the container is started, we can create the directory first
_, _, err = container.Exec(ctx, []string{"mkdir", "-p", "/scripts"})
require.NoError(t, err)

// because the container path is a directory, it will use the copy dir method as fallback
err = container.CopyFileToContainer(ctx, dataDirectory, "/scripts", 0o700)
if err != nil {
t.Fatal(err)
}
// }

require.NoError(t, err)
require.NoError(t, container.Terminate(ctx))
}

func TestCopyDirectoryToRunningContainerAsDir(t *testing.T) {
ctx, cnl := context.WithTimeout(context.Background(), 30*time.Second)
defer cnl()

// Not using the assertations here to avoid leaking the library into the example
// copyDirectoryToRunningContainerAsDir {
waitForPath, err := filepath.Abs(filepath.Join(".", "testdata", "waitForHello.sh"))
if err != nil {
t.Fatal(err)
}
dataDirectory, err := filepath.Abs(filepath.Join(".", "testdata"))
if err != nil {
t.Fatal(err)
}

container, err := GenericContainer(ctx, GenericContainerRequest{
ContainerRequest: ContainerRequest{
Image: "docker.io/bash",
Files: []ContainerFile{
{
HostFilePath: waitForPath,
ContainerFilePath: "/waitForHello.sh",
FileMode: 0o700,
},
},
Cmd: []string{"bash", "/waitForHello.sh"},
},
Started: true,
})
require.NoError(t, err)

// as the container is started, we can create the directory first
_, _, err = container.Exec(ctx, []string{"mkdir", "-p", "/scripts"})
require.NoError(t, err)

err = container.CopyDirToContainer(ctx, dataDirectory, "/scripts", 0o700)
if err != nil {
t.Fatal(err)
}
// }

require.NoError(t, err)
require.NoError(t, container.Terminate(ctx))
}
20 changes: 20 additions & 0 deletions docs/features/common_functional_options.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,3 +86,23 @@ If you need an advanced configuration for the container, you can leverage the fo
- `testcontainers.WithEndpointSettingsModifier`

Please read the [Create containers: Advanced Settings](/features/creating_container.md#advanced-settings) documentation for more information.

#### Customising the ContainerRequest

This option will merge the customized request into the module's own `ContainerRequest`.

```go
container, err := RunContainer(ctx,
/* Other module options */
testcontainers.CustomizeRequest(testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Cmd: []string{"-c", "log_statement=all"},
},
}),
)
```

The above example is updating the predefined command of the image, **appending** them to the module's command.

!!!info
This can't be used to replace the command, only to append options.
76 changes: 22 additions & 54 deletions docs/features/files_and_mounts.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,48 +16,26 @@ It is possible to map a Docker volume into the container using the `Mounts` attr
!!!warning
Bind mounts are not supported, as it could not work with remote Docker hosts.

!!!tip
It is recommended to copy data from your local host machine to a test container using the file copy API
described below, as it is much more portable.

## Copying files to a container

If you would like to copy a file to a container, you can do it in two different manners:

1. Adding a list of files in the `ContainerRequest`, which will be copied before the container starts:

```go
ctx := context.Background()

nginxC, err := GenericContainer(ctx, GenericContainerRequest{
ContainerRequest: ContainerRequest{
Image: "nginx:1.17.6",
ExposedPorts: []string{"80/tcp"},
WaitingFor: wait.ForListeningPort("80/tcp"),
Files: []ContainerFile{
{
HostFilePath: "./testdata/hello.sh",
ContainerFilePath: "/copies-hello.sh",
FileMode: 0o700,
},
},
},
Started: false,
})
```
<!--codeinclude-->
[Copying a list of files](../../docker_files_test.go) inside_block:copyFileOnCreate
<!--/codeinclude-->

2. Using the `CopyFileToContainer` method on a `running` container:

```go
ctx := context.Background()

nginxC, err := GenericContainer(ctx, GenericContainerRequest{
ContainerRequest: ContainerRequest{
Image: "nginx:1.17.6",
ExposedPorts: []string{"80/tcp"},
WaitingFor: wait.ForListeningPort("80/tcp"),
},
Started: true,
})

nginxC.CopyFileToContainer(ctx, "./testdata/hello.sh", "/hello_copy.sh", 0o700)
```
<!--codeinclude-->
[Copying files to a running container](../../docker_files_test.go) inside_block:copyFileAfterCreate
[Wait for hello](../../testdata/waitForHello.sh)
<!--/codeinclude-->

## Copying directories to a container

Expand All @@ -67,30 +45,20 @@ It's important to notice that, when copying the directory to the container, the

You can leverage the very same mechanism used for copying files to a container, but for directories.:

1. The first way is using the `Files` field in the `ContainerRequest` struct, as shown in the previous section, but using the path of a directory as `HostFilePath`.
1. The first way is using the `Files` field in the `ContainerRequest` struct, as shown in the previous section, but using the path of a directory as `HostFilePath`. Like so:

<!--codeinclude-->
[Copying a directory using files](../../docker_files_test.go) inside_block:copyDirectoryToContainer
<!--/codeinclude-->

2. The second way uses the existing `CopyFileToContainer` method, which will internally check if the host path is a directory, calling the `CopyDirToContainer` method if needed:

```go
ctx := context.Background()
// as the container is started, we can create the directory first
_, _, err = myContainer.Exec(ctx, []string{"mkdir", "-p", "/usr/lib/my-software/config"})
// because the container path is a directory, it will use the copy dir method as fallback
err = myContainer.CopyFileToContainer(ctx, "./files", "/usr/lib/my-software/config/files", 0o700)
if err != nil {
// handle error
}
```
<!--codeinclude-->
[Copying a directory to a running container](../../docker_files_test.go) inside_block:copyDirectoryToRunningContainerAsFile
<!--/codeinclude-->

3. The last third way uses the `CopyDirToContainer` method, directly, which, as you probably know, needs the existence of the parent directory in order to copy the directory:

```go
ctx := context.Background()

// as the container is started, we can create the directory first
_, _, err = nginxC.Exec(ctx, []string{"mkdir", "-p", "/usr/lib/my-software/config"})
err = nginxC.CopyDirToContainer(ctx, "./plugins", "/usr/lib/my-software/config/plugins", 0o700)
if err != nil {
// handle error
}
```
<!--codeinclude-->
[Copying a directory to a running container](../../docker_files_test.go) inside_block:copyDirectoryToRunningContainerAsDir
<!--/codeinclude-->
7 changes: 7 additions & 0 deletions docs/features/override_container_command.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ req := ContainerRequest{
}
```

!!!info
If you are using a module, you can use the `testcontainers.CustomizeRequest` option to add arguments to the command. Check the individual module's pages for more information on their commands.

This option will merge the customized request into the module's request, appending any additional `Cmd` arguments to the
module's command. This can't be used to replace the command, only to append options.
Check the individual module's pages for more information on their commands.

## Executing a command

You can execute a command inside a running container, similar to a `docker exec` call:
Expand Down
3 changes: 3 additions & 0 deletions docs/features/wait/http.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ The HTTP wait strategy will check the result of an HTTP(S) request against the c
- the poll interval to be used in milliseconds, default is 100 milliseconds.
- the basic auth credentials to be used.

!!!info
It's important to notice that the HTTP wait strategy will default to the first port exported/published by the image.

Variations on the HTTP wait strategy are supported, including:

## Match an HTTP method
Expand Down
11 changes: 11 additions & 0 deletions docs/modules/postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,3 +91,14 @@ It's possible to use the Postgres container with Timescale or Postgis, to name a
<!--codeinclude-->
[Image for Postgis](../../modules/postgres/postgres_test.go) inside_block:postgis
<!--/codeinclude-->
## Examples
### Using Snapshots
This example shows the usage of the postgres module's Snapshot feature to give each test a clean database without having
to recreate the database container on every test or run heavy scripts to clean your database. This makes the individual
tests very modular, since they always run on a brand-new database.

<!--codeinclude-->
[Test with a reusable Postgres container](../../modules/postgres/postgres_test.go) inside_block:snapshotAndReset
<!--/codeinclude-->
Loading

0 comments on commit 48fc228

Please sign in to comment.