NestJS integration tests with Test Containers

| Mar 5, 2024 min read
NestJS integration tests with Test Containers

The Backstory

Hey everyone,

I’ve been wrestling with a topic that’s been on my mind for quite some time now. Setting up my tests in NestJS left me scratching my head, wondering, “What’s next?” The resources I stumbled upon online were somewhat helpful but didn’t quite hit the mark for me. That being said, I did come across a couple of gems that I believe are worth your time: check out this article and this one. While they offer valuable insights, they still felt incomplete to me.

This got me thinking—shouldn’t there be a way to streamline database setup in NestJS tests, similar to what I’ve previously discussed in my Baeldung article? The idea of simplifying the testing process intrigued me, and I knew…

The Challenge

As I dove deeper into the setup, I outlined a few key objectives to streamline the testing process in NestJS, specifically focusing on database interactions. Here’s what I aimed for:

  1. Minimal Database Setup: The database should initialize and configure itself automatically, avoiding any elaborate setup procedures.
  2. Leveraging Testcontainers: This powerful tool should enable efficient service testing without the need for excessive stubbing or manual intervention.
  3. Seamless Testcontainer Shutdown: Once the testing context terminates, Testcontainers should gracefully exit as well.
  4. One-time DB Migrations: Database migrations ought to be executed just once at the start of the testing phase.
  5. Data Bootstrapping: There should be an option to execute an additional .sql script for pre-loading the database with specific data sets.

These goals aim to create a robust, flexible, and efficient testing environment for NestJS applications.

Codebase

Setting up the project with NestJS is our first step. Here’s how to get started:

  1. Install the NestJS CLI: Run npm i -g @nestjs/cli. The latest version as of writing is 10.0.0.
  2. Create a new project: Execute nest new project-name and follow the on-screen instructions.
  3. Add essential packages: We’ll need an ORM (TypeORM), a database driver (MySQL), and the Testcontainers library. Install them using:
npm install --save @nestjs/typeorm typeorm mysql2
npm install --save-dev testcontainers @testcontainers/mysql

Optionally, for easier configuration management, consider installing @nestjs/config:

npm install --save @nestjs/config

I prefer organizing my projects using domain-driven design, which I find more intuitive. By the end of this guide, you can expect your project structure to look like this:

├── README.md
├── migrations
│   └── 1709499839546-AddUserTeams.ts
├── nest-cli.json
├── package-lock.json
├── package.json
├── src
│   ├── account
│   │   ├── account.module.ts
│   │   ├── team.entity.ts
│   │   ├── team.repository.ts
│   │   ├── user.entity.ts
│   │   └── user.repository.ts
│   ├── app.controller.spec.ts
│   ├── app.controller.ts
│   ├── app.module.ts
│   ├── app.service.ts
│   ├── database
│   │   ├── database.config.ts
│   │   └── database.module.ts
│   └── main.ts
├── test
│   └── it
│       ├── account
│       │   └── account.it.spec.ts
│       ├── import.sql
│       ├── it.jest.json
│       ├── setup.ts
│       ├── teardown.ts
│       └── util.ts
├── tsconfig.build.json
└── tsconfig.json

First, define your database configuration in database.config.ts to hold essential connection details:

export interface DatabaseConfig {
  host: string;
  port: number;
  database: string;
  username: string;
  password: string;
  logging: boolean;
}

export const dbConfig = () => ({
  database: {
    host: process.env.DB_HOST || "localhost",
    port: process.env.DB_PORT || 3306,
    database: process.env.DB_DATABASE || "bytesmith",
    username: process.env.DB_USERNAME || "root",
    password: process.env.DB_PASSWORD || "my-secret-pw",
    logging: process.env.DB_LOGGING_ENABLED == "true" || false,
  } as DatabaseConfig,
});

Next, let’s define database.module.ts which holds the actual implementation required to setup TypeORM:

import { Module } from "@nestjs/common";
import { TypeOrmModule } from "@nestjs/typeorm";
import { ConfigModule, ConfigService } from "@nestjs/config";
import {DatabaseConfig, dbConfig} from "./database.config";

/**
 * Worth noting:
 * https://github.com/typeorm/typeorm/issues/4998
 */
@Module({
  imports: [
    TypeOrmModule.forRootAsync({
      imports: [
        ConfigModule.forRoot({
          load: [dbConfig],
          envFilePath: `.env.${process.env.NODE_ENV}`,
        }),
      ],
      useFactory: (configService: ConfigService) => ({
        type: "mysql",
        ...configService.get<DatabaseConfig>("database"),
        synchronize: false,
        autoLoadEntities: true,
        relationLoadStrategy: "join",
      }),
      inject: [ConfigService],
    }),
  ],
  controllers: [],
  providers: [],
})
export class DatabaseModule {}

I added a comment worth noting about some limitations with transformers and default values with TypeORM that caused me a headache while debugging what’s wrong ;) Above code we can consider a production code, so I provided some reasonable defaults, especially: synchronize: false. We also injected ConfigService and resolved db config internally which makes it easier to actually encapsulate configuration logic into meaningful units.

Let’s dive into implementing our initial entities for the project:

@Entity({
    name: "teams"
})
export class TeamEntity {

    @PrimaryGeneratedColumn()
    id: number

    @Column()
    name: string

    @OneToOne(() => UserEntity)
    @JoinColumn({name: "owner_id"})
    owner: UserEntity;

    @Column({
        name: "owner_id"
    })
    ownerId: number
}

Followed by the UserEntity:

@Entity({
    name: "users"
})
export class UserEntity {

    @PrimaryGeneratedColumn()
    id: number

    @Column()
    firstName: string

    @Column()
    lastName: string

    @Column()
    confirmed: boolean

    @ManyToOne(() => TeamEntity)
    @JoinColumn({ name: "team_id" })
    teamEntity: TeamEntity;

    @Column({
        name: "team_id"
    })
    teamId: number;
}

I prefer a lean approach to entities, keeping database specifics minimal within them. This practice makes maintenance easier and keeps DB logic where it belongs—in the DB configuration. My experience with ORM tools (yes, Hibernate, I’m talking about you too) is that I find bugs in implementations in the least expected moments.

Next, let’s talk about repository classes. There’s some ambiguity in utilizing the Repository pattern effectively with NestJS and TypeORM. Official TypeORM documentation page provides an example that feels outdated in 2024:

// user.repository.ts
export const UserRepository = dataSource.getRepository(User).extend({
    findByName(firstName: string, lastName: string) {
        return this.createQueryBuilder("user")
            .where("user.firstName = :firstName", { firstName })
            .andWhere("user.lastName = :lastName", { lastName })
            .getMany()
    },
})

// user.controller.ts
export class UserController {
    users() {
        return UserRepository.findByName("Timber", "Saw")
    }
}

The previous @EntityCustomRepository(UserEntity) annotation is no longer available. However, echoing Joey’s sentiment, there must be a better solution. Indeed, I had to forge my own path to discover it, leading to my first (and possibly only so far) verified Stack Overflow answer.

Simplicity often leads to the best solutions. Interestingly, I stumbled upon a similar concept in the NestJS documentation, albeit they referred to it as a “service”, which seemed a bit off to me. Nevertheless, this inspired me to develop a workaround for the custom repository conundrum:

@Injectable()
export class TeamRepository extends Repository<TeamEntity>{
    constructor(
        @InjectRepository(TeamEntity)
            repository: Repository<TeamEntity>,
    ) {
        super(repository.target, repository.manager, repository.queryRunner);
    }
}

Analogically, a UserRepository is also essential to our architecture. With the repositories in place, our next step involves setting up the initial migration. It’s crucial to structure this migration using multiple statements. This approach enables us to establish a bidirectional relationship between teams and their owners (users), ensuring data integrity and facilitating easier data management:

import {MigrationInterface, QueryRunner} from "typeorm";

export class AddUserTeams1709499839546 implements MigrationInterface {
    public async up(queryRunner: QueryRunner): Promise<void> {
        await queryRunner.query(`
            CREATE TABLE IF NOT EXISTS users (
                id INT AUTO_INCREMENT PRIMARY KEY,
                firstName VARCHAR(255) NOT NULL,
                lastName VARCHAR(255) NOT NULL,
                confirmed BOOLEAN NOT NULL,
                team_id INT,
                INDEX IDX_team_id (team_id)
            ) ENGINE=InnoDB;
        `);

        await queryRunner.query(`
            CREATE TABLE IF NOT EXISTS teams (
                id INT AUTO_INCREMENT PRIMARY KEY,
                name VARCHAR(255) NOT NULL
            ) ENGINE=InnoDB;
        `);

        await queryRunner.query(`
            ALTER TABLE teams
            ADD COLUMN owner_id INT,
            ADD CONSTRAINT FK_teams_owner_id_users FOREIGN KEY (owner_id) REFERENCES users(id) ON DELETE SET NULL ON UPDATE CASCADE,
            ADD UNIQUE INDEX IDX_owner_id (owner_id);
        `);

        await queryRunner.query(`
            ALTER TABLE users
            ADD CONSTRAINT FK_users_team_id_teams FOREIGN KEY (team_id) REFERENCES teams(id) ON DELETE SET NULL ON UPDATE CASCADE;
        `);
    }

    public async down(queryRunner: QueryRunner): Promise<void> {
        await queryRunner.query("ALTER TABLE users DROP FOREIGN KEY FK_users_team_id_teams;");
        await queryRunner.query("DROP TABLE IF EXISTS users;");
        await queryRunner.query("DROP TABLE IF EXISTS teams;");
    }
}

To use the repositories, we need to add TypeORM to our account.module. I like keeping configurations tidy and contained, so that’s what we’ll do:

@Module({
    imports:[
        TypeOrmModule.forFeature(
            [
                UserEntity,
                TeamEntity
            ]
        )
    ],
    providers: [
        UserRepository,
        TeamRepository,
    ]
})
export class AccountModule {
}

Then in app.module.ts we can simply do:

@Module({
    imports: [
        DatabaseModule,
        AccountModule
    ],
    controllers: [AppController],
    providers: [AppService],
})
export class AppModule {
}

Test setup

As I mentioned before, my setup for tests looks like this:

.
└── it
    ├── account
    │   └── account.it.spec.ts
    ├── import.sql
    ├── it.jest.json
    ├── setup.ts
    ├── teardown.ts
    └── util.ts

Now, we’re getting to a crucial part of our setup. Jest lets us set up and tear down our tests easily. We do this by specifying setup and teardown files in the jest.json config, using default export functions:

{
  "moduleFileExtensions": [
    "js",
    "json",
    "ts"
  ],
  "rootDir": "../..",
  "testEnvironment": "node",
  "testRegex": ".it.spec.ts$",
  "transform": {
    "^.+\\.(t|j)s$": "ts-jest"
  },
  "globalSetup": "./test/it/setup.ts",
  "globalTeardown": "./test/it/teardown.ts"
}

Let’s dive into how we can use globalSetup to kick off test containers and set up our environment:

import {DataSource} from "typeorm";
import * as fs from "fs";
import {MySqlContainer} from "@testcontainers/mysql";
import {getDatasource} from "./util";

const init = async () => {
    await Promise.all([
        initMysql()
    ]);
};

const initMysql = async () => {
    const mysql = await new MySqlContainer("mysql:8")
        .withDatabase("bytesmith")
        .withUser("root")
        .withRootPassword("my-secret-pw")
        .start();

    global.mysql = mysql;

    process.env.DB_HOST = mysql.getHost();
    process.env.DB_PORT = mysql.getPort().toString();
    process.env.DB_USERNAME = mysql.getUsername();
    process.env.DB_PASSWORD = mysql.getUserPassword();
    process.env.DB_DATABASE = mysql.getDatabase();
    process.env.DB_LOGGING_ENABLED = "true";

    const datasource = await getDatasource();
    await datasource.runMigrations();
    await insertTestData(datasource);
};

const insertTestData = async (datasource: DataSource) => {
    const importSql = fs.readFileSync("./test/it/import.sql").toString();
    for (const sql of importSql.split(";").filter((s) => s.trim() !== "")) {
        await datasource.query(sql);
    }
};

export default init;

In the init function, you’re not limited to just setting up a database. For example, in another project, we successfully integrated localstack for AWS testing alongside the database.

Within the initMysql function, after spinning up a Testcontainers database, I attach its instance to the global object for easy teardown later. This setup is handy for any pre-launch tasks like running migrations, which can be handled through TypeORM’s methods:

export const getDatasource = async () => {
    if (datasource) {
        return datasource;
    }
    datasource = new DataSource({
        type: "mysql",
        host: process.env.DB_HOST,
        port: Number(process.env.DB_PORT),
        database: process.env.DB_DATABASE,
        username: process.env.DB_USERNAME,
        password: process.env.DB_PASSWORD,
        migrations: [`migrations/*`],
        logging: true,
        entities: [`**/*.entity.ts`],
        relationLoadStrategy: "join",
    });
    await datasource.initialize();
    return datasource;
};

This function relies on the environment variables we set earlier. Plus, we have the option to run more queries from import.sql if needed, but that’s up to you.

Moving on to wrapping things up, let’s look at how we manage the teardown process:

import { getDatasource } from "./util";

const teardown = async () => {
  await global.mysql.stop();
  await (await getDatasource()).destroy();
};

export default teardown;

In this step, we’ll stop the test containers and clean up by destroying the previously created datasource.

With everything in place, it’s time to craft a straightforward test case:

import {AccountModule} from "../../../src/account/account.module";
import {DatabaseModule} from "../../../src/database/database.module";
import {Logger} from "@nestjs/common";
import {Test} from "@nestjs/testing";
import {UserRepository} from "../../../src/account/user.repository";
import {TeamRepository} from "../../../src/account/team.repository";
import {v4 as uuid} from "uuid"

describe("Should create user with team", () => {
    let userRepository: UserRepository
    let teamRepository: TeamRepository

    beforeAll(async () => {
        const moduleRef = await Test.createTestingModule({
            imports: [
                AccountModule,
                DatabaseModule
            ],
        })
            .compile();
        moduleRef.useLogger(new Logger());
        userRepository = moduleRef.get(UserRepository);
        teamRepository = moduleRef.get(TeamRepository);
    });

    it("Should create simple team", async () => {
        // given - team name
        const teamName = uuid()

        //when
        const team = await teamRepository.save(
            {
                name: teamName,
            }
        );

        //then
        const teams = await teamRepository.find({
            where: {
                name: teamName
            }
        })
        expect(teams).toHaveLength(1);
    });
});

One last thing is to configure a script in package.json pointing to correct configuration file:

"test:it": "jest --config ./test/it/it.jest.json"

Now you can run it with npm run test:it and enjoy better quality tests :)

Conclusion

And that’s it! I’ve dialed in my test setup to where it needs to be: efficient and fast. That’s a game-changer for me.

Check out the full code in my GitHub repository if you’re curious to see it all come together.

Thanks for hanging in there till the end. Your support means a lot.

Cheers,

Jed