Docker Database Integration Testing

 

Background

One of the the most important things in the Software Engineering world is the ability to automate your tests properly against your database, and by this I mean your actual database. Something that can startup your database for you incase it's not running and then also run the tests and finally shutdown your database when the tests are done. 

In my experience I have seen developers introduce a bit of manual work to get the following going : 

• Starting up the database before running tests.
• Clearing out database manually when done or writing something to do that.
• Sometimes, and I mean sometimes, shutting down the database when done with the tests.

One of the great solutions to this problem, is to use some nice "In-Memory" databases. This is great and has helped with automation of the key points mentioned above. No need to start the database and clear it out manually after running tests etc  ...

Now in most cases the in-memory databases don't necessarily match up to the actual database you will be running in the Development, QA and Production environments, i.e, when using something like H2 IN -Memory Database, then we need to keep in mind that it's not really the MSSQL Server or Postgres DB that you are running on the actual environments. This means that you may be limited when testing something more intense, complex and database specific processes and operations. 

Would it not be nicer to be able to test out against an actual database that we running on the various environments? I think it would be awesome. 

So then one day I was with my Chief Architect & CEO discussing about putting a small system together, all the way from tech stack selection, frameworks and to do some RND. I was excited that I will finally get to put this tool to practice and see how it works. I wanted something that can have much like the In-Memory databases except it should be a real database we will be running on. 

Things To Keep In Mind

Before we get started. If you will be checking out the article's code repository for reference then may be set up these tools before: 

• Java 11
• Maven 3.6.2
• Docker

Otherwise this depends on some assumptions : 

• You are familiar with docker.
• You already have a Java project that already has datasource connection component. 
• Some understanding of maven.
• You have worked with JUnit, in this case we are talking about JUnit 5.

Context

Now that you are ready with the tools needed, we are going to go through a library named, Test Containers. This is a nice light weight Docker API of sorts. Through the power of JUnit 5 we will also be able to startup and shutdown the database automatically. The database for this article is Postgres. Test Containers supports a lot of database services, so you are not tied to Postgres database. Remember this also uses Docker meaning you can pretty much use Test Containers for anything else besides a database service, further more you can even play with some Docker Compose files. So this really opens a whole you avenue of possibilities in the Software Engineering world.

Getting Started

Adding Dependencies

If you check that home page there's a Gradle equivalent that you can try of you are using it. In our case you will need : 

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgresql</artifactId>
    <version>1.14.3</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers</artifactId>
    <version>1.14.3</version>
    <scope>test</scope>
</dependency>

Docker

For this to work you need docker to be running, then you don't have to worry about : 

• The actual database service ...
• Nor the database container ...
• Or even, the container image. 

Test Containers will sort out the rest for you, even if you don't have the image on your docker. 

Datasource Configuration

It's as simple as using the "Default" Test Container datasource configurations : 

URL : jdbc:tc:postgresql:13:///test

Username : test

Password : test

Drive Class Name : org.testcontainers.jdbc.ContainerDatabaseDriver

Notice the "tc" inside the URL literal. That's how you know that it's a Test Containers url. The default database name is "test" by default, if you check the end of the url literal. This is the same with username and password. 

So then we are almost there. Believe or not, that's all you need. Now the final piece. Some Java JUnit 5 code to test out the magic. 

Java & JUnit

We are going to use the JUnit - @ClassRule, annotation to dow some work before before the the rest of the actual test loads. This is similar to the the @BeforeClass annotation. The reason we will be using this is to kick start our Docker container before the tests even get to run. So it's preparation for the actual test cases. So create a new test case for you existing Data Access code. Add this class field or variable in your code.

 @ClassRule
public static PostgreSQLContainer postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer("postgres:13");

Write some sample code to test this out. In my case I have something along the lines of ...

// Some code here
...

@Test
public void assert_That_We_Can_Save_A_Book() {
    Book saved_lordOfTheRings_001 = bookDataAccess.saveAndFlush( lordOfTheRings_001);
    assertNotNull( saved_lordOfTheRings_001 );
    assertNotNull( saved_lordOfTheRings_001.getId() );
    assertEquals( saved_lordOfTheRings_001.getTitle(), lordOfTheRings_001.getTitle() );
}

So that's pretty much it. You can run your test cases and it should integrate into your database and store some information. We try to test this out. 

Validating The Data Graphically

So we can look into it by debugging our test case with a break point immediately after the line that saves a database record and saves. 

Start up any docker client of your choice and then. At this point you will not have have any Postgres docker container running unless you already were using Postgres in docker. The name of the container that will be run by your test cases will be a random name, there's no way you will miss it plus you can check out the image version on your docker client i.e. postgres:13.

So this is my docker client before running the tests. By the way the docker client I am using is Portainer. 

So now we are going to run a test case and pause it just after saving to the database. 

As you can see, my debug break point is in place and has paused just after saving to the database. Now let's go back to our docker client interface, Portainer in my case... 

Notice the two new containers being created : 

• testcontainers-ryuk-40abb5dc-... : it the test containers service running in there. 
• eloquent_robinson : The more important one is your test database running, as you can see that it's a random name, notice the image, which is postgres:13 that's our guy right there. The port for this thread at this moment is 32769. 

The port number is important because it's also random like the container name. This is by design according to the Test Containers engineers. For this Test Case run we are going to try to connect to the database while the debug is still on. So that we see our saved data. So get on to your Postgress database client and connect using the following properties : 

• Username : test
• Password : test
• Database Name : test
• Port : 32769

The URL will be something like : jdbc:postgresql://localhost:32769/test

Connect and run a simple query statement to see your data, saved through that test case. You should get something like the one in the image below : 

Ka-BOOM! There you go. You have not only managed to run a successful integration test, but also validated that it does indeed save your data to the docker hosted database, as you expected. Done, done and done! 

In Closing

I hope this was fun and you have cases where this can help ease your software development processes.

Leave some comments below and you may checkout the GitHub Docker Database Test - Source Code for reference.

Java EE Micro Services Using Thorntail

Context

The Software Development world is growing every minute and as it evolves the systems are becoming more and more normalised and refined into Micro Services, this has also inspired the likes of Spring Boot and many other frameworks.

I have realised that when it comes to enterprise software most Java developers are either using Spring or  Java EE / Jakarta EE. For the who are using Spring we have Spring Boot to help us with fast project kick off and development. More to that some use it for the Micro Service pattern or methodology. What happens when you you as a developer want to use JEE? Spring Boot may not be what you are looking for and perhaps you are looking for something along the lines of WildFly.

We will have a look at a baby project along these lines, a new sidecar project supporting WildFly to enable deconstructing the application server as is and pasting just enough of it back together with your application to create a self-contained executable jar. This project is formerly known as "Wildly Swarm" and now known, as Thorntail. I mentioned Spring Boot and now you can imagine where this is going. 

1. Assumptions

Now remember that we spoke about Micro Services so the best use case for this article would be something along the lines of some sort of API. In this case we will create a RESTFul API. We assume that you have an understanding of the following tech / tools.

• WildFly AS
• Java 8 
• Maven
• JAX-RS

2. Project Setup Using Maven

Let's set up the project using maven, as our automated build tool and also  grab some wildly fractions (dependencies) we need. Worry not we will cover a little bit about fractions as we go. We will start off with some basic properties settings. These can be anything you prefer so feel free to change these properties up. Just the important one is that since this will be a web application we want to package it as a ".war" file, so keep that one as is in the example.

<groupId>za.co.anylytical.showcase</groupId>
<artifactId>thorntail-rest-services</artifactId>
<name>Thorntail REST API Showcase</name>
<version>1.0.0-SNAPSHOT</version>
<packaging>war</packaging>

We have more of our properties, mainly for our versions in this case. We are also now telling maven that our Java version will be 8. At the moment [07-03-2020] the latest version of Thorntail is "2.6.0.Final". So we will be using the latest version. As of Jboss 7.x we no longer really need the ".web.xml" so the tag  <failOnMissingWebXml>  also is there to make sure that we don't get failures as we will not be using the ".web.xml" file in this example. 

<properties>
    <version.thorntail>2.6.0.Final</version.thorntail>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <failOnMissingWebXml>false</failOnMissingWebXml>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

Maven being a very good and strong dependency management tool, it also has a very cool feature using BOM (Bill Of Materials) which pretty much allows us to access a set collection of various dependencies and hand pick the ones we need for our app. So in this case we will import the Thorntail specific BOM and then choose the dependencies we need for our JAX-RS API.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.thorntail</groupId>
            <artifactId>bom-all</artifactId>
            <version>${version.thorntail}</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
    </dependencies>
</dependencyManagement>

There's a Thorntail Maven Plugin maven plugin available that can help us work on packing our application much easier. Let's set that up next.

<build>
    <finalName>thorntail-api</finalName>
    <plugins>
        <plugin>
            <groupId>io.thorntail</groupId>
            <artifactId>thorntail-maven-plugin</artifactId>
            <version>${version.thorntail}</version>
            <executions>
                <execution>
                    <goals>
                        <goal>package</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Earlier in the article we mentioned "fractions". This is a concept based on Thorntail, a "fraction" can include all or none of the WildFly components. Ultimately a fraction contributes configuration or capabilities to a runtime system. This is simply because we are loading in smaller pieces that we need, thus, Fractions. Looking at our current topic, (Thorntail) these fractions are in the form of dependencies. Let's go shopping for some ingredients that make up our REST Service API.

<dependencies>
    <dependency>
        <groupId>io.thorntail</groupId>
        <artifactId>swagger</artifactId>
    </dependency>
    <dependency>
        <groupId>io.thorntail</groupId>
        <artifactId>jaxrs</artifactId>
    </dependency>
    <dependency>
        <groupId>io.thorntail</groupId>
        <artifactId>swagger-webapp</artifactId>
    </dependency>
</dependencies>

That's it for the project set up with maven. As you can see, the fractions we are pulling in are very basic and specific, not need for the entire heavy WildFly. We have hand picked only Swagger, together with its own web application piece, for the Web UI and finally the core piece we need for our REST API, JAX-RS. At this point we are now ready for some code implementation. 

3. Java REST Resource Implementation Using JAX-RS API

Let's now create a Rest resource implementation using Java and JAX-RS API. 

package za.co.anylytical.showcase.rest;
 
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
 
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
 
@Path("/text")
@Api( value = "/text", tags = "Text API")
@Produces( MediaType.APPLICATION_JSON)
public class SampleResource {

    @GET
    @ApiOperation(
            value = "Gets a text message",
            notes = "Returns the message as HTTP Resposne",
            response = Response.class)
    @Produces( MediaType.APPLICATION_JSON)
    public Response get() {
        return Response.status(Response.Status.OK)
                .entity( "Hey, there, you requested for text?")
                .build();
    }
}

According to the JAX-RS standards, the next step is to include a class extending javax.ws.rs.core.Application to define the rest @ApplicationPath and also register our rest resources. 

package za.co.anylytical.showcase.config;
 
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;
 
@ApplicationPath("/")
public class ResourceConfig extends Application {
 
}

We will be adding the second last piece to our implementation before we can test it out. We will include a class implementing the ContainerResponseFilter interface, which works like a filter for the ContainerResponse extension point on the server side, in order to filter the response message after the invocation has executed:

package za.co.anylytical.showcase.filters;
 
import java.io.IOException;
 
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerResponseContext;
import javax.ws.rs.container.ContainerResponseFilter;
import javax.ws.rs.ext.Provider;
 
 
@Provider
public class CORSFilter implements ContainerResponseFilter {
 
    @Override
    public void filter(ContainerRequestContext requestContext, ContainerResponseContext responseContext) throws IOException {
        responseContext.getHeaders().add("Access-Control-Allow-Origin", "*");
        responseContext.getHeaders().add("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS");
        responseContext.getHeaders().add("Access-Control-Max-Age", "-1");
        responseContext.getHeaders().add("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
    }
 
}

Let's configure our Wildfly port and context-root just because port 8080 is so over rated, plus we will be looking at an alternative to setting up, similar to the spring boot way with the application.properties or .yml file. In your source code resources folder, add a new file project-defaults.yml. 

Add the following piece of yml in your file. 

thorntail:
  http:
    port: 8881
  context:
    path: that-service

We are now all set, it's time to build our app and then try to run it and see. Let's talk to maven to see if it can help us bootstrap the rest service.

4. Run The Project Using Maven

Build the app. Make sure you are in the root project folder where you can run your pom file.

$ mvn clean install

Run the application, remember there's no standalone application server. So we are actually going to just execute our ".jar" file as mentioned in the beginning of this article. Let's get on with it then. 

$ java -jar target/thorntail-api-thorntail.jar

NB: At this point it will boot up very quick and you should see something along these lines. Just click the image below to enlarge it.

5. Swagger Checkout

So let's check the swagger out for some basic testing and checks. Now head over to the link http://localhost:8881/swagger-ui, and you should see a swagger screen with the default API loaded. 

NB: A quick look at the default swagger home page once you launch it. Just click the image below to enlarge it.

You probably want to check your API, so while you are on that swagger screen just insert the following URL into the top text box. http://localhost:8881/that-service/swagger.json and then press enter. This will load your API in the web app and you should be able to see your API as shown in the image below.

NB: Web view of what it looks like once you have pointed it to your API. Just click the image below to enlarge it.

Conclusion

So there you go. A standalone service using Wildfly and some Java EE, similar way you would have it using Spring Boot. We have learned how to build one and the nice part was that fractions section which allowed us to hand pick which bits of wildfly we would want. Sometimes working with a full fledged application server is necessarily bloated and heavy. Incase you wondered how you can work with Java EE in Micro Service then this is another way you can do it.  

Note: Postman example of our API test, incase you were wondering. 

You may also find the GitHub Code Base incase you want to double check our exercise. I hope this has been helpful, leave comments in the section below.