Quarkus Extension Custom Dev Service - Part I

LE BARO Romain included in Development

2024-10-08 About 2300 words 11 minutes

Contents

Today, we’re tackling a major topic with a use case that is not documented by Quarkus but is extremely useful. Quarkus offers extensions with “dev services.” Once included in your project, these extensions automatically launch services when the project starts in dev mode. We will see how to create our own “dev service” for Quarkus applications. The idea is to provide an extension for each of our services to make them easily usable by others.

Use Case

Imagine a company with multiple projects and several teams. It’s common for services to be shared between different projects. Two solutions are usually implemented:

A shared dev instance
A Docker Compose to be run alongside the project in dev mode.

The first solution involves complicated administration with data management across different teams. If one team breaks the service, it impacts the others. During an update, all teams are affected as well.

The second solution is more flexible, as the services run on the developer’s machine. However, it adds an extra step when starting the project in dev mode. It also raises the issue of updating the Docker Compose.

A “dev service” helps solve these problems. Distributed as a Maven extension, they can be kept up to date using a Maven BOM. Quarkus starts them automatically when the project launches, eliminating the need to manually manage Docker containers.

The Extension

Our extension will need to start a Quarkus service with its database. The extension should allow the service to start with default parameters and make them customizable.

Let’s start by creating the extension with the following command:

1
2
3


mvn io.quarkus.platform:quarkus-maven-plugin:3.14.4:create-extension -N \
    -DgroupId=info.techland \
    -DextensionId=demo-extension

This will create an extension with the following structure:

demo-extension
|__ deployment
|__ integration-tests
|__ runtime
|__ pom.xml

At the end, the architecture of our extension will look like the following diagram:

Diagram of our extension for the database.

Creating the Database for Our Dev Service

Our service needs a database to function. So we will configure our extension to launch a database in a Docker container. Before that, let’s create some configurations for our extension to make it customizable.

DemoExtensionBuildTimeConfig

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


package info.techland.demo.extension.deployment;

import io.quarkus.runtime.annotations.ConfigPhase;
import io.quarkus.runtime.annotations.ConfigRoot;
import io.smallrye.config.ConfigMapping;


@ConfigMapping(prefix = "quarkus.demo")
@ConfigRoot(phase = ConfigPhase.BUILD_TIME)
public interface DemoExtensionBuildTimeConfig {

    /**
     * Dev services configuration.
     */
    DevServicesConfig devservices();

}

This class registers the entry point of our dev service configuration. Our configuration will be accessible in the application.properties file under the quarkus.demo key.

DevServiceConfig

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59


package info.techland.demo.extension.deployment;

import java.util.Map;

import io.quarkus.runtime.annotations.ConfigGroup;
import io.smallrye.config.WithDefault;

@ConfigGroup
public interface DevServicesConfig {

    /**
     * If DevServices has been explicitly enabled or disabled. DevServices is generally enabled
     * by default, unless there is an existing configuration present.
     * <p>
     * When DevServices is enabled Quarkus will attempt to automatically configure and start
     * a demo instance when running in Dev or Test mode and when Docker is running.
     */
    @WithDefault("true")
    boolean enabled();


    /**
     * Indicates if the demo server managed by Quarkus Dev Services is shared.
     * When shared, Quarkus looks for running containers using label-based service discovery.
     * If a matching container is found, it is used, and so a second one is not started.
     * Otherwise, Dev Services for demo starts a new container.
     * <p>
     * The discovery uses the {@code quarkus-dev-service-redis} label.
     * The value is configured using the {@code service-name} property.
     * <p>
     * Container sharing is only used in dev mode.
     */
    @WithDefault("true")
    boolean shared();

    /**
     * The value of the {@code quarkus-dev-service-demo} label attached to the started container.
     * This property is used when {@code shared} is set to {@code true}.
     * In this case, before starting a container, Dev Services for demo looks for a container with the
     * {@code quarkus-dev-service-demo} label
     * set to the configured value. If found, it will use this container instead of starting a new one. Otherwise, it
     * starts a new container with the {@code quarkus-dev-service-demo} label set to the specified value.
     * <p>
     * This property is used when you need multiple shared demo servers.
     */
    @WithDefault("demo")
    String serviceName();

    /**
     * Environment variables that are passed to the container.
     */
    Map<String, String> containerEnv();

    /**
     * Configuration for the postgresql database use by demo service.
     * @return The database configuration
     */
    DatabaseConfig db();
}

This class configures our dev service with access to the database configuration of the dev service. The database configuration will be accessible under the quarkus.dev.db key. Each entry has a default value to allow the extension to be used without particular configuration.

DatabaseConfig

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53


package info.techland.demo.extension.deployment;

import io.quarkus.runtime.annotations.ConfigGroup;
import io.smallrye.config.WithDefault;

import java.util.Map;
import java.util.OptionalInt;

@ConfigGroup
public interface DatabaseConfig {

    /**
     * The database password used by the demo service.
     */
    @WithDefault("demo")
    String username();

    /**
     * The database password used by the demo service.
     */
    @WithDefault("demo")
    String password();

    /**
     * The database name used by the demo service
     */
    @WithDefault("demo")
    String name();

    /**
     * The number of reactive connection to use.
     */
    @WithDefault("20")
    int reactiveMaxSize();

    /**
     * The port used by the database.
     * @return The database port
     */
    OptionalInt port();

    /**
     * Name of the postgresql image to use to start the database
     * @return The image name.
     */
    @WithDefault("docker.io/postgres")
    String imageName();

    /**
     * Environment variables that are passed to the container.
     */
    Map<String, String> containerEnv();
}

The database configuration contains connection information for the PostgreSQL database used by default. Again, we favor convention over configuration by applying default values.

DemoExtensionProcessor

To ask Quarkus to start a container, we will need to configure our DemoExtensionProcessor. Let’s start by adding a createContainer method that will act as a @BuildStep to start and configure our container.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


  @BuildSteps(onlyIf = {GlobalDevServicesConfig.Enabled.class})
  public class DemoExtensionProcessor {
  
    /**
     * The current configuration. either from running container instance or an instance that is about to start.
     */
    private static volatile DevServicesConfig runningConfiguration;
    private static volatile RunningDevService demoDatabaseDevService;

    @BuildStep
    public void createContainer(DockerStatusBuildItem dockerStatusBuildItem,
                                List<DevServicesSharedNetworkBuildItem> devServicesSharedNetworkBuildItem,
                                DemoBuildTimeConfig buildTimeConfig,
                                GlobalDevServicesConfig devServicesConfig,
                                BuildProducer<DevServicesResultBuildItem> devServicesResult) {

      runningConfiguration = buildTimeConfig.devservices();

      try {
          List<String> errors = new ArrayList<>();
          DemoDatabaseRunningDevService newDatabaseDevService = startDatabaseContainer(
                  dockerStatusBuildItem,
                  !devServicesSharedNetworkBuildItem.isEmpty(),
                  devServicesConfig.timeout.orElse(Duration.of(0, ChronoUnit.SECONDS)),
                  runningConfiguration,
                  errors
          );


          // Get the RunningDevService instance.
          demoDatabaseDevService = newDatabaseDevService.getRunningDevService();
      } catch (Throwable t) {
          throw new RuntimeException(t);
      }
      Log.info("Dev Services for Demo started.");

      // Call to toBuildItem method to get the DevServicesResultBuildItem need by the BuildStep producer.
      devServicesResult.produce(demoDatabaseDevService.toBuildItem());
    }
  }

Our method is injected with the following build items:

DockerStatusBuildItem : Allows retrieving the status of a Docker instance.
DevServicesSharedNetworkBuildItem : Determines if containers are started on a shared network.
DemoBuildTimeConfig : Configuration of our extension.
GlobalDevServicesConfig : Global configuration for dev services provided by Quarkus.
DevServicesResultBuildItem : The build item to produce for referencing our container to Quarkus.

Our method then calls the startDatabaseContainer method, which is responsible for creating an instance of DemoDatabaseRunningDevService. This class provides a RunningDevService, via devServicesResult.produce(demoDatabaseDevService.toBuildItem()), which is returned by our @BuildStep to request Quarkus to launch a container.

startDatabaseContainer

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30



private DemoDatabaseRunningDevService startDatabaseContainer(DockerStatusBuildItem dockerStatusBuildItem, boolean useSharedNetwork, Duration timeout, DevServicesConfig devServicesConfig, List<String> errors) {
    if (!devServicesConfig.enabled()) {
        // explicitly disabled
        Log.debug("Not starting Dev Services for Demo as it has been disabled in the config");
        return null;
    }


    if (!dockerStatusBuildItem.isDockerAvailable()) {
        Log.warn("Please get a working docker instance");
        errors.add("Please get a working docker instance");
        return null;
    }

    final Optional<ContainerAddress> maybeExistingContainer = demoDBDevModeContainerLocator.locateContainer(
            devServicesConfig.serviceName() + "_db",
            devServicesConfig.shared(),
            LaunchMode.current());

    // Get the image name from the configuration.
    String imageName = devServicesConfig.db().imageName();
    DockerImageName dockerImageName = DockerImageName.parse(imageName).asCompatibleSubstituteFor(imageName);

    return maybeExistingContainer
            .map(existingContainer -> new DemoDatabaseRunningDevService(new RunningDevService(DemoDatabaseContainer.CONTAINER_NAME, existingContainer.getId(),
                    null, Map.of("url", existingContainer.getHost() + ":" + existingContainer.getPort()))))
            .orElseGet(() -> new DemoDatabaseRunningDevService(DemoDatabaseContainer.CONTAINER_NAME, useSharedNetwork, timeout, devServicesConfig,
                    dockerImageName, devServicesConfig.serviceName() + "_db"));
}

We start with a few checks. If dev services are globally disabled, the launch is canceled, and if Docker is unavailable, an error is returned. The method then checks if another container already exists. This may happen if two Quarkus services use the same extension. This is done using a ContainerLocator:

1

  private static final ContainerLocator demoDBDevModeContainerLocator = new ContainerLocator(DemoDatabaseContainer.DEV_SERVICE_LABEL, POSTGRES_INTERNAL_PORT);

If a database container is already running, the method returns a RunningDevService with the information from the existing container. Otherwise, a new RunningDevService is returned for the new container.

DemoDatabaseRunningDevService

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52


public class DemoDatabaseRunningDevService {

    private DemoDatabaseContainer databaseContainer;
    private final DevServicesResultBuildItem.RunningDevService runningDevServiceSupplier;

    public DemoDatabaseRunningDevService(String feature, boolean useSharedNetwork, Duration timeout, DevServicesConfig devServicesConfig, DockerImageName
            dockerImageName, String serviceName) {
        runningDevServiceSupplier = this.run(feature, useSharedNetwork, timeout, devServicesConfig, dockerImageName, serviceName);
    }

    public DemoDatabaseRunningDevService(DevServicesResultBuildItem.RunningDevService runningDevService) {
        runningDevServiceSupplier = runningDevService;
        databaseContainer = null;
    }

    public DevServicesResultBuildItem.RunningDevService  run(String feature, boolean useSharedNetwork, Duration timeout, DevServicesConfig devServicesConfig, DockerImageName
            dockerImageName, String serviceName) {

        if(devServicesConfig.db().port().isPresent()) {
            databaseContainer = new DemoDatabaseContainer(dockerImageName, devServicesConfig.db().port().getAsInt(), useSharedNetwork, serviceName);
        } else {
            databaseContainer = new DemoDatabaseContainer(dockerImageName, useSharedNetwork, serviceName);
        }

        Optional.ofNullable(timeout).ifPresent(databaseContainer::withStartupTimeout);
        databaseContainer.withEnv(devServicesConfig.db().containerEnv());


        databaseContainer.withEnv("POSTGRES_USER", devServicesConfig.db().username());
        databaseContainer.withEnv("POSTGRES_PASSWORD", devServicesConfig.db().password());
        databaseContainer.withEnv("POSTGRES_DB",devServicesConfig.db().name());
        databaseContainer.withNetwork(useSharedNetwork ? Network.SHARED : Network.newNetwork());
        databaseContainer.start();

        return new DevServicesResultBuildItem.RunningDevService(feature, databaseContainer.getContainerId(),
                databaseContainer::close,
                Map.of(
                    "url", databaseContainer.getHost() + ":" + databaseContainer.getPort(),
                    "url-internal", databaseContainer.getContainerName().substring(1) + ":" + POSTGRES_INTERNAL_PORT
                )
        );

    }

    public Optional<GenericContainer<DemoDatabaseContainer>> getDatabaseContainer() {
        return Optional.ofNullable(databaseContainer);
    }

    public DevServicesResultBuildItem.RunningDevService getRunningDevService() {
        return runningDevServiceSupplier;
    }
}

Our class defines two constructors. The first simply returns the existing RunningDevService in case the container already exists. The second creates a new RunningDevService. The run method first checks if a port is specified in the dev-service configuration. If so, our container will be started on this port; otherwise, a random available port will be allocated (this is the default behavior).

Once the instance of our DemoDatabaseContainer is created, we configure it by passing the environment variables it expects via the withEnv method. Next comes the configuration of the Docker network used. If the user has explicitly specified that the containers should be attached to the shared Docker network create by TestContainer, we use the withNetwork method with the value Network.SHARED. Otherwise, a new network is created for the container.

Finally, we return our RunningDevService, containing the ID of our container along with the configuration that will be useful later.

DemoDatabaseContainer

Now, let’s look at the DemoDatabaseContainer class to finish this first post on creating a DevService with Quarkus.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65


import io.quarkus.devservices.common.ConfigureUtil;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.utility.DockerImageName;

public class DemoDatabaseContainer extends GenericContainer<DemoDatabaseContainer> {


    public static final String CONTAINER_NAME="demo-db";
    private static final String DEV_SERVICE_LABEL="quarkus-dev-service-db";
    private final String serviceName;
    private final Integer fixedExposedPort;
    public final static int POSTGRES_INTERNAL_PORT = 5432;
    private final boolean useSharedNetwork;
    private String hostname;

    public DemoDatabaseContainer(DockerImageName image, boolean useSharedNetwork, String serviceName) {
        this(image, null, useSharedNetwork, serviceName);
    }

    public DemoDatabaseContainer(DockerImageName image, Integer exposedPort, boolean useSharedNetwork, String serviceName) {
        super(image);
        this.fixedExposedPort = exposedPort;
        this.useSharedNetwork = useSharedNetwork;
        this.serviceName = serviceName;
    }

    @Override
    protected void configure() {
        super.configure();

        if (this.useSharedNetwork) {
            hostname = ConfigureUtil.configureSharedNetwork(this, "demo");
            return;
        }

        if(this.fixedExposedPort != null) {
          addFixedExposedPort(this.fixedExposedPort, POSTGRES_INTERNAL_PORT);
        } else {
            addExposedPort(POSTGRES_INTERNAL_PORT);
        }

        withLabel(DEV_SERVICE_LABEL, serviceName);

        super.setWaitStrategy(Wait.forListeningPorts(POSTGRES_INTERNAL_PORT));
    }

    @Override
    public String getHost() {
        if(this.useSharedNetwork) {
            return hostname;
        }
        return super.getHost();
    }

    public int getPort() {
        if (useSharedNetwork) {
            return POSTGRES_INTERNAL_PORT;
        }
        if (fixedExposedPort != null) {
            return fixedExposedPort;
        }
        return getFirstMappedPort();
    }
}

This class extends GenericContainer and contains the configuration used by TestContainer to start our Docker container. In the configure method, we first check if the container should use the shared network. If so, ConfigureUtil.configureSharedNetwork handles everything. Otherwise, we configure the port (fixed or random), add a label, and specify the port to listen to for service startup completion.

Conclusion

Our database extension is now functional. It provides an implementation of a GenericContainer through a RunningDevService to our DemoExtensionProcessor. Everything is customizable via Quarkus’s standard configuration. All we need to do is to include the extension in a Quarkus application via a Maven import to have the database started in parallel.

In the next article, we’ll see how to launch a service in its own container using this database to expose an API to our Quarkus application.