WildFly 18 / JBoss EAP 7.3

Configure the application server according to instructions in the following sections.

Download and Extract WildFly

If you have not downloaded WildFly already, you can use the following command to do so:

wget https://download.jboss.org/wildfly/18.0.0.Final/wildfly-18.0.0.Final.zip -O /tmp/wildfly-18.0.0.Final.zip
unzip -q /tmp/wildfly-18.0.0.Final.zip -d /opt/
ln -snf /opt/wildfly-18.0.0.Final /opt/wildfly

If you are using JBoss EAP 7.3, you can download it from Red Hat's website.

Remove RESTEasy-Crypto

The application server can sometimes load its own version of Bouncy Castle, resulting in incompatibility and/or conflict issues.

A typical error occurring as a consequence of the wrong Bouncy Castle being loaded in EJBCA is:

ClassCastException setting BagAttributes, can not set friendly name: : java.lang.ClassCastException: org.bouncycastle.jcajce.provider.asymmetric.x509.X509CertificateObject cannot be cast to org.bouncycastle.jce.interfaces.PKCS12BagAttributeCarrier

To avoid issues and prevent Wildfly/JBoss from loading the Bouncy Castle library included in the RESTEasy-Crypto module rather than the library included in EJBCA, run the following:

sed -i 's|.*org.jboss.resteasy.resteasy-crypto.*||' /opt/wildfly/modules/system/layers/base/org/jboss/as/jaxrs/main/module.xml
rm -rf /opt/wildfly/modules/system/layers/base/org/jboss/resteasy/resteasy-crypto

The following displays an example log message in case of a conflict:

ERROR [StartupSingletonBean] BouncyCastle is not loaded by an EJBCA classloader, version conflict is likely

Removing reasteasy-crypto has to be done every time you install a patch in JBoss EAP. When applying patches it may be added back with the patch under modules/system/layers/base/.overlays/.


Create a Custom Configuration

Replace /opt/wildfly/bin/standalone.conf with the following configuration file.

/opt/wildfly/bin/standalone.conf
if [ "x$JBOSS_MODULES_SYSTEM_PKGS" = "x" ]; then
JBOSS_MODULES_SYSTEM_PKGS="org.jboss.byteman"
fi
 
if [ "x$JAVA_OPTS" = "x" ]; then
JAVA_OPTS="-Xms<HEAP_SIZE>m -Xmx<HEAP_SIZE>m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m"
JAVA_OPTS="$JAVA_OPTS -Dhttps.protocols=TLSv1.2,TLSv1.3"
JAVA_OPTS="$JAVA_OPTS -Djdk.tls.client.protocols=TLSv1.2,TLSv1.3"
JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true"
JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS"
JAVA_OPTS="$JAVA_OPTS -Djava.awt.headless=true"
JAVA_OPTS="$JAVA_OPTS -Djboss.tx.node.id=<TX_NODE_ID>"
JAVA_OPTS="$JAVA_OPTS -Djdk.tls.ephemeralDHKeySize=2048"
else
echo "JAVA_OPTS already set in environment; overriding default settings with values: $JAVA_OPTS"
fi

Set Allowed Memory Usage

By default, 512 MB of heap (RAM) is allowed to be used by the application server. This is not sufficient to run EJBCA and it is recommended to allocate at least 2048 MB of RAM. To increase the default value, run the following command:

sed -i -e 's/<HEAP_SIZE>/2048/g' /opt/wildfly/bin/standalone.conf

Set Transaction Node Id

Set the transaction node ID to a unique value.

sed -i -e "s/<TX_NODE_ID>/$(od -A n -t d -N 1 /dev/urandom | tr -d ' ')/g" /opt/wildfly/bin/standalone.conf

Configure WildFly as a Service

Modern Linux systems use systemd to start and stop services. The WildFly package already contains the necessary files to run as a service, but they need to be installed manually. Once started as a service, WildFly will run as the wildfly user, and you need to add this user as well.

cp /opt/wildfly/docs/contrib/scripts/systemd/launch.sh /opt/wildfly/bin
cp /opt/wildfly/docs/contrib/scripts/systemd/wildfly.service /etc/systemd/system
mkdir /etc/wildfly
cp /opt/wildfly/docs/contrib/scripts/systemd/wildfly.conf /etc/wildfly
systemctl daemon-reload
useradd -M wildfly
chown -R wildfly:wildfly /opt/wildfly-18.0.0.Final/

Start WildFly

Start WildFly as a service.

systemctl start wildfly

Create an Elytron Credential Store

You can protect passwords by storing them in a credential store. The credential is encrypted with a master password which is fetched by WildFly on startup.

Create a Master Password

Create a script which outputs the master password to stdout and ensure the script can only be executed by the wildfly user.

echo '#!/bin/sh' > /usr/bin/wildfly_pass
echo "echo '$(openssl rand -base64 24)'" >> /usr/bin/wildfly_pass
chown wildfly:wildfly /usr/bin/wildfly_pass
chmod 700 /usr/bin/wildfly_pass

Create the Credential Store

Create a credential store in /opt/wildfly/standalone/configuration encrypted with the password echoed by the wildfly_pass script.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/credential-store=defaultCS:add(location=credentials, relative-to=jboss.server.config.dir, credential-reference={clear-text="{EXT}/usr/bin/wildfly_pass", type="COMMAND"}, create=true)'

Add Database Driver

For MariaDB and PostgreSQL, the database driver can be added by hot-deploying it into the deployment directory. This will be picked up by WildFly and deployed so you can create a data source straight away. You can use a generic name, without version number, to get a generic driver-name for the data source command.

Example command for MariaDB:

wget https://downloads.mariadb.com/Connectors/java/latest/mariadb-java-client-2.3.0.jar -O /opt/wildfly/standalone/deployments/mariadb-java-client.jar

Example command for PostgreSQL:

wget https://jdbc.postgresql.org/download/postgresql-42.2.9.jar -O /opt/wildfly/standalone/deployments/postgresql.jar

If you are using another type of database, copy the JDBC driver to the deployment directory and make note of the driver class and driver-name shown in the server log, for later use when adding the data source. For example:

... INFO [org.jboss.as.connector.deployers.jdbc] (...) WFLYJCA0005: Deploying non-JDBC-compliant driver class org.postgresql.Driver (version 42.2.5)
... INFO [org.jboss.as.connector.deployers.jdbc] (...) WFLYJCA0018: Started Driver service with driver-name = postgresql-jdbc3.jar

Add Datasource

To add a data source for EJBCA to use, run the commands in JBoss CLI according to the examples below.

Note that --driver-name is should be the same as the filename of the JAR file you copied in the step above. The --jndi-name is is defined by the property datasource.jndi-name in database.properties, and the default value is used in the following example. The --user-name and --password defines the username and password of the EJBCA database user. They correspond to the username and password defined by the properties database.username and database.password in database.properties. In production, this password should be changed to a "real" password.

MariaDB

For MariaDB, run the following:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/credential-store=defaultCS:add-alias(alias=dbPassword, secret-value="ejbca")'
/opt/wildfly/bin/jboss-cli.sh --connect 'data-source add --name=ejbcads --driver-name="mariadb-java-client.jar" --connection-url="jdbc:mysql://127.0.0.1:3306/ejbca" --jndi-name="java:/EjbcaDS" --use-ccm=true --driver-class="org.mariadb.jdbc.Driver" --user-name="ejbca" --credential-reference={store=defaultCS, alias=dbPassword} --validate-on-match=true --background-validation=false --prepared-statements-cache-size=50 --share-prepared-statements=true --min-pool-size=5 --max-pool-size=150 --pool-prefill=true --transaction-isolation=TRANSACTION_READ_COMMITTED --check-valid-connection-sql="select 1;"'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

PostgreSQL

For PostgreSQL, run the following:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/credential-store=defaultCS:add-alias(alias=dbPassword, secret-value="foo123")'
/opt/wildfly/bin/jboss-cli.sh --connect 'data-source add --name=ejbcads --driver-name="org.postgresql.Driver" --connection-url="jdbc:postgresql://127.0.0.1/ejbca" --jndi-name="java:/EjbcaDS" --use-ccm=true --driver-name="postgresql.jar" --driver-class="org.postgresql.Driver" --user-name="ejbca" --credential-reference={store=defaultCS, alias=dbPassword} --validate-on-match=true --background-validation=false --prepared-statements-cache-size=50 --share-prepared-statements=true --min-pool-size=5 --max-pool-size=150 --pool-prefill=true --transaction-isolation=TRANSACTION_READ_COMMITTED --check-valid-connection-sql="select 1;"'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Other Databases

If you are using another type of database, adapt the statement above with the correct --driver-name, --connection-url, --driver-class and --check-valid-connection-sql.

Configure WildFly Remoting

EJBCA needs to use JBoss Remoting for the EJBCA CLI to work. Configure it to use a separate port 4447 and remove any other dependency on remoting except for what EJBCA needs.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=remoting/http-connector=http-remoting-connector:write-attribute(name=connector-ref,value=remoting)'
/opt/wildfly/bin/jboss-cli.sh --connect '/socket-binding-group=standard-sockets/socket-binding=remoting:add(port=4447,interface=management)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/http-listener=remoting:add(socket-binding=remoting,enable-http2=true)'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Configure Logging

Configure logging in JBoss to be able to dynamically change logging. For production systems, INFO log level is recommended:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.ejbca:add(level=INFO)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.cesecore:add(level=INFO)'

To later enable DEBUG log level, if needed, use the following:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.ejbca:write-attribute(name=level, value=DEBUG)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.cesecore:write-attribute(name=level, value=DEBUG)'

Additional Logging Configuration

You may additionally want to add the following configuration:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.jboss:add(level=WARN)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.cesecore.config.ConfigurationHolder:add(level=WARN)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.hibernate.dialect.H2Dialect:add(level=ERROR)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=logging/logger=org.wildfly:add(level=WARN)'

Remove Old Log Files

Log files are rotated daily by default, but old log files are not deleted automatically. You can delete old log files using a cronjob:

/etc/cron.daily/wildfly
#!/bin/sh
# Remove log files older than 7 days
find /opt/wildfly/standalone/log/ -type f -mtime +7 -name 'server.log*' -execdir rm -- '{}' \;

Make the file executable:

chmod +x /etc/cron.daily/wildfly

HTTP(S) Listener Configuration with 3-port Separation

The following section explains how to set up Undertow with 3-port separation. Port 8080 is used for HTTP (unencrypted traffic), port 8442 for HTTPS (encrypted) traffic without only server authentication and port 8443 for HTTP with both server and client authentication.

Remove Existing TLS and HTTP Configuration

Run the following commands in JBoss CLI to remove existing TLS and HTTP configuration (just to be safe):

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/http-listener=default:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/https-listener=https:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/socket-binding-group=standard-sockets/socket-binding=http:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/socket-binding-group=standard-sockets/socket-binding=https:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Add New Interfaces and Sockets

To add new interfaces and sockets, use the following:

/opt/wildfly/bin/jboss-cli.sh --connect '/interface=http:add(inet-address="0.0.0.0")'
/opt/wildfly/bin/jboss-cli.sh --connect '/interface=httpspub:add(inet-address="0.0.0.0")'
/opt/wildfly/bin/jboss-cli.sh --connect '/interface=httpspriv:add(inet-address="0.0.0.0")'
/opt/wildfly/bin/jboss-cli.sh --connect '/socket-binding-group=standard-sockets/socket-binding=http:add(port="8080",interface="http")'
/opt/wildfly/bin/jboss-cli.sh --connect '/socket-binding-group=standard-sockets/socket-binding=httpspub:add(port="8442",interface="httpspub")'
/opt/wildfly/bin/jboss-cli.sh --connect '/socket-binding-group=standard-sockets/socket-binding=httpspriv:add(port="8443",interface="httpspriv")'

Configure TLS

Configure TLS according to the following instructions. Update the keystore password for keystore.jks to match httpsserver.password and the truststore password for truststore.jks to match java.truststore in web.properties.

Make sure the password of the keystore and truststore in this section are correct, in order for the commands not to fail. In production, the passwords should be changed to "real" passwords.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/credential-store=defaultCS:add-alias(alias=httpsKeystorePassword, secret-value="serverpwd")'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/credential-store=defaultCS:add-alias(alias=httpsTruststorePassword, secret-value="changeit")'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/key-store=httpsKS:add(path="keystore/keystore.jks",relative-to=jboss.server.config.dir,credential-reference={store=defaultCS, alias=httpsKeystorePassword},type=JKS)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/key-store=httpsTS:add(path="keystore/truststore.jks",relative-to=jboss.server.config.dir,credential-reference={store=defaultCS, alias=httpsTruststorePassword},type=JKS)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/key-manager=httpsKM:add(key-store=httpsKS,algorithm="SunX509",credential-reference={store=defaultCS, alias=httpsKeystorePassword})'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/trust-manager=httpsTM:add(key-store=httpsTS)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/server-ssl-context=httpspub:add(key-manager=httpsKM,protocols=["TLSv1.2"],use-cipher-suites-order=false,cipher-suite-filter="TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256")'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/server-ssl-context=httpspriv:add(key-manager=httpsKM,protocols=["TLSv1.2"],use-cipher-suites-order=false,cipher-suite-filter="TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",trust-manager=httpsTM,need-client-auth=true)'

Add HTTP(S) Listeners

To add HTTP(S) listeners:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/http-listener=http:add(socket-binding="http", redirect-socket="httpspriv")'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/https-listener=httpspub:add(socket-binding="httpspub", ssl-context="httpspub", max-parameters=2048)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/https-listener=httpspriv:add(socket-binding="httpspriv", ssl-context="httpspriv", max-parameters=2048)'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

HTTP Protocol Behavior Configuration

The following shows an example HTTP Protocol Behavior Configuration.

If you are using OCSP GET requests, setting URI encoding and allowing encoding for query and backslash are needed.

The commands below ensure the address in the EJBCA WS WSDL matches the hostname and port the client uses to access the server.

/opt/wildfly/bin/jboss-cli.sh --connect '/system-property=org.apache.catalina.connector.URI_ENCODING:add(value="UTF-8")'
/opt/wildfly/bin/jboss-cli.sh --connect '/system-property=org.apache.catalina.connector.USE_BODY_ENCODING_FOR_QUERY_STRING:add(value=true)'
/opt/wildfly/bin/jboss-cli.sh --connect '/system-property=org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH:add(value=true)'
/opt/wildfly/bin/jboss-cli.sh --connect '/system-property=org.apache.tomcat.util.http.Parameters.MAX_COUNT:add(value=2048)'
/opt/wildfly/bin/jboss-cli.sh --connect '/system-property=org.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH:add(value=true)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=webservices:write-attribute(name=wsdl-host, value=jbossws.undefined.host)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=webservices:write-attribute(name=modify-wsdl-address, value=true)'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Optional Configuration

The following sections cover optional configurations.

Remove Welcome Content

Removes the hardcoded welcome page in WildFly.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/host=default-host/location="\/":remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/configuration=handler/file=welcome-content:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

You can also remove the actual files and save some disk space with:

rm -rf /opt/wildfly/welcome-content/

Redirect to Application for Unknown URLs

Known URLs for EJBCA starts with /ejbca/, /crls, /certificates or /.well-known/ (EST and ACME) according to the following example:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/configuration=filter/rewrite=redirect-to-app:add(redirect=true,target="/ejbca/")'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/host=default-host/filter-ref=redirect-to-app:add(predicate="method(GET) and not path-prefix(/ejbca,/crls,/certificates,/.well-known) and not equals({\%{LOCAL_PORT}, 4447})")'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

URL Rewriting for the CRL Servlet

Since EJBCA 7.2, the ability to customize the URL for the CRL store servlet has been removed. For more information, see ECA-7059 and the EJBCA 7.2 Upgrade Notes.

However, you can download CRLs on a different URL by adding a rewrite rule in Undertow, see JBoss EAP 7.2 Model Reference. As an example, to allow a client to download CRLs from the URL /ejbca/crls/ instead of /ejbca/publicweb/crls/, add the following configuration:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/configuration=filter/rewrite=crl-rewrite:add(target="/ejbca/publicweb/crls/$${1}")'
/opt/wildfly/bin/jboss-cli.sh --connect "/subsystem=undertow/server=default-server/host=default-host/filter-ref=crl-rewrite:add(predicate=\"method(GET) and regex('/crls/(.*)')\")"
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Enable OCSP Revocation Checking

WildFly can check the validity of client certificates against the OCSP responder defined by the certificate's AIA extension:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/trust-manager=httpsTM:write-attribute(name=ocsp, value={})'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Remove the ExampleDS Data Source

To optionally remove the ExampleDS data source:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=ee/service=default-bindings:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect 'data-source remove --name=ExampleDS'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Remove Unneeded Subsystems and Extensions

To optionally remove the unneeded subsystems and extensions:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=jdr:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=sar:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=jmx:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=pojo:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=ee-security:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=microprofile-metrics-smallrye:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=microprofile-health-smallrye:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=microprofile-opentracing-smallrye:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/extension=org.wildfly.extension.microprofile.health-smallrye:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/extension=org.wildfly.extension.microprofile.opentracing-smallrye:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/extension=org.jboss.as.jdr:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/extension=org.jboss.as.jmx:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/extension=org.jboss.as.sar:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/extension=org.jboss.as.pojo:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/extension=org.wildfly.extension.ee-security:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=distributable-web:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=infinispan/cache-container=ejb:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=infinispan/cache-container=server:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=ejb3/cache=distributable:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=ejb3/passivation-store=infinispan:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=security/security-domain=jaspitest:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=datasources/jdbc-driver=h2:remove()'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Enable AJP Connector

Enabling the AJP connector is only needed if you terminate the TLS connection at a proxy (such as Apache or nginx) running in front of WildFly:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/ajp-listener=ajp-listener:add(socket-binding=ajp, scheme=https, enabled=true)'

Add Support for Sending Email

To add support for sending email (SMTP), configure EjbcaMail. Specify the hostname, port, address and SMTP credentials according to the following example:

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/credential-store=defaultCS:add-alias(alias=smtpPassword, secret-value="foo123")'
/opt/wildfly/bin/jboss-cli.sh --connect '/socket-binding-group=standard-sockets/remote-destination-outbound-socket-binding=ejbca-mail-smtp:add(port="993", host="my.mail.server")'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=mail/mail-session="java:/EjbcaMail":add(jndi-name=java:/EjbcaMail, from=noreply@mymail)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=mail/mail-session="java:/EjbcaMail"/server=smtp:add(outbound-socket-binding-ref=ejbca-mail-smtp, tls=true, username=smtpuser, credential-reference={store=defaultCS, alias=smtpPassword})'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

Only Deploy at Startup

Historically, application servers are really bad at cleaning up memory from previous deployments and hot (re-)deploy is discouraged in production. To avoid manual deployment with the management interface, you can specify that the deployment directory should be scanned once at application server startup by setting the scan-interval to 0. This also prevents an attacker from loading a malicious JAR file by dropping it in the deployments directory.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=deployment-scanner/scanner=default:write-attribute(name=scan-interval,value=0)'

Increase Deployment Timeout

If you are using HSMs with smart card authentication or if the database needs to reindex when WildFly boots, you may have to increase the deployment timeout to be able to deploy EJBCA correctly. The deployment timeout is specified in seconds, the command below sets it to 5 minutes.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=deployment-scanner/scanner=default:write-attribute(name=deployment-timeout,value=300)'

Disable Management Web Console

If you only plan on using the JBoss CLI anyway there is little reason to keep this around.

/opt/wildfly/bin/jboss-cli.sh --connect '/core-service=management/management-interface=http-interface:write-attribute(name=console-enabled,value=false)'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Increase Maximum Upload Size

Increase the maximum size of POST requests to 25 MB. This may be needed if you need to upload large domain block lists, zip packages with profiles, statedumps ect.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=undertow/server=default-server/http-listener=http/:write-attribute(name=max-post-size,value=25485760)'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

Enable WildFly Audit Logging

WildFly can audit log changes made to the management model and output audit log events in JSON format to the file /opt/wildfly/standalone/data/audit-log.log. Audit logging is disabled by default. To enable it, run:

/opt/wildfly/bin/jboss-cli.sh --connect '/core-service=management/access=audit/logger=audit-log:write-attribute(name=enabled,value=true)'

Configure OCSP Logging

If you have enabled the OCSP audit log or the OCSP transaction log, you should configure an asynchronous logger in WildFly for best performance.

For the OCSP transaction log:

/subsystem=logging/logger=org.cesecore.certificates.ocsp.logging.TransactionLogger:add(use-parent-handlers=false)
/subsystem=logging/logger=org.cesecore.certificates.ocsp.logging.TransactionLogger:write-attribute(name=level, value=INFO)
/subsystem=logging/async-handler=ocsp-tx-async:add(queue-length="100")
/subsystem=logging/async-handler=ocsp-tx-async:write-attribute(name=level, value=DEBUG)
/subsystem=logging/async-handler=ocsp-tx-async:write-attribute(name="overflow-action", value="BLOCK")
/subsystem=logging/logger=org.cesecore.certificates.ocsp.logging.TransactionLogger:add-handler(name=ocsp-tx-async)
/subsystem=logging/periodic-rotating-file-handler=ocsp-tx:add(autoflush=true, append=true, suffix=".yyyy-MM-dd", file={path=ocsp-tx.log,relative-to=jboss.server.log.dir})
/subsystem=logging/async-handler=ocsp-tx-async:add-handler(name=ocsp-tx)

For the OCSP audit log:

/subsystem=logging/logger=org.cesecore.certificates.ocsp.logging.AuditLogger:add(use-parent-handlers=false)
/subsystem=logging/logger=org.cesecore.certificates.ocsp.logging.AuditLogger:write-attribute(name=level, value=INFO)
/subsystem=logging/async-handler=ocsp-audit-async:add(queue-length="100")
/subsystem=logging/async-handler=ocsp-audit-async:write-attribute(name=level, value=DEBUG)
/subsystem=logging/async-handler=ocsp-audit-async:write-attribute(name="overflow-action", value="BLOCK")
/subsystem=logging/logger=org.cesecore.certificates.ocsp.logging.AuditLogger-handler(name=ocsp-audit-async)
/subsystem=logging/periodic-rotating-file-handler=ocsp-audit:add(autoflush=true, append=true, suffix=".yyyy-MM-dd", file={path=ocsp-audit.log,relative-to=jboss.server.log.dir})
/subsystem=logging/async-handler=ocsp-audit-async:add-handler(name=ocsp-audit)

Archive rotated log files to save disk space using a cron job:

/etc/cron.daily/archive-rotated-ocsp-logs.sh
#!/bin/sh
# Compress the OCSP audit log and the OCSP transaction log after rotation
xz /opt/wildfly/standalone/log/ocsp-tx.log.*
xz /opt/wildfly/standalone/log/ocsp-audit.log.*

Make the file executable:

chmod +x /etc/cron.daily/archive-rotated-ocsp-logs.sh

ACME Enrollment

WildFly can enroll for TLS certificates using ACME. If you enroll from EJBCA, see the EJBCA documentation on how to enable and configure ACME.

  1. If you are enrolling against your internal PKI, make sure it is trusted by Java.

    Replace <ROOT_NAME> with the name of your root CA and <ROOT_CA_PATH> with the path to the root CA certificate in PEM format. The location of your cacerts file (/usr/lib/jvm/java-11-openjdk-amd64/lib/security/cacerts) may vary.

    sudo keytool -importcert -noprompt -trustcacerts -alias <ROOT_NAME> -file <ROOT_CA_PATH> -keystore /usr/lib/jvm/java-11-openjdk-amd64/lib/security/cacerts -storepass changeit

  2. Create a keystore where the keyspair of the ACME account will be stored.

    /opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/credential-store=defaultCS:add-alias(alias=accountsPassword, secret-value="foo123")'
    /opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/key-store=accountsKS:add(path=accounts.jks,relative-to=jboss.server.config.dir,credential-reference={store=defaultCS, alias=accountsPassword},type=JKS)'

  3. Define the ACME directory to use.

    Replace <ACME_DIRECTORY_URL> and <ACME_DIRECTORY_URL_STAGING> with the ACME directory URL for production and staging respectively. If you are enrolling directly against an EJBCA server, use something like https://hostname:8442/ejbca/acme/directory.

    /opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/certificate-authority=AcmeCA:add(url="<ACME_DIRECTORY_URL>",staging-url="<ACME_DIRECTORY_URL_STAGING>")'

  4. Register an ACME account with the CA.

    Replace <HOSTNAME> with the hostname of your machine and <EMAIL> with your email address.

    /opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/certificate-authority-account=AcmeAccount:add(certificate-authority=AcmeCA,alias=<HOSTNAME>,key-store=accountsKS,contact-urls=[mailto:<EMAIL>])'

  5. Enroll for a certificate.

    Replace <HOSTNAME> with the hostname of your machine.

    /opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=elytron/key-store=httpsKS:obtain-certificate(alias=<HOSTNAME>,domain-names=[<HOSTNAME>],certificate-authority-account=AcmeAccount,agree-to-terms-of-service)'

Performance Tuning

The following section covers WildFly configuration for maximising EJBCA performance. The best setup could vary from installation to installation and should be manually fine-tuned. However, the "default" configuration would typically be a good starting point. You should also consider increasing the heap (RAM) for EJBCA instances under high load to something like 4 GB.

/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=io/worker=default/:write-attribute(name=task-core-threads,value=25)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=io/worker=default/:write-attribute(name=task-max-threads,value=100)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=io/worker=default/:write-attribute(name=io-threads,value=100)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=ejb3/strict-max-bean-instance-pool=slsb-strict-max-pool:undefine-attribute(name=derive-size)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=ejb3/strict-max-bean-instance-pool=slsb-strict-max-pool:write-attribute(name=max-pool-size, value=32)'
/opt/wildfly/bin/jboss-cli.sh --connect '/subsystem=datasources/data-source=ejbcads/:write-attribute(name=max-pool-size,value=150)'
/opt/wildfly/bin/jboss-cli.sh --connect ':reload'

images/s/-2y7bau/8703/189cb2l/_/images/icons/emoticons/warning.svg Wait for the reload to complete by checking the server log or the result of :read-attribute(name=server-state) before continuing.

Clear Command Line History

Clear the command line history to prevent accidental viewing of passwords put into the credential store.

history -c

Create a Snapshot of the Configuration

Create a snapshot of the current configuration to make sure you can revert back to a working state.

/opt/wildfly/bin/jboss-cli.sh --connect ':take-snapshot'

Next Step: Deploying EJBCA

For instructions on how to build and deploy EJBCA, see Deploying EJBCA.