Getting Started with Minikube on Fedora 30

Introduction

To install Minikube there are 3 things to do:

1. Install a hypervisor, e.g. VirtualBox, KVM
2. "Install" (download and add in path) kubectl
3. "Install" (download and add in path) Minikube

Install kubectl

Reference: https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-on-linux

$ curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl

$ chmod +x ./kubectl

$ sudo mv ./kubectl /usr/local/bin/kubectl

$ kubectl version

Install MiniKube

$ wget https://github.com/kubernetes/minikube/releases/download/v1.6.0-beta.1/minikube-1.6.0.beta.1.rpm .
    
$ sudo rpm -pi minikube-1.6.0.beta.1.rpm

$ sudo rpm -ql minikube
/usr/bin/minikube

$ sudo rpm -q --scripts minikube

Test

https://kubernetes.io/docs/setup/learning-environment/minikube/

Start Minikube and create a cluster. Here I specify VirtualBox as hypervisor, since I have also KVM installed.

$ minikube start --vm-driver=virtualbox

Deploy image 'k8s.gcr.io/echoserver' version 1.10 and name deployment 'hello-minikube'.

$ kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10

Expose deployment on NodePort and port 8080, i.e. create routing.

$ kubectl expose deployment hello-minikube --type=NodePort --port=8080

Check deployment has finished.

$ kubectl get pod

Retrieve internal IP and port and access web app.

$ minikube service hello-minikube --url
http://192.168.99.100:30024

$ curl http://192.168.99.100:30024
Hostname: hello-minikube-797f975945-fmswk

Pod Information:
 -no pod information available-

Server values:
 server_version=nginx: 1.13.3 - lua: 10008

Request Information:
 client_address=172.17.0.1
 method=GET
 real path=/
 query=
 request_version=1.1
 request_scheme=http
 request_uri=http://192.168.99.100:8080/

Request Headers:
 accept=*/*
 host=192.168.99.100:30024
 user-agent=curl/7.65.3

Request Body:
 -no body in request-

How to Install VirtualBox 6 on Fedora 30

https://www.if-not-true-then-false.com/2010/install-virtualbox-with-yum-on-fedora-centos-red-hat-rhel/

Introduction to YAML

Introduction

The YAML format has grown in popularity and is used for example in Kubernetes and Ansible.

Comparison XML, JSON and YAML

XML	JSON	YAML
<Servers> <Server> <id>12345</id> <name>My Web Server</name> <status>ACTIVE</status> </Server> </Servers>	{ "Servers": [ { "id": "12345", "name": "My Web Server", "status": "ACTIVE" } ] }	Servers: - id: 1234 name: My Web Server status: ACTIVE

Tools for validating JSON

There are numerous tools out that for JSON, one for Linux and Bash is jq - a lightweight and flexible command-line JSON processor.

$ sudo yum install jq

$ cat servers.json | jq '.Servers[0].id'
"12345"

Tools for validating YAML

And a Linux and Bash tool for YAML is yq

$ sudo yum install python3 python3-pip
$ sudo pip3 install yq

$ cat servers.yaml | yq '.'
{
  "Servers": [
    {
      "id": 1234,
      "name": "My Web Server",
      "status": "ACTIVE"
    }
  ]
}

YAML Data Types

WARNING: YAML is not structured in the same sense as XML, JSON, XHTML, etc. It uses spaces (and not tabs and do not mix space and tabs) as seperator. So pay extra attention to how many spaces you use.

Key Value Pair

Fruit: Orange
Vegetable: Lettuce
Liquid: Wine

$ cat key_value_pair.yaml | yq '.'
{
  "Fruit": "Orange",
  "Vegetable": "Lettuce",
  "Liquid": "Wine"
}

List (Array)

Fruits:
-   Orange
-   Apple
-   Banana

Vegetables:
-   Carrot
-   Tomatoes
-   Onion

$ cat array_list.yaml | yq '.'
{
  "Fruits": [
    "Orange",
    "Apple",
    "Banana"
  ],
  "Vegetables": [
    "Carrot",
    "Tomatoes",
    "Onion"
  ]
}

Dictionary (Map)

Banana:
    Calories: 105
    Fat: 0.4 g
    Carbs: 31 g

Grapes:
    Calories: 27
    Fat: 0.7 g
    Carbs: 56 g    

$ cat dictionary_map.yaml | yq '.'
{
  "Banana": {
    "Calories": 105,
    "Fat": "0.4 g",
    "Carbs": "31 g"
  },
  "Grapes": {
    "Calories": 27,
    "Fat": "0.7 g",
    "Carbs": "56 g"
  }
}

More Advanced Examples. List of Dictionary (Array of Map)

Fruits:
-   Orange:
        Calories: 105
        Fat: 0.4 g
        Carbs: 31 g
-   Apple:
        Calories: 27
        Fat: 0.7 g
        Carbs: 56 g
        
$ cat array_list_of_dictionary_map.yaml | yq '.'

$ cat array_list_of_dictionary_map.yaml | yq '.Fruits[1]'
{
  "Apple": {
    "Calories": 27,
    "Fat": "0.7 g",
    "Carbs": "56 g"
  }
}

Differences between List and Dictionary

Dictionary - Unordered

List - Ordered

How to Install IDM Master and Replica on RHEL 7

Minimum Hardware

4 GB RAM

https://bugzilla.redhat.com/show_bug.cgi?id=1436295

Prerequisite

I have created two virtual machine, since this is a development setup I will hardcode hostname and IP in /etc/hosts and manually set hostnames.

• rhel7.7-idm-master.magnuskkarlsson.local
• 192.168.122.113
• rhel7.7-idm-replica1.magnuskkarlsson.local
• 192.168.122.99

# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.7 (Maipo)

# systemctl stop firewalld; systemctl disable firewalld

# hostnamectl set-hostname rhel7.7-idm-master.magnuskkarlsson.local
# hostnamectl set-hostname rhel7.7-idm-replica1.magnuskkarlsson.local

# echo "192.168.122.113 rhel7.7-idm-master.magnuskkarlsson.local" >> /etc/hosts
# echo "192.168.122.99 rhel7.7-idm-replica1.magnuskkarlsson.local" >> /etc/hosts

Update Date & Time

Make sure NTP is setup and synchronized.

systemctl restart chronyd
chronyc sources
chronyc tracking
timedatectl

Installation of IDM Master (version 4.6.5)

Install IDM Master on rhel7.7-idm-master.magnuskkarlsson.local.

# yum install -y ipa-server

# ipa-server-install --domain magnuskkarlsson.local \
    --realm MAGNUSKKARLSSON.LOCAL \
    -p foo123123 -a foo123123 -U 

Installation of IDM Replica (version 4.6.5)

Install IDM Replica on rhel7.7-idm-replica1.magnuskkarlsson.local.

First install idm/ipa client and register host in idm. Then setup host as replica.

# yum install -y ipa-server

# ipa-client-install --server=rhel7.7-idm-master.magnuskkarlsson.local \
    --domain=magnuskkarlsson.local \
    --principal=admin \
    --password=foo123123 -U

# ipa-replica-install --setup-ca --principal=admin --admin-password=foo123123 -U

Test

First kerberos login on master - rhel7.7-idm-master.magnuskkarlsson.local and add a user.

# kinit admin

# ipa user-add --first="Magnus K" \
    --last=Karlsson \
    --cn="Magnus K Karlsson" \
    --principal=magnuskkarlsson \
    --password \
    --all magnuskkarlsson

# ipa user-find magnuskkarlsson

Then kerberos login on replica - rhel7.7-idm-replica1.magnuskkarlsson.local and search for user and check that user is replicated.

# kinit admin

# ipa user-find magnuskkarlsson

Eclipse Microprofile LDAP Health Check with Java EE 8 and JBoss EAP 7.2

Introduction

In my previous blog I wrote about Eclipse Microprofile Health with Java EE 8 and JBoss EAP 7.2 and pointed to several Health Checks built in Spring, but for LDAP Springs Health Check is not very good. A better implementation is suggested below. And also based on standard Eclipse Microprofile Health.

Eclipse Microprofile LDAP Health Check

package se.magnuskkarlsson.example.microprofile;

import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.annotation.PostConstruct;
import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;
import javax.naming.Context;
import javax.naming.NamingEnumeration;
import javax.naming.NamingException;
import javax.naming.directory.SearchControls;
import javax.naming.directory.SearchResult;
import javax.naming.ldap.InitialLdapContext;

import org.eclipse.microprofile.config.inject.ConfigProperty;
import org.eclipse.microprofile.health.Health;
import org.eclipse.microprofile.health.HealthCheck;
import org.eclipse.microprofile.health.HealthCheckResponse;

@Health
@ApplicationScoped
public class LDAPHealthCheck implements HealthCheck {

    private final Logger log = Logger.getLogger(LDAPHealthCheck.class.getName());

    @Inject
    @ConfigProperty(name = "ldapHealthCheck.providerURL")
    protected String providerURL;

    @Inject
    @ConfigProperty(name = "ldapHealthCheck.securityPrincipal")
    protected String securityPrincipal;

    @Inject
    @ConfigProperty(name = "ldapHealthCheck.securityCredentials")
    protected String securityCredentials;

    @Inject
    @ConfigProperty(name = "ldapHealthCheck.baseCtxDN")
    protected String baseCtxDN;

    @PostConstruct
    public void init() {
    }

    @Override
    public HealthCheckResponse call() {
        List<String> result = null;
        try {
            result = searchBaseContextDN();
        } catch (Exception e) {
            log.log(Level.SEVERE, "Failed to perform LDAP health check search baseCtxDN='" + baseCtxDN + "'.", e);
        }
        boolean state = (result != null && !result.isEmpty()) ? true : false;
        String data = (result != null && !result.isEmpty()) ? result.toString() : null;
        return HealthCheckResponse.named("ldap-health-check").withData(baseCtxDN, data).state(state).build();
    }

    protected List<String> searchBaseContextDN() throws NamingException {
        InitialLdapContext ctx = null;
        try {
            Properties env = new Properties();
            env.setProperty(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
            env.setProperty(Context.PROVIDER_URL, providerURL);
            env.setProperty(Context.SECURITY_AUTHENTICATION, "simple");
            env.setProperty(Context.SECURITY_PRINCIPAL, securityPrincipal);
            env.put(Context.SECURITY_CREDENTIALS, securityCredentials);

            log.info("Logging into LDAP server, env=" + env);
            ctx = new InitialLdapContext(env, null);
            log.info("Logged into LDAP server, " + ctx);

            // filter
            String filter = "(objectClass=*)";

            // scope
            SearchControls ctls = new SearchControls();
            ctls.setSearchScope(SearchControls.ONELEVEL_SCOPE);

            // search for objects using filter and scope
            NamingEnumeration<SearchResult> answer = ctx.search(baseCtxDN, filter, ctls);
            List<String> result = new ArrayList<String>();
            while (answer.hasMore()) {
                SearchResult searchResult = answer.next();
                if (searchResult != null) {
                    result.add(searchResult.toString());
                }
            }

            log.info("Result base context dn, " + result);
            return result;
        } finally {
            if (ctx != null) {
                try {
                    // Close the context when we're done
                    ctx.close();
                } catch (NamingException IGNORE) {
                }
            }
        }
    }

}

Test

http://127.0.0.1:9990/health

[standalone@localhost:9990 /] /subsystem=microprofile-health-smallrye:check

How To Install VisualVM on Fedora 30 and OpenJDK 11

Installation

VisualVM is not bundle with OpenJDK 11, but can be easily downloaded, unzipped and ran.

https://visualvm.github.io/download.html

$ wget https://github.com/visualvm/visualvm.src/releases/download/1.4.4/visualvm_144.zip

$ unzip visualvm_144.zip

$ visualvm_144/bin/visualvm

Feature

https://visualvm.github.io/features.html

How to Read MSUPN in X509Certificate

Maven dependency

        <!-- The prov module provides all the JCA/JCE provider functionality. -->
        <dependency>
            <groupId>org.bouncycastle</groupId>
            <artifactId>bcprov-jdk15on</artifactId>
            <version>${bouncycastle.version}</version>
        </dependency>
        <!-- The pkix module is the home for code for X.509 certificate generation 
            and the APIs for standards that rely on ASN.1 such as CMS, TSP, PKCS#12, OCSP, CRMF, 
            and CMP. -->
        <dependency>
            <groupId>org.bouncycastle</groupId>
            <artifactId>bcpkix-jdk15on</artifactId>
            <version>${bouncycastle.version}</version>
        </dependency>

The source code for The Bouncy Castle Crypto Package For Java.

The Java code

package se.magnuskkarlsson.example.bouncycastle;

import java.security.cert.CertificateParsingException;
import java.security.cert.X509Certificate;
import java.util.Collection;
import java.util.List;
import java.util.logging.Logger;

import org.bouncycastle.asn1.ASN1ObjectIdentifier;
import org.bouncycastle.asn1.ASN1Primitive;
import org.bouncycastle.asn1.ASN1Sequence;
import org.bouncycastle.asn1.ASN1String;
import org.bouncycastle.asn1.ASN1TaggedObject;
import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
import org.bouncycastle.asn1.x509.GeneralName;

public class X509CertificateUtils {

    public static final String MSUPN_OID = "1.3.6.1.4.1.311.20.2.3";
    private final Logger log = Logger.getLogger(X509CertificateUtils.class.getName());

    public String getMSUPNFromX509Certificate(X509Certificate cert) throws CertificateParsingException {
        Collection<List<?>> sans = JcaX509ExtensionUtils.getSubjectAlternativeNames(cert);
        for (List<?> san : sans) {

            log.info("Read X509 SAN " + sans);
            int sanType = (int) san.get(0);
            if (sanType == GeneralName.otherName) {

                ASN1Sequence sanASN1Sequence = (ASN1Sequence) san.get(1);
                String msupn = getSANFromASN1Sequence(sanASN1Sequence);
                if (msupn != null) {
                    return msupn;
                }
            }
        }
        return null;
    }

    private String getSANFromASN1Sequence(ASN1Sequence sanASN1Sequence) {
        ASN1ObjectIdentifier oid = (ASN1ObjectIdentifier) sanASN1Sequence.getObjectAt(0);
        if (!MSUPN_OID.equals(oid.getId())) {
            log.warning("Invalid MSUPN OID, expected '" + MSUPN_OID + "' got '" + oid.getId() + "'.");
            return null;
        }

        ASN1TaggedObject sanASN1TaggedObject = (ASN1TaggedObject) sanASN1Sequence.getObjectAt(1);
        ASN1Primitive sanASN1Primitive = sanASN1TaggedObject.getObject();

        if (sanASN1Primitive instanceof ASN1String) {
            return ((ASN1String) sanASN1Primitive).getString();
        }
        log.warning("Invalid ASN.1 Primitive class, expected ASN1String, got " + sanASN1Primitive.getClass());
        return null;
    }

}

Eclipse Microprofile Metrics and Application Metrics with Wildfly 18

Introduction

In my previous blog Eclipse Microprofile Metrics with Wildfly 18 and Prometheus, we were getting started with Eclipse Microprofile Metrics and Wildfly 18.

All Metrics: http://127.0.0.1:9990/metrics

Basic Metrics: http://127.0.0.1:9990/metrics/base

Vendor Specific Metrics: http://127.0.0.1:9990/metrics/vendor

Application Specific Metrics: http://127.0.0.1:9990/metrics/application

And in this blog we will focus how to write application specific metrics. Eclipse Microprofile Metrics is by default active in Wildfly 18, so no activation or configurations are needed. The application is a standard Java EE 8 web application with JAX-RS.

For documentation see Metrics Specification https://github.com/eclipse/microprofile-metrics/releases

@Counted

The @Counted annotation which counts how many time a request has been made.

    @GET
    @Path("/firstName")
    @Counted
    public Response getFirstName() {
        JsonObject json = Json.createObjectBuilder().add("@Counted", "HELLO " + System.currentTimeMillis()).build();
        return Response.ok(json.toString()).build();
    }

# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getFirstName_total counter
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getFirstName_total 1.0

@Gauge

The @Gauge is the most basic metric type that you can use as it just returns a value.

    @GET
    @Path("/time")
    @Produces(MediaType.TEXT_PLAIN)
    @Gauge(unit = "time")
    public Long getTime() {
        return System.currentTimeMillis();
    }

# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTime_time gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTime_time 1.571012533582E12

@Metered

The @Metered annotation measures the rate at which a set of events occur.

    @GET
    @Path("/title")
    @Metered(unit = MetricUnits.MILLISECONDS)
    public Response getTitle() {
        JsonObject json = Json.createObjectBuilder().add("@Metered", "HELLO " + System.currentTimeMillis()).build();
        return Response.ok(json.toString()).build();
    }

# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_total counter
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_total 2.0
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_rate_per_second 0.10850140606928724
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_one_min_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_one_min_rate_per_second 0.030703655021877174
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_five_min_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_five_min_rate_per_second 0.0065567799035988195
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_fifteen_min_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getTitle_fifteen_min_rate_per_second 0.002209922141215539

@Timed

The @Timed annotatoin measures the duration of an event.

    @GET
    @Path("/occupation")
    @Timed(unit = MetricUnits.MILLISECONDS)
    public Response getOccupation() {
        try {
            Thread.sleep((long) (Math.random() * 1000));
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        JsonObject json = Json.createObjectBuilder().add("@Timed", "HELLO " + System.currentTimeMillis()).build();
        return Response.ok(json.toString()).build();
    }

# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_rate_per_second 0.10850160127626114
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_one_min_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_one_min_rate_per_second 0.031982234148270686
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_five_min_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_five_min_rate_per_second 0.0066114184713530035
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_fifteen_min_rate_per_second gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_fifteen_min_rate_per_second 0.0022160607980413085
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_min_seconds gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_min_seconds 0.326822982
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_max_seconds gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_max_seconds 0.492277962
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_mean_seconds gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_mean_seconds 0.4076894175173451
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_stddev_seconds gauge
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_stddev_seconds 0.08270655402029932
# TYPE application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds summary
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds_count 2.0
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds{quantile="0.5"} 0.326822982
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds{quantile="0.75"} 0.492277962
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds{quantile="0.95"} 0.492277962
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds{quantile="0.98"} 0.492277962
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds{quantile="0.99"} 0.492277962
application_se_magnuskkarlsson_example_microprofile_boundary_PersonResource_getOccupation_seconds{quantile="0.999"} 0.492277962

JSON Output

You can also query the application metrics with accept type json.

$ curl -H "Accept: application/json" http://127.0.0.1:9990/metrics/application

{
    "se.magnuskkarlsson.example.microprofile.boundary.PersonResource.getTitle": {
        "fiveMinRate": 0.004020212971829918,
        "fifteenMinRate": 0.0021635871943593244,
        "meanRate": 0.00440791116923509,
        "count": 3,
        "oneMinRate": 0.015991588875909905
    },
    "se.magnuskkarlsson.example.microprofile.boundary.PersonResource.getMiddleName": {
        "current": 0,
        "min": 0,
        "max": 0
    },
    "se.magnuskkarlsson.example.microprofile.boundary.PersonResource.getOccupation": {
        "p99": 928.815422,
        "min": 326.822982,
        "max": 928.815422,
        "mean": 928.7665329420207,
        "p50": 928.815422,
        "p999": 928.815422,
        "stddev": 5.110449494753552,
        "p95": 928.815422,
        "p98": 928.815422,
        "p75": 928.815422,
        "fiveMinRate": 0.0040261670318063715,
        "fifteenMinRate": 0.0021665192890272054,
        "meanRate": 0.004407906126221853,
        "count": 3,
        "oneMinRate": 0.01599160852294868
    },
    "se.magnuskkarlsson.example.microprofile.boundary.PersonResource.getFirstName": 3
}

Eclipse Microprofile Metrics with Wildfly 18 and Prometheus

Eclipse Microprofile Metrics

"This specification aims at providing a unified way for Microprofile servers to export Monitoring data ("Telemetry") to management agents and also a unified Java API, that all (application) programmers can use to expose their telemetry data." [https://microprofile.io/project/eclipse/microprofile-metrics]

Source code https://github.com/eclipse/microprofile-metrics/

Eclipse Microprofile Metrics Specification https://github.com/eclipse/microprofile-metrics/releases

JBoss EAP 7.2

Does not support Eclipse Microprofile Metrics.

Wildfly 18.0.0.Final

Wildfly 18.0.0.Final supports Eclipse Microprofile Metrics 2.0.0 [1], which is part of Eclipse Microprofile 3.0.

[1] $JBOSS_HOME/modules/system/layers/base/org/eclipse/microprofile/metrics/api/main/microprofile-metrics-api-2.0.2.jar

Documentation https://docs.wildfly.org/18/Admin_Guide.html#MicroProfile_Metrics_SmallRye

Configuration

[standalone@localhost:9990 /] /subsystem=microprofile-metrics-smallrye:read-resource(recursive=true, include-defaults=true)
{
    "outcome" => "success",
    "result" => {
        "exposed-subsystems" => ["*"],
        "prefix" => expression "${wildfly.metrics.prefix:wildfly}",
        "security-enabled" => false
    }
}

Wildfly exposes Metrics via HTTP Management Interface, i.e. http://127.0.0.1:9990/metrics.

Prometheus

Prometheus is an open-source monitoring and alerting platform. Its main features are:

• "Prometheus implements a highly dimensional data model. Time series are identified by a metric name and a set of key-value pairs."
• "PromQL allows slicing and dicing of collected time series data in order to generate ad-hoc graphs, tables, and alerts."
• "Prometheus has multiple modes for visualizing data: a built-in expression browser, Grafana integration, and a console template language."
• "Prometheus stores time series in memory and on local disk in an efficient custom format. Scaling is achieved by functional sharding and federation."
• "Each server is independent for reliability, relying only on local storage. Written in Go, all binaries are statically linked and easy to deploy."
• "Alerts are defined based on Prometheus's flexible PromQL and maintain dimensional information. An alertmanager handles notifications and silencing."
• "Client libraries allow easy instrumentation of services. Over ten languages are supported already and custom libraries are easy to implement."
• "Existing exporters allow bridging of third-party data into Prometheus. Examples: system statistics, as well as Docker, HAProxy, StatsD, and JMX metrics."

[https://prometheus.io/]

Prometheus can either be locally installed or via Docker.

For local installation, download latest Prometheus version unpack it and run './prometheus'.

To use Docker, use Prometheus Image at https://hub.docker.com/r/prom/prometheus.

Prometheus Docker Image source (Dockerfile) https://github.com/prometheus/prometheus/blob/master/Dockerfile.

Prometheus Docker Image documentation https://prometheus.io/docs/prometheus/latest/installation/.

Before we can use Prometheus for Wildfly we need to add Wildfly metrics endpoint to Prometheus configuration. First download Prometheus and edit prometheus.yml in the root of the zipped installation.

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

  # this is the configuration to poll metrics from WildFly 18
  # https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
  - job_name: 'wildfly18'
    scrape_interval: 2s
    metrics_path: '/metrics'
    scheme: 'http'
    static_configs:
    - targets: ['127.0.0.1:9990']

Then start Prometheus with './prometheus', then open http://127.0.0.1:9090/. To test it check which metrics wildfly is exposing, by calling it metrics endpoint.

$ curl http://127.0.0.1:9990/metrics

# HELP base_cpu_processCpuLoad Displays the "recent cpu usage" for the Java Virtual Machine process.
# TYPE base_cpu_processCpuLoad gauge
base_cpu_processCpuLoad 1.5940700593791097E-4

Go back to Prometheus and enter base_cpu_processCpuLoad.

Eclipse Microprofile OpenTracing with Java EE 8 and JBoss EAP 7.2

MicroProfile OpenTracing

"The MicroProfile OpenTracing specification defines behaviors and an API for accessing an OpenTracing compliant Tracer object within your JAX-RS application. The behaviors specify how incoming and outgoing requests will have OpenTracing Spans automatically created." [https://microprofile.io/project/eclipse/microprofile-opentracing]

JBoss EAP 7.2

"SmallRye OpenTracing component and is provided by the microprofile-opentracing-smallrye subsystem."

"This subsystem implements Microprofile 1.1, which includes support for tracing requests to JAX-RS endpoints and CDI beans and is included in the default JBoss EAP 7.2 configuration."

"The microprofile-opentracing-smallrye subsystem ships with the Jaeger Java Client as the default tracer"

"Each individual WAR deployed to the JBoss EAP server automatically has its own Tracer instance."

"Instead, you configure the Jaeger Client by setting system properties or environment variables. See the Jaeger documentation [1] for information about how to configure the Jaeger Client. See Configuration via Environment [2] in the Jaeger documentation for the list of valid system properties."

"Because this feature is provided as Technology Preview, the current configuration options, particularly those related to configuring the Jaeger Java Client tracer using system properties and environment variables, might change in incompatible ways in future releases."

[JBoss EAP 7.2 Configuration Guide Eclipse Microprofile OpenTracing]

• [1] https://github.com/jaegertracing/jaeger-client-java
• [2] https://github.com/jaegertracing/jaeger-client-java/tree/master/jaeger-core#configuration-via-environment

The microprofile-opentracing-smallrye subsystem does not offer any configuration in JBoss EAP 7.2

/subsystem=microprofile-opentracing-smallrye:read-resource-description(recursive=true)
{
    "outcome" => "success",
    "result" => {
        "description" => "Wildfly Extension for Eclipse MicroProfile OpenTracing With SmallRye",
        "attributes" => {},
        "operations" => undefined,
        "notifications" => undefined,
        "children" => {}
    }
}

Jaeger

See https://www.jaegertracing.io/.

Jaeger Openshift Template

See https://github.com/jaegertracing/jaeger-openshift. To run, run below command.

$ oc process -f https://raw.githubusercontent.com/jaegertracing/jaeger-openshift/master/all-in-one/jaeger-all-in-one-template.yml | oc create -f -

Jaeger Docker Image

But we are going to run the all-in-one docker image, see https://hub.docker.com/r/jaegertracing/all-in-one

To install Docker, see http://magnus-k-karlsson.blogspot.com/2019/08/install-docker-community-edition-ce-on.html.

Source code (Dockerfile) https://github.com/jaegertracing/jaeger/blob/master/cmd/all-in-one/Dockerfile

To run

$ docker run -d -p 5775:5775/udp -p 6831:6831/udp -p 6832:6832/udp -p 5778:5778 -p 14268:14268 -p 14250:14250 -p 16686:16686 jaegertracing/all-in-one

Web Application

Now we are going to write our application, use Java EE 8 and Eclipse Microprofile 1.4 from http://magnus-k-karlsson.blogspot.com/2019/10/eclipse-microprofile-and-java-ee-8-with.html.

Now lets write a @Traced JAX-RS endpoint and a CDI bean.

package se.magnuskkarlsson.example.microprofile.boundary;

import javax.inject.Inject;
import javax.json.Json;
import javax.json.JsonObject;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

import org.eclipse.microprofile.opentracing.Traced;

import se.magnuskkarlsson.example.microprofile.control.PersonService;

@Traced
@Path("/persons")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public class PersonResource {

    @Inject
    protected PersonService personService;

    @GET
    public Response hello() {
        JsonObject json = Json.createObjectBuilder().add("message", personService.getMessage()).build();
        return Response.ok(json.toString()).build();
    }

}

package se.magnuskkarlsson.example.microprofile.control;

import org.eclipse.microprofile.opentracing.Traced;

@Traced
public class PersonService {

    public String getMessage() {
        return "HELLO " + System.currentTimeMillis();
    }

}

Also see JBoss EAP 7.2 Development Guide Eclipse Microprofile OpenTracing

Now we need Jaeger client configuration, configuration.properties

JAEGER_SERVICE_NAME=magnuskkarlsson
JAEGER_ENDPOINT=http://127.0.0.1:14268/api/traces
JAEGER_SAMPLER_TYPE=const
JAEGER_SAMPLER_PARAM=1

Build, deploy and start JBoss EAP 7.2 with above configuration file.

$ ./standalone.sh -P /home/magnuskkarlsson/eclipse-workspace/javaee8-microprofile1.4-example/configuration.properties
...
20:06:17,308 DEBUG [io.jaegertracing.thrift.internal.senders.ThriftSenderFactory] (ServerService Thread Pool -- 72) Using the HTTP Sender to send spans directly to the endpoint.
20:06:17,409 DEBUG [io.jaegertracing.internal.senders.SenderResolver] (ServerService Thread Pool -- 72) Using sender HttpSender()
20:06:17,417 INFO  [io.jaegertracing.Configuration] (ServerService Thread Pool -- 72) Initialized tracer=JaegerTracer(version=Java-0.30.6.redhat-00001, serviceName=magnuskkarlsson, reporter=RemoteReporter(sender=HttpSender(), closeEnqueueTimeout=1000), sampler=ConstSampler(decision=true, tags={sampler.type=const, sampler.param=true}), tags={hostname=localhost.localdomain, jaeger.version=Java-0.30.6.redhat-00001, ip=127.0.0.1}, zipkinSharedRpcSpan=false, expandExceptionLogs=false)
...

Now open web application and make a few requests http://localhost:8080/javaee8-microprofile1.4-example/rest/persons.

Now lets see if those traces are sent to Jaeger, open http://127.0.0.1:16686/search

Eclipse Microprofile Health with Java EE 8 and JBoss EAP 7.2

In my previous blogs we have layed out a maven project for Eclipse Microprofile and tested Eclipse Microprofile Configuration.

In this blog we will test Eclipse Microprofile Health with JBoss EAP 7.2. First all Eclipse Microprofile in JBoss EAP 7.2 are "Technology Previews only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs)". [https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.2/html-single/configuration_guide/index#microprofile_health_check]

"By default, the MicroProfile Health SmallRye subsystem only examines if the server is running."

You can invoke/call Health in two ways. Through CLI or HTTP.

[standalone@localhost:9990 /] /subsystem=microprofile-health-smallrye:check
{
    "outcome" => "success",
    "result" => {
        "outcome" => "UP",
        "checks" => []
    }
}

The HTTP endpoint is only exposed through the management interface. "The default address for the /health endpoint, accessible from the management interfaces, is http://127.0.0.1:9990/health."

And also there is not much of configuration you can do.

[standalone@localhost:9990 /] /subsystem=microprofile-health-smallrye:read-resource-description(recursive=true, inherited=true)
{
    "outcome" => "success",
    "result" => {
        "description" => "WildFly Extension for Eclipse MicroProfile Health With SmallRye",
        "capabilities" => [
            {
                "name" => "org.wildfly.extension.microprofile.health.smallrye",
                "dynamic" => false
            },
            {
                "name" => "org.wildlfy.microprofile.health.reporter",
                "dynamic" => false
            }
        ],
        "attributes" => {"security-enabled" => {
            "type" => BOOLEAN,
            "description" => "True if authentication is required to access the HTTP endpoint on the HTTP management interface.",
            "expressions-allowed" => true,
            "required" => false,
            "nillable" => true,
            "default" => true,
            "access-type" => "read-write",
            "storage" => "configuration",
            "restart-required" => "all-services"
        }},
        "operations" => undefined,
        "notifications" => undefined,
        "children" => {}
    }
}

This is all not that exciting. But what is when you start to write application specific health check. You do that by implementing org.eclipse.microprofile.health.HealthCheck and annotating class with org.eclipse.microprofile.health.Health.

The best practice is to implement a specific health check in a separate class and then will jboss global health check aggregates all the result from each health check to the overall health check result. So lets start to write a JPA health check.

package se.magnuskkarlsson.example.microprofile;

import java.util.logging.Level;
import java.util.logging.Logger;

import javax.enterprise.context.ApplicationScoped;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

import org.eclipse.microprofile.health.Health;
import org.eclipse.microprofile.health.HealthCheck;
import org.eclipse.microprofile.health.HealthCheckResponse;

@Health
@ApplicationScoped
public class JPAHealthCheck implements HealthCheck {

    private final Logger log = Logger.getLogger(JPAHealthCheck.class.getName());

    @PersistenceContext
    protected EntityManager em;

    @Override
    public HealthCheckResponse call() {
        Integer result = null;
        try {
            result = (Integer) em.createNativeQuery("SELECT 1").getSingleResult();
        } catch (Exception e) {
            log.log(Level.SEVERE, "Failed to perform JPA health check.", e);
        }
        boolean state = (result == 1) ? true : false;
        return HealthCheckResponse.named("jpa-health-check").withData("SELECT 1", result).state(state).build();
    }

}

Build, deploy and test.

[standalone@localhost:9990 /] /subsystem=microprofile-health-smallrye:check
{
    "outcome" => "success",
    "result" => {
        "outcome" => "UP",
        "checks" => [{
            "name" => "health-test",
            "state" => "UP",
            "data" => {"JPA 'SELECT 1'" => "1"}
        }]
    }
}

Now build a second health check for disk space, but here we will deliberate hard code the output state to down, to test jboss aggregated state value.

package se.magnuskkarlsson.example.microprofile;

import java.io.File;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.enterprise.context.ApplicationScoped;

import org.eclipse.microprofile.health.Health;
import org.eclipse.microprofile.health.HealthCheck;
import org.eclipse.microprofile.health.HealthCheckResponse;

@Health
@ApplicationScoped
public class DiskSpaceHealthCheck implements HealthCheck {

    private final Logger log = Logger.getLogger(DiskSpaceHealthCheck.class.getName());

    @Override
    public HealthCheckResponse call() {
        File path = new File("/");
        long diskFreeInBytes = 0L;
        try {
            diskFreeInBytes = path.getUsableSpace();
        } catch (Exception e) {
            log.log(Level.SEVERE, "Failed to perform disk space health check.", e);
        }
        boolean state = false;
        return HealthCheckResponse.named("disk-space-health-check").withData(path.getAbsolutePath(), diskFreeInBytes)
                .state(state).build();
    }

}

Build, deploy and test.

[standalone@localhost:9990 /] /subsystem=microprofile-health-smallrye:check
{
    "outcome" => "success",
    "result" => {
        "outcome" => "DOWN",
        "checks" => [
            {
                "name" => "jpa-health-check",
                "state" => "UP",
                "data" => {"SELECT 1" => 1}
            },
            {
                "name" => "disk-space-health-check",
                "state" => "DOWN",
                "data" => {"/" => 409898860544L}
            }
        ]
    }
}

As you can see the overall state is DOWN.

To get more inspiration for others health check, see spring boots health indicators - https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/actuate/health/AbstractHealthIndicator.html

• Data Source Health Indicator
• Disk Space Health Indicator
• JMS Health Indicator
• LDAP Health Indicator
• Mail Health Indicator

And finally it is of interest to read the Eclipse Microprofile Health Specification.

Eclipse Microprofile Configuration with Java EE 8 and JBoss EAP 7.2

Here we are going to look at Eclipse Microprofile Configuration with Java EE 8 and JBoss EAP 7.2.

The configuration project is a convenient project to easily load configuration from:

• System Property, e.g. -Dnum.size=20
• Properties file, e.g. configuration.properties

To get started we use the previous blog project setup, with pom.xml, web.xml, beans.xml and persistence.xml. Please see https://magnus-k-karlsson.blogspot.com/2019/10/eclipse-microprofile-and-java-ee-8-with.html.

And the injection configuration:

package se.magnuskkarlsson.example.microprofile.boundary;

import javax.inject.Inject;
import javax.json.Json;
import javax.json.JsonObject;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

import org.eclipse.microprofile.config.inject.ConfigProperty;

@Path("/persons")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public class PersonResource {

    @Inject
    @ConfigProperty(name = "num.size", defaultValue = "12")
    int numSize;

    @GET
    public Response hello() {
        JsonObject json = Json.createObjectBuilder().add("message", "HELLO " + numSize).build();
        return Response.ok(json.toString()).build();
    }

}

And to make JAX-RS complete we also need

package se.magnuskkarlsson.example.microprofile;

import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;

@ApplicationPath("/rest")
public class JAXRSApplication extends Application {

}

Now we can build and deploy the web application. To configure we have now two options

• System Properties, e.g. ./standalone.sh -Dnum.size=20
• Property File, e.g. ./standalone.sh -P ../standalone/configuration/configuration.properties

Test both ways, but for production, property file is the recommended way.

Eclipse Microprofile and Java EE 8 with JBoss EAP 7.2

The Eclipse Microprofile is a spin of from Java EE and is a fast pace standardization community for building standardized Microservices. There are several subprojects within Microprofile and there are constantly more added all the time. To see all full list visit https://microprofile.io/projects/.

To get started to use Eclipse Microprofile you simple add the maven dependency org.eclipse.microprofile:microprofile.

Before using Eclipse Microprofile, you must first check which version your Java Container supports, alternative you can packages the binaries inside your web application.

For JBoss EAP 7.2 it does not support the entire Eclipse Microprofile version 1.4, but a subset of it. To get started use the following pom.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>se.magnuskkarlsson.examples</groupId>
    <artifactId>javaee8-microprofile1.4-example</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
        <failOnMissingWebXml>false</failOnMissingWebXml>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-api</artifactId>
            <version>8.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.eclipse.microprofile</groupId>
            <artifactId>microprofile</artifactId>
            <version>1.4</version>
            <type>pom</type>
            <scope>provided</scope>
        </dependency>

        <!-- Test Support -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-core</artifactId>
            <version>2.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-all</artifactId>
            <version>1.10.19</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.199</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.17</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <finalName>${project.artifactId}</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <release>11</release>
                    <showDeprecation>true</showDeprecation>
                    <showWarnings>true</showWarnings>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

And the src/main/webapp/WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd"
    version="4.0">

</web-app>

And the src/main/webapp/WEB-INF/beans.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://xmlns.jcp.org/xml/ns/javaee" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/beans_2_0.xsd"
    version="2.0" bean-discovery-mode="all">

</beans>

And the src/main/resources/META-INF/persistence.xml

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://xmlns.jcp.org/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence http://xmlns.jcp.org/xml/ns/persistence/persistence_2_2.xsd"
    version="2.2">

    <persistence-unit name="PROD"></persistence-unit>
</persistence>

Getting Started with RH SSO 7.3 (Keycloak)

Introduction

In this blog we are going to install and configure RH SSO 7.3 (upstream Keycloak) from scratch and then develop a simple Java Web application, protected with RH SSO (Keycloak).

We are going to run the RH SSO 7.3 and JBoss EAP 7.2 on the same machine and locally, but before that we need to install Java 11 (OpenJDK 11). Since this is a developing machine we are going to install every OpenJDK 11 packages.

$ sudo dnf install -y java-11-openjdk java-11-openjdk-src java-11-openjdk-demo java-11-openjdk-devel java-11-openjdk-jmods java-11-openjdk-javadoc java-11-openjdk-headless java-11-openjdk-javadoc-zip

If you have multiple of Java versions installed, please set Java 11 as default

$ sudo update-alternatives --config java

$ java -version
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment 18.9 (build 11.0.4+11)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.4+11, mixed mode, sharing)

RH SSO 7.3 (Keycloak)

Now install RH SSO from ZIP (in production you would install it via RPM).

$ unzip rh-sso-7.3.0.GA.zip
$ mv rh-sso-7.3 rh-sso-7.3.0
$ cd rh-sso-7.3.0/bin/

$ ./standalone.sh

Now configure Admin account, by open http://localhost:8080/auth/ and then open http://localhost:8080/auth/admin/ and login as Admin.

1. In the left menu create a new Realm 'demo'.
3. In the demo realm create a new User with username 'joe'
5. After created User, click on User Credential tab.
7. And reset User 'joe' password and don't forget to set Temporary to false.
9. Then create a new Role, by clicking on left menu Role and then Add role.
10. Lets name the Role to 'OIDCDEMO_USER'.
12. Go back to User by clicking on User in lefter meny and find user 'joe'.
14. After clicking on User 'joe', then click on tab Role Mappings.
16. Select our newly created Role 'OIDCDEMO_USER' under Available Roles and click Add selected.

Web Application Server - JBoss EAP 7.2

Install via zip (in production you would install via RPM). After extracting the zip files, we install OpenID Connector to EAP via adapter-elytron-install-offline.cli and then starts EAP with a port offset of 100. To test it open http://localhost:8180/

$ unzip jboss-eap-7.2.0.zip
$ mv jboss-eap-7.2 jboss-eap-7.2.0-oidc
$ unzip rh-sso-7.3.0.GA-eap7-adapter.zip -d jboss-eap-7.2.0-oidc
$ cp jboss-eap-7.2.0-oidc/standalone/configuration/standalone.xml jboss-eap-7.2.0-oidc/standalone/configuration/standalone.xml.ORIG

$ cd jboss-eap-7.2.0-oidc/bin
$ ./jboss-cli.sh --file=adapter-elytron-install-offline.cli

$ ./standalone.sh -Djboss.socket.binding.port-offset=100

The Java EE 8 Web Application

Now lets create a simple Java EE 8 Web Application, with a single index.jsp and a logout.jsp.

$ mkdir rh-sso-7.3-demo
$ cd rh-sso-7.3-demo 
$ mkdir -p src/main/java/se/magnuskkarlsson/example/oidc
$ mkdir -p src/main/resources/META-INF
$ mkdir -p src/main/webapp/WEB-INF
$ touch src/main/webapp/WEB-INF/web.xml
$ mkdir -p src/test/java/se/magnuskkarlsson/example/oidc
$ mkdir -p src/test/resources
$ vi pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>se.magnuskkarlsson.examples</groupId>
    <artifactId>rh-sso-7.3-demo</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
        <failOnMissingWebXml>false</failOnMissingWebXml>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-api</artifactId>
            <version>8.0</version>
            <scope>provided</scope>
        </dependency>

        <!-- Test Support -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-core</artifactId>
            <version>2.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-all</artifactId>
            <version>1.10.19</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.199</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.17</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <finalName>${project.artifactId}</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <release>11</release>
                    <showDeprecation>true</showDeprecation>
                    <showWarnings>true</showWarnings>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Now the important part of web.xml, where we configure our security.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd"
    version="4.0">

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Unprotected pages</web-resource-name>
            <url-pattern>/logout.jsp</url-pattern>
        </web-resource-collection>
        <!-- <user-data-constraint> <transport-guarantee>CONFIDENTIAL</transport-guarantee> </user-data-constraint> -->
    </security-constraint>

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Protected pages</web-resource-name>
            <url-pattern>/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>OIDCDEMO_USER</role-name>
        </auth-constraint>
        <!-- <user-data-constraint> <transport-guarantee>CONFIDENTIAL</transport-guarantee> </user-data-constraint> -->
    </security-constraint>

    <login-config>
        <auth-method>KEYCLOAK</auth-method>
        <!-- <realm-name>demo</realm-name> -->
    </login-config>

    <security-role>
        <role-name>OIDCDEMO_USER</role-name>
    </security-role>

    <session-config>
        <session-timeout>15</session-timeout>
        <cookie-config>
            <http-only>true</http-only>
            <!-- <secure>true</secure> -->
        </cookie-config>
        <tracking-mode>COOKIE</tracking-mode>
    </session-config>
</web-app>

And our index.jsp and logout.jsp

<%@ page import="java.util.*"%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Secure Web App</title>
<style>
table, th, td {
    border: 1px solid black;
    border-collapse: collapse;
}

th, td {
    padding: 10px;
}
</style>
</head>
<body>
    <table border="1">
        <!-- Security -->
        <tr>
            <td>getAuthType:</td>
            <td><%=request.getAuthType()%></td>
        </tr>
        <tr>
            <td>isSecure:</td>
            <td><%=request.isSecure()%></td>
        </tr>
        <tr>
            <td>getRemoteUser:</td>
            <td><%=request.getRemoteUser()%></td>
        </tr>
        <tr>
            <td>getUserPrincipal:</td>
            <td><%=request.getUserPrincipal()%></td>
        </tr>
        <tr>
            <td>getSession(false):</td>
            <td><%=request.getSession(false)%></td>
        </tr>
        <tr>
            <td>isUserInRole(<%=request.getParameter("UserInRole")%>):
            </td>
            <td>
                <div>
                    <form action="index.jsp" method="get">
                        <input type="text" id="UserInRole" name="UserInRole" /> <input type="submit" name="Submit" />
                    </form>
                </div>
                <div>
                    <%=request.isUserInRole(request.getParameter("UserInRole"))%>
                </div>
            </td>
        </tr>
        <!-- Cookies and Headers -->
        <tr>
            <td>Cookies:</td>
            <td>
                <%
                    for (int i = 0; request.getCookies() != null && i < request.getCookies().length; ++i) {
                        Cookie cookie = request.getCookies()[i];
                        out.println(cookie.getPath() + " " + cookie.getDomain() + " " + cookie.getName() + "="
                                + cookie.getValue() + " " + cookie.isHttpOnly() + " " + cookie.getSecure() + "<br/>");
                    }
                %>
            </td>
        </tr>
        <tr>
            <td>Header:</td>
            <td>
                <%
                    for (Enumeration<String> headers = request.getHeaderNames(); headers.hasMoreElements();) {
                        String header = headers.nextElement();
                        out.println(header + ": " + request.getHeader(header) + "<br/>");
                    }
                %>
            </td>
        </tr>
        <!-- Locale Address -->
        <tr>
            <td>getLocalAddr:</td>
            <td><%=request.getLocalAddr()%></td>
        </tr>
        <tr>
            <td>getLocalName:</td>
            <td><%=request.getLocalName()%></td>
        </tr>
        <tr>
            <td>getLocalPort:</td>
            <td><%=request.getLocalPort()%></td>
        </tr>
        <!-- Remote Address -->
        <tr>
            <td>getRemoteAddr:</td>
            <td><%=request.getRemoteAddr()%></td>
        </tr>
        <tr>
            <td>getLocalName:</td>
            <td><%=request.getRemoteHost()%></td>
        </tr>
        <tr>
            <td>getRemotePort:</td>
            <td><%=request.getRemotePort()%></td>
        </tr>
        <!-- Different Paths -->
        <tr>
            <td>getContextPath:</td>
            <td><%=request.getContextPath()%></td>
        </tr>
        <tr>
            <td>getPathInfo:</td>
            <td><%=request.getPathInfo()%></td>
        </tr>
        <tr>
            <td>getQueryString:</td>
            <td><%=request.getQueryString()%></td>
        </tr>
        <tr>
            <td>getRequestURI:</td>
            <td><%=request.getRequestURI()%></td>
        </tr>
        <tr>
            <td>getRequestURL:</td>
            <td><%=request.getRequestURL()%></td>
        </tr>
        <tr>
            <td>getServletPath:</td>
            <td><%=request.getServletPath()%></td>
        </tr>
    </table>
</body>
</html>

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Logout</title>
</head>
<body>
    <%
        if (request.getSession(false) != null) {
            request.getSession(false).invalidate();
        }
        request.logout();
        /*
        "You can log out of a web application in multiple ways. For Java EE servlet containers, you 
        can call HttpServletRequest.logout(). For other browser applications, you can redirect 
        the browser to 
        http://auth-server/auth/realms/{realm-name}/protocol/openid-connect/logout?redirect_uri=encodedRedirectUri, 
        which logs you out if you have an SSO session with your browser."
        [https://access.redhat.com/documentation/en-us/red_hat_single_sign-on/7.3/html/securing_applications_and_services_guide/openid_connect_3#logout]
        */
    %>
    <h3>You are logged out</h3>
</body>
</html>

Configure OpenID Connect for Web Application

Now we need to configure OpenID Connect for our Web Application in RH SSO

And finally add OpenID Connect configuration to our web application src/main/webapp/WEB-INF/keycloak.json.

{
 "realm": "demo",
 "auth-server-url": "http://localhost:8080/auth",
 "ssl-required": "external",
 "resource": "rh-sso-7.3-demo",
 "public-client": true,
 "enable-cors": true
}

References

1. https://www.keycloak.org/docs/latest/getting_started/index.html
2. https://github.com/keycloak/keycloak-quickstarts/tree/latest/app-profile-jee-vanilla
3. https://access.redhat.com/documentation/en-us/red_hat_single_sign-on/7.3/html/securing_applications_and_services_guide/openid_connect_3#java_adapters

Minimalistic POM for Java 11 and Java EE 8

One of the greatest news with Java EE 7 and 8, is that there is ONE dependency for the entire Java EE.

And here is a minimalistic POM for your Java EE 8 projects with Java 11. I also added junit, hamcrest, mockito and in-memory DB H2 and mysql jdbc driver which are not necessary, but I use them a lot, so I added them for convenience.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>se.magnuskkarlsson.examples</groupId>
    <artifactId>dockerwar</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
        <failOnMissingWebXml>false</failOnMissingWebXml>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-api</artifactId>
            <version>8.0</version>
            <scope>provided</scope>
        </dependency>

        <!-- Test Support -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-core</artifactId>
            <version>2.1</version>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-all</artifactId>
            <version>1.10.19</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.199</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.17</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <release>11</release>
                    <showDeprecation>true</showDeprecation>
                    <showWarnings>true</showWarnings>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

If you need in memory database checkout "H2 Database Engine Cheat Sheet" [http://www.h2database.com/html/cheatSheet.html]

HTTP Client in Java 8 HttpURLConnection

The standard HTTP Client in Java 8 is HttpURLConnection, but working with it is not so intuitively, so I have written a small helper class, that does:

• All HTTP method: GET, POST, PUT, DELETE, etc.
• Sets HTTP Request Headers
• Sets HTTP POST Body
• Read response
• Read response Body
• Read response Headers

package se.magnuskkarlsson.example.httpurlconnection;

import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class HttpURLConnectionBuilder {

    // See javax.ws.rs.core.HttpHeaders

    // Body type in request
    public static final String CONTENT_TYPE = "Content-Type: application/json";

    // Accepted response by client
    public static final String ACCEPT = "Accept";

    // See javax.ws.rs.core.MediaType

    // Media Type 'application/json'
    public final static String APPLICATION_JSON = "application/json";

    private URL url;
    private String method;
    private Map<String, String> headers = new HashMap<String, String>();
    private String body;

    public HttpURLConnectionBuilder() {
    }

    // ----------------------- Logic Methods -----------------------

    public HttpURLConnectionBuilder get(URL url) {
        return method(url, "GET");
    }

    public HttpURLConnectionBuilder post(URL url) {
        return method(url, "POST");
    }

    public HttpURLConnectionBuilder put(URL url) {
        return method(url, "PUT");
    }

    public HttpURLConnectionBuilder delete(URL url) {
        return method(url, "DELETE");
    }

    public HttpURLConnectionBuilder method(URL url, String method) {
        this.url = url;
        this.method = method;
        return this;
    }

    public HttpURLConnectionBuilder header(String name, String value) {
        this.headers.put(name, value);
        return this;
    }

    public HttpURLConnectionBuilder body(String body) {
        this.body = body;
        return this;
    }

    public HttpURLConnectionResponse send() throws IOException {
        HttpURLConnection conn = (HttpURLConnection) this.url.openConnection();

        // Set Request Method
        conn.setRequestMethod(this.method);

        // Set Request Headers
        for (String name : this.headers.keySet()) {
            conn.setRequestProperty(name, this.headers.get(name));
        }

        // Trace Log Request
        System.out.println(conn.getRequestMethod() + " " + conn.getURL());
        for (String key : conn.getRequestProperties().keySet()) {
            System.out.println(
                    "  >> " + key + ": " + conn.getRequestProperty(key));
        }

        // Set Request Body
        if (this.body != null) {
            conn.setDoOutput(true);
            try (OutputStreamWriter output = new OutputStreamWriter(
                    new BufferedOutputStream(conn.getOutputStream()),
                    StandardCharsets.UTF_8)) {

                output.write(this.body);
            }
        }

        // Send Request and Get Response Code
        int responseCode = conn.getResponseCode();

        // Get Response Body
        InputStream responseStream = null;
        if (responseCode >= 200 && responseCode <= 299) {

            responseStream = conn.getInputStream();
        } else {

            responseStream = conn.getErrorStream();
        }
        String responseBody = null;
        try (BufferedReader in = new BufferedReader(new InputStreamReader(
                responseStream, StandardCharsets.UTF_8))) {

            StringBuilder builder = new StringBuilder();
            for (String line = null; (line = in.readLine()) != null;) {
                builder.append(line);
            }
            responseBody = builder.toString();
        }

        // Get Response Headers
        Map<String, List<String>> responseHeaders = conn.getHeaderFields();

        // Trace Log Response
        System.out.println("<< " + conn.getHeaderField(null));
        for (String key : conn.getHeaderFields().keySet()) {
            if (key != null) {
                System.out.println(
                        "  << " + key + ": " + conn.getHeaderField(key));
            }
        }

        return new HttpURLConnectionResponse(responseCode, responseBody,
                responseHeaders);
    }

    // ----------------------- Helper Methods -----------------------

    // ----------------------- Get and Set Methods -----------------------

}

And the response class is a simple immutable POJO.

package se.magnuskkarlsson.example.httpurlconnection;

import java.util.List;
import java.util.Map;

public class HttpURLConnectionResponse {

    private final int statusCode;
    private final String body;
    private final Map<String, List<String>> headers;

    public HttpURLConnectionResponse(int statusCode, String body,
            Map<String, List<String>> headers) {

        this.statusCode = statusCode;
        this.body = body;
        this.headers = headers;
    }

    // ----------------------- Logic Methods -----------------------

    // ----------------------- Helper Methods -----------------------

    // ----------------------- Get and Set Methods -----------------------

    public int getStatusCode() {
        return statusCode;
    }

    public String getBody() {
        return body;
    }

    public Map<String, List<String>> getHeaders() {
        return headers;
    }

}

Now lets test it, with a simple REST service.

package se.magnuskkarlsson.example.httpurlconnection;

import java.util.logging.Logger;

import javax.json.Json;
import javax.json.JsonObject;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriBuilder;
import javax.ws.rs.core.UriInfo;

@Path("/persons")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public class PersonResource {

    private final Logger log = Logger.getLogger(PersonResource.class.getName());

    // ----------------------- Logic Methods -----------------------

    // Common Response Codes For All Requests
    // 200 OK Request was successful
    // 401 Unauthorized User not authenticated
    // 403 Forbidden User not authorized for endpoint request, often due to
    // missing security privileges

    // Common Error Response Codes For GET
    // 404 Not Found Resource not found
    // 400 Bad Request Filter parameters provided were not valid Recommended
    // Response Codes

    @GET
    @Path("/{personId}")
    public Response getById(@PathParam("personId") String personId) {
        if ("404".equals(personId)) {

            JsonObject json = createProblemDetails(null, "Not Found", 404,
                    "Person with ID " + personId + " does not exists.", null);
            return Response.status(Response.Status.NOT_FOUND).entity(json)
                    .build();
        }
        JsonObject json = Json.createObjectBuilder().add("personId", personId)
                .add("name", "Magnus").build();
        return Response.ok(json).build();
    }

    // Common Response Codes For POST
    // 200 OK A custom action or processing request was successful
    // 201 Created Created resource successfully
    // 202 Accepted Long-running request has been accepted
    // 400 Bad Request Fields were not valid or required fields were missing

    @POST
    public Response create(String name, @Context UriInfo uriInfo) {
        log.info("name=" + name);
        UriBuilder builder = uriInfo.getAbsolutePathBuilder();
        builder.path("32");
        // CREATED 201
        return Response.created(builder.build()).build();
    }

    // Common Error Response Codes For PUT or PATCH
    // 200 OK Updated resource successfully
    // 400 Bad Request Fields were not valid or required fields were missing
    // 404 Not Found Resource not found

    // Common Error Response Codes For DELETE
    // 204 No Content Resource deleted
    // 404 Not Found Resource not found

    // ----------------------- Helper Methods -----------------------

    // Problem Details for HTTP APIs https://tools.ietf.org/html/rfc7807
    protected JsonObject createProblemDetails(String type, String title,
            int status, String detail, String instance) {
        type = (type != null) ? type : "about:blank";
        instance = (instance != null) ? instance : "";
        return Json.createObjectBuilder().add("type", type).add("title", title)
                .add("status", status).add("detail", detail)
                .add("instance", instance).build();
    }

    // ----------------------- Get and Set Methods -----------------------

}

And now lets write a System Test for your REST service.

package se.magnuskkarlsson.example.httpurlconnection;

import static org.hamcrest.CoreMatchers.is;
import static se.magnuskkarlsson.example.httpurlconnection.HttpURLConnectionBuilder.*;

import java.net.URL;

import org.hamcrest.CoreMatchers;
import org.junit.Assert;
import org.junit.Test;

public class HttpURLConnectionST {

    @Test
    public void test() throws Exception {
        URL url = new URL(
                "http://localhost:8080/example-java8-httpurlconnection/rest/persons/12");

        HttpURLConnectionResponse response = new HttpURLConnectionBuilder()
                .get(url) //
                .header(CONTENT_TYPE, APPLICATION_JSON) //
                .header(ACCEPT, APPLICATION_JSON) //
                .send();

        Assert.assertThat(response.getStatusCode(), is(200));
        Assert.assertThat(response.getBody(),
                is("{\"personId\":\"12\",\"name\":\"Magnus\"}"));
    }

    @Test
    public void testNonExisting() throws Exception {
        URL url = new URL(
                "http://localhost:8080/example-java8-httpurlconnection/rest/persons/404");

        HttpURLConnectionResponse response = new HttpURLConnectionBuilder()
                .get(url) //
                .header(CONTENT_TYPE, APPLICATION_JSON) //
                .header(ACCEPT, APPLICATION_JSON) //
                .send();

        Assert.assertThat(response.getStatusCode(), is(404));
        Assert.assertThat(response.getBody(),
                CoreMatchers.containsString("\"title\":\"Not Found\""));
    }

    @Test
    public void testPost() throws Exception {
        URL url = new URL(
                "http://localhost:8080/example-java8-httpurlconnection/rest/persons");

        HttpURLConnectionResponse response = new HttpURLConnectionBuilder()
                .post(url) //
                .header(CONTENT_TYPE, APPLICATION_JSON) //
                .header(ACCEPT, APPLICATION_JSON) //
                .body("{\"name\":\"FOO\"}") //
                .send();

        Assert.assertThat(response.getStatusCode(), is(201));
        Assert.assertThat(response.getBody(), is(""));
        Assert.assertThat(response.getHeaders().get("Location").get(0),
                CoreMatchers.notNullValue());
    }

}

Reading NSS DB from Java 11 with SunPKCS11

Introduction

In my previous blog (Reading NSS DB from Java 8 with SunPKCS11) I showed how to read a NSS DB from Java 8 with SunPKCS11. In Java 11 a lot of internal packages is no longer visible and you will get compilation error if you try to access them. The same is true for sun.security.pkcs11.SunPKCS11, since it is a sun class. But SunPKCS11 is still available. What they have done is to make SunPKCS11 always available, i.e. you do not need to add, i.e. Security.addProvider(provider);

Java 8

String configName = "/home/magnuskkarlsson/NetBeansProjects/example-nssdb/pkcs11.cfg";
sun.security.pkcs11.SunPKCS11 provider = new sun.security.pkcs11.SunPKCS11(configName);
Security.addProvider(provider);

Java 11

String configName = "/home/magnuskkarlsson/NetBeansProjects/example-nssdb/pkcs11.cfg";
Provider prototype = Security.getProvider("SunPKCS11");
Provider provider = prototype.configure(configName);

One gotcha with this in Java 11, is that you need to specify the provider when you try to do cryptographic operation like signature, otherwise you will get the following error.

Exception in thread "main" java.security.InvalidKeyException: No installed provider supports this key: sun.security.pkcs11.P11Key$P11PrivateKey

So how have they solved, so that SunPKCS11 is always loaded? Through the java.security which has also moved, due to changes in directory layout in Java 11.

Java 8: $JAVA_HOME/lib/security/java.security

Java 11: $JAVA_HOME/conf/security/java.security

So in Java 11 you can see the following

security.provider.12=SunPKCS11
#security.provider.1=SunPKCS11 ${java.home}/lib/security/nss.cfg

And if you look inside $JAVA_HOME/lib/security/nss.cfg, you see a ready to use EMPTY NSS DB configuration. To read about all SunPKCS11 NSS DB configuration see https://docs.oracle.com/en/java/javase/11/security/pkcs11-reference-guide1.html#GUID-7989F8B4-7260-4908-8203-99056B2D060E

name = NSS
nssLibraryDirectory = /usr/lib64
nssDbMode = noDb
attributes = compatibility
handleStartupErrors = ignoreMultipleInitialisation

Testing Time

Now lets test the SunPKCS11 with Java 11. First lets create a new NSS DB with the new SQLite format (cert9.db, key4.db, and pkcs11.txt) and then add a keypair and a self-signed certificate in the internal token.

$ echo "redhat123" > password.internal
$ mkdir nssdb_sql

$ certutil -N -d sql:nssdb_sql -f password.internal

$ certutil -S -x -d sql:nssdb_sql -f password.internal -n mkk -s 'CN=MKK, O=MKK Consultancy, C=SE' -k rsa -g 2048 -v 24 -Z SHA256 -t ',,'

And don't forget to add an empty secmod.db.

$ touch nssdb_sql/secmod.db

Then create the SunPKCS11 configuration file pkcs11.cfg.

name = NSScrypto
nssLibraryDirectory = /usr/lib64 
nssSecmodDirectory = /home/magnuskkarlsson/NetBeansProjects/example-nssdb/nssdb_sql
nssDbMode = readWrite
nssModule = keystore

package se.magnuskkarlsson.example.nssdb;

import static java.nio.charset.StandardCharsets.UTF_8;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.Provider;
import java.security.PublicKey;
import java.security.Security;
import java.security.Signature;
import java.security.cert.X509Certificate;
import java.util.Base64;

import java.util.Enumeration;

// https://docs.oracle.com/javase/8/docs/technotes/guides/security/p11guide.html#NSS
// https://docs.oracle.com/en/java/javase/11/security/pkcs11-reference-guide1.html#GUID-85EA1017-E59C-49B9-9207-65B7B2BF171E
// https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Reference/NSS_tools_:_certutil
/*
          certutil supports two types of databases: the legacy
          security databases (cert8.db, key3.db, and secmod.db)
          and new SQLite databases (cert9.db, key4.db, and
          pkcs11.txt).
 */
 /*
name = NSScrypto
nssLibraryDirectory = /usr/lib64 
nssSecmodDirectory = /home/magnuskkarlsson/NetBeansProjects/example-nssdb/nssdb_sql
nssDbMode = readWrite
nssModule = keystore
 */
public class NSSSunPKCS11Tool {

    public static char[] password = "redhat123".toCharArray();

    public static void main(String[] args) throws Exception {
        String configName = "/home/magnuskkarlsson/NetBeansProjects/example-nssdb/pkcs11.cfg";

        // Java 8
//        sun.security.pkcs11.SunPKCS11 provider = new sun.security.pkcs11.SunPKCS11(configName);
//        Security.addProvider(provider);
//        
        // Java 11
        Provider prototype = Security.getProvider("SunPKCS11");
        Provider provider = prototype.configure(configName);

        KeyStore ks = KeyStore.getInstance("PKCS11", provider);
        ks.load(null, password);
        System.out.println("Successfully loaded NSS DB.");
        System.out.println("------------------------------");
        for (Enumeration aliases = ks.aliases(); aliases.hasMoreElements();) {
            String alias = aliases.nextElement();
            X509Certificate cert = (X509Certificate) ks.getCertificate(alias);
            PublicKey publicKey = cert.getPublicKey();
            PrivateKey privateKey = (PrivateKey) ks.getKey(alias, password);
            System.out.println("alias: " + alias);
            System.out.println("privateKey: " + privateKey);
            System.out.println("cert subject dn: " + cert.getSubjectX500Principal().toString());

            if (privateKey != null) {
                String plainText = "HELLO WORLD";

                Signature privateSignature = Signature.getInstance("SHA256withRSA", provider);
                privateSignature.initSign(privateKey);
                privateSignature.update(plainText.getBytes(UTF_8));
                byte[] signature = privateSignature.sign();
                String signatureBase64 = Base64.getEncoder().encodeToString(signature);
                System.out.println(signatureBase64);

                Signature publicSignature = Signature.getInstance("SHA256withRSA", provider);
                publicSignature.initVerify(publicKey);
                publicSignature.update(plainText.getBytes(UTF_8));
                boolean verify = publicSignature.verify(signature);
                System.out.println("verify: " + verify);
            }

            System.out.println("------------------------------");
        }
    }
}

Then lets run it. I have here added the debug flag -Djava.security.debug=sunpkcs11 which is of course not necessary, but useful when running labs to understand things better.

$ mvn clean install; java -cp target/example-nssdb-1.0.0-SNAPSHOT.jar -Djava.security.debug=sunpkcs11 se.magnuskkarlsson.example.nssdb.NSSSunPKCS11Tool

SunPKCS11 loading /home/magnuskkarlsson/NetBeansProjects/example-nssdb/pkcs11.cfg
NSS modules: [NSS Internal PKCS #11 Module (CRYPTO, /usr/lib64/libsoftokn3.so, slot 0), NSS Internal PKCS #11 Module (KEYSTORE, /usr/lib64/libsoftokn3.so, slot 1)]
sunpkcs11: Initializing PKCS#11 library /usr/lib64/libsoftokn3.so
Information for provider SunPKCS11-NSScrypto
Library info:
  cryptokiVersion: 2.20
  manufacturerID: Mozilla Foundation              
  flags: 0
  libraryDescription: NSS Internal Crypto Services    
  libraryVersion: 3.46
All slots: 1, 2
Slots with tokens: 1, 2
Slot info for slot 2:
  slotDescription: NSS User Private Key and Certificate Services                   
  manufacturerID: Mozilla Foundation              
  flags: CKF_TOKEN_PRESENT
  hardwareVersion: 3.46
  firmwareVersion: 0.00
Token info for token in slot 2:
  label: NSS Certificate DB              
  manufacturerID: Mozilla Foundation              
  model: NSS 3           
  serialNumber: 0000000000000000
  flags: CKF_RNG | CKF_LOGIN_REQUIRED | CKF_USER_PIN_INITIALIZED | CKF_DUAL_CRYPTO_OPERATIONS | CKF_TOKEN_INITIALIZED
  ulMaxSessionCount: CK_EFFECTIVELY_INFINITE
  ulSessionCount: 1
  ulMaxRwSessionCount: CK_EFFECTIVELY_INFINITE
  ulRwSessionCount: 0
  ulMaxPinLen: 500
  ulMinPinLen: 0
  ulTotalPublicMemory: 1
  ulFreePublicMemory: 1
  ulTotalPrivateMemory: 1
  ulFreePrivateMemory: 1
  hardwareVersion: 0.00
  firmwareVersion: 0.00
  utcTime: 0000000000000000
...
sunpkcs11: login succeeded
Successfully loaded NSS DB.
------------------------------
alias: mkk
privateKey: SunPKCS11-NSScrypto RSA private key, 2048 bitstoken object, sensitive, extractable)
cert subject dn: CN=MKK, O=MKK Consultancy, C=SE
bKqsRr+exiclVEB1Q59/M/KrElu+dCFKDK9+mertjlj1lJ7LG7Gwrws6CX/m6l3A9bf4nt+yQNYYt/2x3WFITquuBsbMKfyV6J7UHYS7J92+EUNHNXaFR2QVDo5v3Ecy4oPD9ln7LATl1jJnfSs0kiYB7HbOIWjufxfrY65sgQUyR+I3uQaj0+PDJ8WbrUbqCvzdFv3MH+Jv9kDvdp1eBkrPD+yczLdIQy7kDJRmzN34gU6tW85RZ0PpgjZomV3TO3S6hJxeZqH/ijd5yLKlpfQAM2V4dADsnlGmS9KUrZ6JOU/eIFRX6CD0X/eNsCqgUp+vn7JD/SE4NJabUdrVQw==
Demoting session, active: 3
verify: true
------------------------------

Reading NSS DB with SoftHSM Token with Java JSS

Introduction

In my previous blog I showed you how to read a NSS DB with SunPKCS11.. The SunPKCS11 working for both the legacy NSS database format (cert8.db, key3.db, and secmod.db) and the new SQLite format (cert9.db, key4.db, and pkcs11.txt), but SunPKCS11 cannot only read entries in the internal NSS database, not other tokens, such HSM or SmartCard.

To be able to read other tokens in a NSS DB you need to use Network Security Services for Java (JSS) for Java - https://github.com/dogtagpki/jss.

Here in this blog I will use SoftHSM to simulate a real HSM.

SoftHSM v2

References:

• https://www.dogtagpki.org/wiki/NSS_Database
• https://www.dogtagpki.org/wiki/SoftHSM
• man 5 softhsm2.conf

Install

# yum install -y softhsm

Configure a new token in SoftHSM named Dogtag and set PIN and SO (Security Officer/Admin) PIN to the first available slot.

# softhsm2-util --init-token --label "Dogtag" --so-pin redhat321 --pin redhat321 --free
Slot 0 has a free/uninitialized token.
The token has been initialized and is reassigned to slot 634761745

List the files for SoftHSM

# ls -ald /var/lib/softhsm
drwxr-x---. 3 ods ods 20 Sep 15 00:53 /var/lib/softhsm

# ls -ald /var/lib/softhsm/tokens
drwxrwx--T. 3 ods ods 50 Sep 15 00:55 /var/lib/softhsm/tokens

# ls -ald /var/lib/softhsm/tokens/372c8761-c4e8-baab-fa9c-6eb4a5d5b211
drwx------. 2 root root 62 Sep 15 00:55 /var/lib/softhsm/tokens/372c8761-c4e8-baab-fa9c-6eb4a5d5b211

# ls -al /var/lib/softhsm/tokens/372c8761-c4e8-baab-fa9c-6eb4a5d5b211/
total 8
drwx------. 2 root root  62 Sep 15 00:55 .
drwxrwx--T. 3 ods  ods   50 Sep 15 00:55 ..
-rw-------. 1 root root   8 Sep 15 00:55 generation
-rw-------. 1 root root   0 Sep 15 00:55 token.lock
-rw-------. 1 root root 320 Sep 15 00:55 token.object

Disable p11-kit

Here we are going to disable p11-kit and manually add SoftHSM module to a NSS DB.

Reference: https://pagure.io/freeipa/issue/7810

# rm -f /etc/crypto-policies/local.d/nss-p11-kit.config && update-crypto-policies

# reboot

NSS DB

Now lets create our NSS DB. We are going to use the new format SQLite and will explicitly use the prefix 'sql:' to mark that.

First install nss-tools.

# yum install -y nss-tools

Then create our new NSS DB.

# echo "redhat321" > password.softhsm
# echo "redhat123" > password.internal
# mkdir nssdb_sql_softhsm
# certutil -N -d sql:nssdb_sql_softhsm -f password.internal

Then we manually add the SoftHSM module to that NSS DB.

# rpm -ql softhsm | grep libsofthsm2.so
/usr/lib64/libsofthsm2.so
/usr/lib64/pkcs11/libsofthsm2.so

# modutil -dbdir sql:nssdb_sql_softhsm -add softhsm -libfile /usr/lib64/pkcs11/libsofthsm2.so -force

# modutil -dbdir sql:nssdb_sql_softhsm -add softhsm -libfile /usr/lib64/pkcs11/libsofthsm2.so -force

# certutil -U -d sql:nssdb_sql_softhsm

    slot: NSS Internal Cryptographic Services
   token: NSS Generic Crypto Services
     uri: pkcs11:token=NSS%20Generic%20Crypto%20Services;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203

    slot: NSS User Private Key and Certificate Services
   token: NSS Certificate DB
     uri: pkcs11:token=NSS%20Certificate%20DB;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203

    slot: SoftHSM slot ID 0x25d5b211
   token: Dogtag
     uri: pkcs11:token=Dogtag;manufacturer=SoftHSM%20project;serial=fa9c6eb4a5d5b211;model=SoftHSM%20v2

Now lets create a RSA keypair and self-sign a certificate containing the public key. There are a lot of arguments for that so I copied the help text below, to easier follow.

# certutil -S -x -d sql:nssdb_sql_softhsm -h Dogtag -f password.softhsm -n mkk -s 'CN=MKK, O=MKK Consultancy, C=SE' -k rsa -g 2048 -v 24 -Z SHA256 -t ',,'
...
"certutil: could not change trust on certificate: SEC_ERROR_TOKEN_NOT_LOGGED_IN: The operation failed because the PKCS#11 token is not logged in."

# certutil --help
...
-S              Make a certificate and add to database
   -n key-name       Specify the nickname of the cert
   -s subject        Specify the subject name (using RFC1485)
   -c issuer-name    The nickname of the issuer cert
   -t trustargs      Set the certificate trust attributes (see -A above)
   -k key-type-or-id Type of key pair to generate ("dsa", "ec", "rsa" (default))
   -h token-name     Name of token in which to generate key (default is internal)
   -g key-size       Key size in bits, RSA keys only (min 512, max 8192, default 2048)
   --pss             Create a certificate restricted to RSA-PSS (rsa only)
   -q pqgfile        Name of file containing PQG parameters (dsa only)
   -q curve-name     Elliptic curve name (ec only)
                     See the "-G" option for a full list of supported names.
   -x                Self sign
   --pss-sign        Sign the certificate with RSA-PSS (the issuer key must be rsa)
   -m serial-number  Cert serial number
   -w warp-months    Time Warp
   -v months-valid   Months valid (default is 3)
   -f pwfile         Specify the password file
   -d certdir        Cert database directory (default is ~/.netscape)
   -P dbprefix       Cert & Key database prefix
   -p phone          Specify the contact phone number ("123-456-7890")
   -Z hashAlg        
                     Specify the hash algorithm to use. Possible keywords:
                     "MD2", "MD4", "MD5", "SHA1", "SHA224",
                     "SHA256", "SHA384", "SHA512"
   -1                Create key usage extension
   -2                Create basic constraint extension
   -3                Create authority key ID extension
   -4                Create crl distribution point extension
   -5                Create netscape cert type extension
   -6                Create extended key usage extension
   -7 emailAddrs     Create an email subject alt name extension
   -8 DNS-names      Create a DNS subject alt name extension
   --extAIA          Create an Authority Information Access extension
   --extSIA          Create a Subject Information Access extension
   --extCP           Create a Certificate Policies extension
   --extPM           Create a Policy Mappings extension
   --extPC           Create a Policy Constraints extension
   --extIA           Create an Inhibit Any Policy extension
   --extSKID         Create a subject key ID extension
   See -G for available key flag options 
   --extNC           Create a name constraints extension
   --extSAN type:name[,type:name]... 
                     Create a Subject Alt Name extension with one or multiple names
                     - type: directory, dn, dns, edi, ediparty, email, ip, ipaddr,
                             other, registerid, rfc822, uri, x400, x400addr
   --extGeneric OID:critical-flag:filename[,OID:critical-flag:filename]... 
                     Add one or multiple extensions that certutil cannot encode yet,
                     by loading their encodings from external files.
                     - OID (example): 1.2.3.4
                     - critical-flag: critical or not-critical
                     - filename: full path to a file containing an encoded extension
...

To test it we list the certificates (-L) and the private keys (-K) for the NSS DB.

# certutil -L -d sql:nssdb_sql_softhsm -h all -f password.softhsm 

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

mkk                                                          u,u,u
Dogtag:mkk                                                   u,u,u

# certutil -K -d sql:nssdb_sql_softhsm -f password.internal 
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services"
certutil: no keys found

# certutil -K -d sql:nssdb_sql_softhsm -h Dogtag -f password.softhsm 
certutil: Checking token "Dogtag" in slot "SoftHSM slot ID 0x25d5b211"
< 0> rsa      1e1581006057a890e9a757314893352dc721a959   Dogtag:mkk

# ll nssdb_sql_softhsm/
total 68
-rw-------. 1 root root 28672 Sep 15 01:02 cert9.db
-rw-------. 1 root root 36864 Sep 15 00:59 key4.db
-rw-------. 1 root root   483 Sep 15 00:59 pkcs11.txt

Network Security Services, NSS, for Java (JSS)

So we have now set up our NSS DB with a SoftHSM token Dogtag. Now move onto the Java part with JSS. First install it jss and its dependency.

# yum install -y jss

# vi NSSJSSTool.java

import static java.nio.charset.StandardCharsets.UTF_8;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.Signature;
import java.security.SignatureException;
import java.util.Arrays;
import java.util.Base64;
import java.util.Enumeration;
import org.mozilla.jss.CryptoManager;
import org.mozilla.jss.crypto.CryptoStore;
import org.mozilla.jss.crypto.CryptoToken;
import org.mozilla.jss.crypto.ObjectNotFoundException;
import org.mozilla.jss.crypto.TokenException;

// NEW project page: https://github.com/dogtagpki/jss
// https://github.com/dogtagpki/jss/blob/master/org/mozilla/jss/tests/KeyStoreTest.java
public class NSSJSSTool {

    public static void main(String[] args) throws Exception {
        String nssDatabasePath = "/home/magnuskkarlsson/NetBeansProjects/example-nssdb/nssdb_sql_softhsm";

        CryptoManager.initialize(nssDatabasePath);

        CryptoManager cryptoManager = CryptoManager.getInstance();

        for (Enumeration tokens = cryptoManager.getAllTokens(); tokens.hasMoreElements();) {

            CryptoToken token = tokens.nextElement();
            System.out.println("----------------------------------");
            System.out.println(token.getName());
            System.out.println("----------------------------------");
            CryptoStore store = token.getCryptoStore();

            for (org.mozilla.jss.crypto.X509Certificate cert : store.getCertificates()) {

                System.out.println(cert.getNickname() + "\t" + cert.getSubjectDN());
                org.mozilla.jss.crypto.PrivateKey privateKey = findPrivateKey(store, cert);
                if (privateKey != null) {
                    checkKeys(privateKey, cert.getPublicKey());
                }
            }
        }
    }

    // loop through all private key to find match
    public static org.mozilla.jss.crypto.PrivateKey findPrivateKey(CryptoStore store,
            org.mozilla.jss.crypto.X509Certificate cert) throws TokenException, ObjectNotFoundException {

        for (org.mozilla.jss.crypto.PrivateKey privateKey : store.getPrivateKeys()) {

            // ObjectNotFoundException If the corresponding public key is not found.
            java.security.PublicKey publicKey = store.findPublicKey(privateKey);
            if (Arrays.equals(cert.getPublicKey().getEncoded(), publicKey.getEncoded())) {
                System.out.println(privateKey);
                return privateKey;
            }
        }
        return null;
    }

    public static void checkKeys(java.security.PrivateKey privateKey, java.security.PublicKey publicKey) throws
            InvalidKeyException, SignatureException, NoSuchAlgorithmException {

        byte[] data = "HELLO WORLD".getBytes(UTF_8);

        byte[] signatureBytes = sign(privateKey, data);

        boolean verify = verify(publicKey, data, signatureBytes);
    }

    public static byte[] sign(java.security.PrivateKey privateKey, byte[] data) throws NoSuchAlgorithmException,
            InvalidKeyException, SignatureException {

        Signature signature = Signature.getInstance("SHA256withRSA");
        signature.initSign(privateKey);
        signature.update(data);
        byte[] signatureBytes = signature.sign();
        String signatureEncoded = Base64.getEncoder().encodeToString(signatureBytes);
        System.out.println(signatureEncoded);
        return signatureBytes;
    }

    public static boolean verify(java.security.PublicKey publicKey, byte[] data, byte[] signatureBytes) throws
            NoSuchAlgorithmException, InvalidKeyException, SignatureException {

        Signature signature = Signature.getInstance("SHA256withRSA");
        signature.initVerify(publicKey);
        signature.update(data);
        boolean verify = signature.verify(signatureBytes);
        System.out.println("verify: " + verify);
        return verify;
    }
}

Now lets compile it and run it.

# javac -cp /usr/lib64/jss/jss4.jar:/usr/share/java/slf4j/slf4j-api.jar:/usr/share/java/slf4j/slf4j-simple.jar NSSJSSTool.java

# java -cp .:/usr/lib64/jss/jss4.jar:/usr/share/java/slf4j/slf4j-api.jar:/usr/share/java/slf4j/slf4j-simple.jar NSSJSSTool
[main] INFO org.mozilla.jss.CryptoManager - CryptoManager: loading JSS library
[main] INFO org.mozilla.jss.CryptoManager - CryptoManager: loaded JSS library from /usr/lib64/jss/libjss4.so
[main] INFO org.mozilla.jss.CryptoManager - CryptoManager: initializing NSS database at /root/nssdb_sql_softhsm
----------------------------------
NSS Generic Crypto Services
----------------------------------
----------------------------------
Internal Key Storage Token
----------------------------------
mkk CN=MKK,O=MKK Consultancy,C=SE
Enter password for Internal Key Storage Token

----------------------------------
Dogtag
----------------------------------
Enter password for Dogtag

Dogtag:mkk CN=MKK,O=MKK Consultancy,C=SE
org.mozilla.jss.pkcs11.PK11RSAPrivateKey@736e9adb
KaFqQUy+mv9nSrEkoIP/8rqG7D2bgXD9R48xw40eP/Sly4dkblSsVyBxIuZ1N89TLbFNoAQZVdP9T0sosPeZmiYOJOrxD6PMDRQoBzXsXLT1guFv6rBshV3FfcCEbdBOYKay8oEmxrg9Ks7/3PujwE0SwufPEtIqEReNFBjp4kQlegmBEGKYp+3lkMB5006kAoM66YlI8Rr68oBzTaf5fFXmIUFjPJJ1U+LiNWU/54qxEOsqePZxms9l1lU8uB01x5GV5r0sSXNPA2Nug3pQKhbVILJfNXXxqHmSKqGAGHgOPOMgiqlqtrNoqjXraZegXSg5RdvVi588FXS/tji9oA==
verify: true