How do you import static in Eclipse?

When you press Ctrl+Shift+O new imports are added and organized. But that will not work for static imports.

But if you place the cursor on the static import method and press Ctrl+Shift+M, then it will be imported.

Example

package se.magnuskkarlsson.example.timer;

import org.hamcrest.CoreMatchers;
import org.junit.Assert;
import org.junit.Test;

public class TimerTest {

    @Test
    public void test() throws Exception {
        Assert.assertThat("hello world", CoreMatchers.is(CoreMatchers.containsString("ell")));
    }
}

Now place cursor on each static method and press Ctrl+Shift+M and it will look like.

package se.magnuskkarlsson.example.timer;

import static org.hamcrest.CoreMatchers.containsString;
import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

import org.junit.Test;

public class TimerTest {

    @Test
    public void test() throws Exception {
        assertThat("hello world", is(containsString("ell")));
    }
}

UTF-8 Charset as Constant in Java 7

Since Java 7 UTF-8 as a constant has been introduced with java.nio.charset.StandardCharsets

Example

new LineNumberReader(new InputStreamReader(new FileInputStream("foo.ini"), StandardCharsets.UTF_8));

Composite Primary Key with Generated and Unique Value with JPA 2.0

Introduction

Sometime you need several column for you primary keys. The background for that can very. There are two solution for that in JPA:

• @IdClass as inline columns
• @EmbeddedId with extract column in separate class

Databas Design

  _____________
 |             |
 |   Student   |
 |_____________|
 |             |
 | *studentId  |
 | *groupId    |
 |  firstName  |
 |             |
 |_____________|

@IdClass

@Entity
@IdClass(StudentPK.class)
public class Student implements Serializable {

    private static final long serialVersionUID = 1L;

    @Id
    private Long studentId;

    @Id
    private Long groupId;

    // the rest is left out for brevity
}

public class StudentPK implements Serializable {

    private static final long serialVersionUID = 1L;

    private Long studentId;

    private Long groupId;

    @Override
    public int hashCode() {
        return (int) (studentId + groupId);
    }

    @Override
    public boolean equals(Object object) {
        if (object instanceof StudentPK) {
            StudentPK otherId = (StudentPK) object;
            return (otherId.studentId == this.studentId)
                    && (otherId.groupId == this.groupId);
        }
        return false;
    }

    public Long getStudentId() {
        return studentId;
    }

    public void setStudentId(Long studentId) {
        this.studentId = studentId;
    }

    public Long getGroupId() {
        return groupId;
    }

    public void setGroupId(Long groupId) {
        this.groupId = groupId;
    }
}

@EmbeddedId

@Entity
public class Student implements Serializable {

    private static final long serialVersionUID = 1L;

    @EmbeddedId
    private StudentPK studentPK;

    // the rest is left out for brevity
}

The StudentPK class remains the same.

Conclusion

The two above option is very much the same and I cannot see any clear advantage for one solution over the other, so I would recommend use the solution that resembles the real world, i.e. SQL and use IdClass.

Automatic Generated Values

When having composite primary key you cannot use @GeneratedValue, but you can use @TableGenerator. Remember there are three strategies for automatic primary key generation values and the only database vendor neutral solution is TABLE.

• @GeneratedValue(strategy = IDENTITY), with automatic generated value on column. Works only for MySQL and MS SQL.
• @GeneratedValue(strategy = SEQUENCE), separate sequence. Works only for PostgreSQL and Oracle.
• @GeneratedValue(strategy = TABLE), the ONLY database vendor neutral solution. ORM specific implementation, does not rely on any database specific technique. Uses separate table to persist increment value and handles increment by it self.

@Entity
@IdClass(StudentPK.class)
public class Student implements Serializable {

    private static final long serialVersionUID = 1L;

    @Id
    @TableGenerator(name = "TABLE_STUDENT_GEN", table = "SEQ_TABLE", pkColumnName = "SEQ_NAME", valueColumnName = "SEQ_COUNT", pkColumnValue = "STUDENT_SEQ")
    @GeneratedValue(strategy = GenerationType.TABLE, generator = "TABLE_STUDENT_GEN")
    @Column(unique = true)
    private Long studentId;

    @Id
    @TableGenerator(name = "TABLE_GROUP_GEN", table = "SEQ_TABLE", pkColumnName = "SEQ_NAME", valueColumnName = "SEQ_COUNT", pkColumnValue = "GROUP_SEQ")
    @GeneratedValue(strategy = GenerationType.TABLE, generator = "TABLE_GROUP_GEN")
    private Long groupId;

    // the rest is left out for brevity
}

The StudentPK class remains the same.

Which will generate the following database.

    _____________        _____________
   |             |      |             |
   |   Student   |      |   SEQ_TABLE |
   |_____________|      |_____________|
   |             |      |             |
   | *studentId  |      |  SEQ_NAME   |
   | *groupId    |      |  SEQ_COUNT  |
   |  firstName  |      |             |
   |_____________|      |_____________|

NOTE You do not need to have automatic generated values for both columns.

NOTE You do not need to have unique constraint for id column, I just added it to show that you can, because sometimes you want to have it.

Mapping OneToMany with JPA 2.0

Introduction

Mapping one-to-many relationship with JPA 2.0, can be done in 2 ways:

• @OneToMany, with optional @JoinColumn
• @OneToMany with @JoinTable

Here I will look closer on option one, which is most common. @JoinTable is more common with @ManyToMany relationship/mapping.

Databas Design

    _____________        _____________
   |             |      |             |
   |   Employee  |      |   Phone     |
   |_____________|      |_____________|
   |             |      |             |
   | *employeeId | -->  | *phoneId    |
   |  firstName  |      | *employeeId |
   |_____________|      | areaCode    |
                        |             |
                        |_____________|

Bidrectional OneToMany Mapping

package se.magnuskkarlsson.example.jpa;

import java.util.ArrayList;
import java.util.List;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.JoinColumn;
import javax.persistence.OneToMany;

@Entity
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long employeeId;

    @OneToMany(mappedBy = "owner")
    private List<Phone> phones;

    @Column
    private String firstName;

    // ----------------------- Logic Methods -----------------------

    public void addPhone(Phone phone) {
        this.getPhones().add(phone);
        if (phone.getOwner() != this) {
            phone.setOwner(this);
        }
    }

    // ----------------------- Helper Methods -----------------------

    // ----------------------- Get and Set Methods -----------------------

    public Long getEmployeeId() {
        return employeeId;
    }

    public void setEmployeeId(Long employeeId) {
        this.employeeId = employeeId;
    }

    public List<Phone> getPhones() {
        if (phones == null) {
            phones = new ArrayList<Phone>();
        }
        return phones;
    }

    public void setPhones(List<Phone> phones) {
        this.phones = phones;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }
}

package se.magnuskkarlsson.example.jpa;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;

@Entity
public class Phone {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long phoneId;

    @ManyToOne(fetch = FetchType.LAZY)
    @JoinColumn(name = "OWNER_ID")
    private Employee owner;

    @Column
    private int areaCode;

    // ----------------------- Logic Methods -----------------------

    // ----------------------- Helper Methods -----------------------

    // ----------------------- Get and Set Methods -----------------------

    public Long getPhoneId() {
        return phoneId;
    }

    public void setPhoneId(Long phoneId) {
        this.phoneId = phoneId;
    }

    public Employee getOwner() {
        return owner;
    }

    public void setOwner(Employee employee) {
        this.owner = employee;
        if (!employee.getPhones().contains(this)) {
            // warning this may cause performance issues if you have a large
            // data set since this operation is O(n)
            employee.getPhones().add(this);
        }
    }

    public int getAreaCode() {
        return areaCode;
    }

    public void setAreaCode(int areaCode) {
        this.areaCode = areaCode;
    }
}

Now lets write some test code to test this.

    private EntityManagerFactory emf;

    private EntityManager em;

    @Before
    public void setUp() throws Exception {
        emf = Persistence.createEntityManagerFactory("it");
        em = emf.createEntityManager();
    }

    @After
    public void tearDown() throws Exception {
        em.close();
        emf.close();
    }

    @Test
    public void testRelationship() throws Exception {
        em.getTransaction().begin();

        Phone phone1 = new Phone();
        em.persist(phone1);
       
        Phone phone2 = new Phone();
        em.persist(phone2);
       
        Employee emp = new Employee();
        emp.addPhone(phone1);
        emp.addPhone(phone2);
        em.persist(emp);

        em.getTransaction().commit();

        em.getTransaction().begin();

        // SELECT DISTINCT
        // http://stackoverflow.com/questions/8199512/jpql-inner-join-without-duplicate-records
        // JOIN FETCH
        // https://en.wikibooks.org/wiki/Java_Persistence/Relationships#Join_Fetching
        String sql = "SELECT DISTINCT e FROM Employee e JOIN FETCH e.phones";
        TypedQuery<Employee> query = em.createQuery(sql, Employee.class);
        List<Employee> list = query.getResultList();
        System.out.println("QUERY RESULT");
        for (Employee e : list) {
            System.out.println(e);
            for (Phone p : e.getPhones()) {
                System.out.println("   " + p);
            }
        }

        em.getTransaction().commit();
    }

And the debug output from Hibernate

Hibernate: 
    insert 
    into
        Phone
        (OWNER_ID) 
    values
        (?)
Hibernate: 
    insert 
    into
        Phone
        (OWNER_ID) 
    values
        (?)
Hibernate: 
    insert 
    into
        Employee
        
    values
        ( )
Hibernate: 
    update
        Phone 
    set
        OWNER_ID=? 
    where
        id=?
Hibernate: 
    update
        Phone 
    set
        OWNER_ID=? 
    where
        id=?
Hibernate: 
    select
        distinct employee0_.EMP_ID as EMP_ID1_1_0_,
        phones1_.id as id1_2_1_,
        phones1_.OWNER_ID as OWNER_ID2_2_1_,
        phones1_.OWNER_ID as OWNER_ID2_1_0__,
        phones1_.id as id1_2_0__ 
    from
        Employee employee0_ 
    inner join
        Phone phones1_ 
            on employee0_.EMP_ID=phones1_.OWNER_ID
QUERY RESULT
se.magnuskkarlsson.example.jpa.Employee@232864a3
   se.magnuskkarlsson.example.jpa.Phone@84c5310
   se.magnuskkarlsson.example.jpa.Phone@32dcce09

As we can see the above create operation, generates 5 SQL statements (3 INSERTS and 2 UPDATES).

The same behavior with multiple INSERT followed by UPDATE, happens when you want to create a new Phone. Then you need to load the Employee class and then call addPhone(Phone). This is not optimal as you can see. Now lets consider another approach with unidirectional mapping.

First Attempt Unidirectional OneToMany Mapping

First we update our mapping annotations./p>

@Entity
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long employeeId;

    // @OneToMany(mappedBy = "owner")
    @OneToMany
    @JoinColumn(name = "employeeId", referencedColumnName = "employeeId")
    private List<Phone> phones;

    @Column
    private String firstName;

    public void addPhone(Phone phone) {
        this.getPhones().add(phone);
        phone.setEmployeeId(getEmployeeId());
    }

    // the rest is left out for brevity
}

@Entity
public class Phone {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long phoneId;

    @Column(nullable = false)
    private Long employeeId;

    @Column
    private int areaCode;

    // the rest is left out for brevity

Hibernate: 
    insert 
    into
        Phone
        (OWNER_ID) 
    values
        (?)
Hibernate: 
    insert 
    into
        Phone
        (OWNER_ID) 
    values
        (?)
Hibernate: 
    insert 
    into
        Employee
        
    values
        ( )
Hibernate: 
    update
        Phone 
    set
        OWNER_ID=? 
    where
        id=?
Hibernate: 
    update
        Phone 
    set
        OWNER_ID=? 
    where
        id=?
Hibernate: 
    select
        distinct employee0_.EMP_ID as EMP_ID1_1_0_,
        phones1_.id as id1_2_1_,
        phones1_.OWNER_ID as OWNER_ID2_2_1_,
        phones1_.OWNER_ID as OWNER_ID2_1_0__,
        phones1_.id as id1_2_0__ 
    from
        Employee employee0_ 
    inner join
        Phone phones1_ 
            on employee0_.EMP_ID=phones1_.OWNER_ID
QUERY RESULT
se.magnuskkarlsson.example.jpa.Employee@46764885
   se.magnuskkarlsson.example.jpa.Phone@6ecc295c
   se.magnuskkarlsson.example.jpa.Phone@7c2a88f4

As we can see, we still need 5 SQL operations. But we can do better. Think how the underlying SQL is working. There is nothing magic when JPA or ORM. Lets first create parent Employee and then create children Phone and explicit set foreign key.

Second Attempt Unidirectional OneToMany Mapping

Mapping is the same, but "business logic" is changed.

        Employee emp = new Employee();
        emp.setFirstName("Bob");
        em.persist(emp);

        Phone phone1 = new Phone();
        phone1.setEmployeeId(emp.getEmployeeId());
        phone1.setAreaCode(613);
        em.persist(phone1);

        Phone phone2 = new Phone();
        phone2.setEmployeeId(emp.getEmployeeId());
        phone2.setAreaCode(416);
        em.persist(phone2);

        // Employee is cached in first second cache, since reusing Session
        em.detach(emp);

Hibernate: 
    insert 
    into
        Employee
        
    values
        ( )
Hibernate: 
    insert 
    into
        Phone
        (OWNER_ID) 
    values
        (?)
Hibernate: 
    insert 
    into
        Phone
        (OWNER_ID) 
    values
        (?)
Hibernate: 
    select
        distinct employee0_.EMP_ID as EMP_ID1_1_0_,
        phones1_.id as id1_2_1_,
        phones1_.OWNER_ID as OWNER_ID2_2_1_,
        phones1_.OWNER_ID as OWNER_ID2_1_0__,
        phones1_.id as id1_2_0__ 
    from
        Employee employee0_ 
    inner join
        Phone phones1_ 
            on employee0_.EMP_ID=phones1_.OWNER_ID
QUERY RESULT
se.magnuskkarlsson.example.jpa.Employee@a7072fd
   se.magnuskkarlsson.example.jpa.Phone@232864a3
   se.magnuskkarlsson.example.jpa.Phone@72f1b266

As we can see the number of SQL statement is now reduced to the natural number that is sensible needed. And when we need to create a new child Phone we do not need to load and add it to parent Employee.

Is there any downside with this solution. Not really if we would like bidirectional behavior we could add a @Transient parameter and set it lazy, just as we would have done with if the mapping was bidirectional. One would maybe think of keeping bidirectional mapping eager, but that is not a good pattern/practice, since you could risk to load huge object graphs into memory, which would impact the performance negative.

@Entity
public class Phone {

    @Transient
    private Employee employee;

    // the rest is left out for brevity
}

The Rest of the Files

To make this complete here is the rest of the code.

src/test/resources/META-INF

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
    version="2.0">

    <persistence-unit name="it" transaction-type="RESOURCE_LOCAL">
        <provider>org.hibernate.ejb.HibernatePersistence</provider>
        <class>se.magnuskkarlsson.example.jpa.Company</class>
        <class>se.magnuskkarlsson.example.jpa.Employee</class>
        <class>se.magnuskkarlsson.example.jpa.Phone</class>
        <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
        <validation-mode>CALLBACK</validation-mode>
        <properties>
            <property name="hibernate.hbm2ddl.auto" value="create-drop" />
<!-- 
            <property name="hibernate.dialect" value="org.hibernate.dialect.H2Dialect" />
            <property name="hibernate.connection.driver_class" value="org.h2.Driver" />
            <property name="hibernate.connection.url" value="jdbc:h2:mem:" />
 -->
            <property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect" />
            <property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver" />
            <property name="javax.persistence.jdbc.url" value="jdbc:mysql://localhost:3306/EXAMPLE" />
            <property name="javax.persistence.jdbc.user" value="root" />
            <property name="javax.persistence.jdbc.password" value="root" />

            <property name="hibernate.show_sql" value="true" />
            <property name="hibernate.format_sql" value="true" />
            <property name="hibernate.generate_statistics" value="false" />
        </properties>
    </persistence-unit>
</persistence>

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>se.magnuskkarlsson.examples</groupId>
    <artifactId>example-jpa</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>war</packaging>
    <name>Example JPA</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
        <hibernate-version>4.2.14.Final</hibernate-version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-api</artifactId>
            <version>7.0</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
            <scope>provided</scope>
        </dependency>

        <!-- Test Support -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.185</version>
            <scope>test</scope>
        </dependency>

        <!-- <dependency> <groupId>org.apache.derby</groupId> <artifactId>derbyclient</artifactId> 
            <version>10.7.1.1</version> <scope>test</scope> </dependency> -->

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.34</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-entitymanager</artifactId>
            <version>${hibernate-version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-infinispan</artifactId>
            <version>${hibernate-version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-validator</artifactId>
            <version>4.3.1.Final</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                    <debug>true</debug>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-ejb-plugin</artifactId>
                <version>2.5</version>
                <configuration>
                    <ejbVersion>3.1</ejbVersion>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.5</version>
            </plugin>
        </plugins>
    </build>
</project>

Simple Login Module in JBoss EAP 6 for Testing

Introduction

When testing locally or in integration tests, it is convenient to use a simple login module. Source code.

Configuration

<subsystem xmlns="urn:jboss:domain:security:1.2">
    <security-domains>
        ...
        <security-domain name="simple-policy" cache-type="default">
            <authentication>
                <login-module code="org.jboss.security.auth.spi.SimpleServerLoginModule" flag="required"/>
            </authentication>
        </security-domain>
    </security-domains>
</subsystem>

Test Web App

WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
    version="3.0">
 
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Secure Content</web-resource-name>
            <url-pattern>/*</url-pattern>
        </web-resource-collection>

        <auth-constraint>
            <role-name>user</role-name>
        </auth-constraint>
    </security-constraint>

    <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>simple-policy</realm-name>
    </login-config>

    <security-role>
        <role-name>user</role-name>
    </security-role>
</web-app>

WEB-INF/jboss-web.xml

<?xml version="1.0" encoding="UTF-8"?>
<jboss-web>
    <context-root>/example-webapp</context-root>
    <security-domain>java:/jaas/simple-policy</security-domain>
</jboss-web>

How to Set Default JVM on Ubuntu

If your system has more than one version of Java, configure which one your system uses by entering the following command in a terminal window

$ sudo update-alternatives --config java

Fast Scala Compilation with Maven and Zinc

If you need faster scala compilation, see http://hohonuuli.blogspot.se/2012/11/fast-scala-compilation-with-maven.html.

How to Log SQL Statements, Parameters and Cache in Hibernate?

Introduction

Current implementation of Hibernate are:

• Hibernate 4.2 implements JPA 2.0 (EE 6)
• Hibernate 4.3+ implements JPA 2.1 (EE 7)

Logging

Do not set logging level in persistence.xml (hibernate.show_sql, hibernate.format_sql), instead use Log4J for changing logging level.

# Log all hibernate SQL DML statements as they are executed
log4j.logger.org.hibernate.SQL=TRACE
# Log all hibernate JDBC parameters
log4j.logger.org.hibernate.type=TRACE
# Log all hibernate second-level cache activity
log4j.logger.org.hibernate.cache=TRACE

You can put this in you src/test/resources/log4j.properties if you want more logging when writing or debugging your test cases.

Reference

• Hiberante 4.2 Logging

What is Docker and Why is it Hot?

What is Docker?

It is the possible replacement for Virtualization.

What is the Difference between Virtualization and Docker?

Below is a good picture from https://www.docker.com/whatisdocker.

So the difference between Virtualization and Docker is that everything runs locally on the same server, without the Virtualization overhead.

Example. Lets download a ready image from the Docker Hub (https://hub.docker.com/). You can either browse the Docker Hub via web browser or via command line.

$ sudo docker search tomcat
...
NAME                                        DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
tomcat                                      Apache Tomcat is an open source implementa...   203       [OK]       
consol/tomcat-7.0                           Tomcat 7.0.57, 8080, "admin/admin"              12                   [OK]
consol/tomcat-8.0                           Tomcat 8.0.15, 8080, "admin/admin"              10                   [OK]
...

Lets pick the official docker image. The first time you run a new Docker Image, it will download it and that will take a little while.

$ sudo docker run -it --rm -p 8888:8080 tomcat:8.0
...
10-Aug-2015 18:52:33.409 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
10-Aug-2015 18:52:33.413 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
10-Aug-2015 18:52:33.413 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 658 ms

And now try it by open http://localhost:8888.

So what happened? We started a new process.

$ ps -aux | grep -i docker
...
root     29569  0.0  0.0 222512 10728 pts/25   Sl+  21:19   0:00 docker run -it --rm -p 8888:8080 tomcat:8.0
...

And a new tomcat process.

$ ps -aux | grep -i java
...
root     29578  1.7  1.2 7172840 200552 pts/1  Ssl+ 21:19   0:04 /usr/bin/java -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/local/tomcat/endorsed -classpath /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/tomcat -Dcatalina.home=/usr/local/tomcat -Djava.io.tmpdir=/usr/local/tomcat/temp org.apache.catalina.startup.Bootstrap start
...

And for network. A new virtual bridge was created, which is basically a virtual router.

$ ifconfig
...
docker0   Link encap:Ethernet  HWaddr 56:84:7a:fe:97:99
...

Why is Docker Better?

• Lower licensing costs. You only need ONE license for the virtualized server for ALL its virtual guests.
• More RAM and CPU. With Docker all process runs on the server directly. No virtualization layer that steals resources.
• Less disks. A virtual guest is at least 20 GB and a Docker image is around 300 MB, since no duplication of OS and all its libraries are needed.

How Does Docker Work?

Docker makes use of proven mature Linux technologies:

• CPU and RAM - cgroups. With cgroups you can restrict how much a process can use of the CPU and RAM, much like the same way CPU and RAM is handled in virtualization.
• Disks - namespaces. With namespaces you can isolate local file system and divide it much like a network filesystem.
• Network - virtual ethernet bridge.

Common Docker Commands

# Lists all docker process
$ sudo docker ps -a

# Print detailed information of a Docker proccess
$ sudo docker inspect <ContainerID>

# List all local Docker images
$ sudo docker images

# Run Docker image
$ sudo docker run -p <host_port1>:<container_port1> -p <host_port2>:<container_port2>

# Search for a ready Docker image at Docker Hub (https://registry.hub.docker.com/)
$ sudo docker search <SearchString>

Installation on Ubuntu 14.04

Install docker.

$ sudo apt-get install docker.io

Test the installation.

$ sudo docker run hello-world
Hello from Docker.
This message shows that your installation appears to be working correctly.
...

Reference

• Docker Documentations https://registry.hub.docker.com/
• Shared ready to use Docker Images https://hub.docker.com/
• Docker Network https://docs.docker.com/articles/networking/
• cgroups https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt
• namespaces http://man7.org/linux/man-pages/man7/namespaces.7.html
• Docker at github https://github.com/docker/docker

Getting Started with Mockito

Introduction

To create effective unit test, they should be fast. But also you might need to mock object out to be able to write unit tests. The most common library to use for mocking is mockito.

Example: Mock expected results

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testIterator() throws Exception {
        Iterator mock = mock(Iterator.class);
        when(mock.next()).thenReturn("Hello").thenReturn("World");
        assertThat("Hello World", is(equalTo(mock.next() + " " + mock.next())));
    }

Example: Mock expected result with input

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testArrayList() throws Exception {
        ArrayList mock = mock(ArrayList.class);
        when(mock.get(1)).thenReturn("Foo");
        assertThat("Foo", is(equalTo(mock.get(1))));
    }

Example: Mock expected results with any input

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testArrayListAny() throws Exception {
        ArrayList mock = mock(ArrayList.class);
        when(mock.get(anyInt())).thenReturn("Foo");
        assertThat("Foo", is(equalTo(mock.get(1))));
    }

Example: Mock and expect exception

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test(expected = IOException.class)
    public void testOutputStream() throws Exception {
        OutputStream mock = mock(OutputStream.class);
        doThrow(new IOException()).when(mock).close();
        mock.close();
    }

Example: Mock and verify that metods were invoked

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testOutputStreamVerify() throws Exception {
        OutputStream mock = mock(OutputStream.class);
        BufferedOutputStream fos = new BufferedOutputStream(mock);
        fos.close();
        verify(mock).close();
    }

Using Hamcrest with JUnit 4

Introduction

The latest JUnit 4 comes with a transitive dependency of hamcrest. What hamcrest brings to the table is to increase the readability of the test code. This is important because codes are often more read than written.

Example

@Test
public void testEqualTo() throws Exception {
    assertThat(Foo.hello(), is(equalTo("Hello World")));
}

Core

The starting point is org.junit.Assert.assertThat(T actual, Matcher matcher) and the matcher methods in org.hamcrest.CoreMatchers.*:

• allOf()
• any()
• anyOf()
• anything()
• describedAs()
• equalTo()
• instanceOf()
• is()
• not()
• notNullValue()
• nullValue()
• sameInstance()

allOf()

allOf() is a simple assertion that just says all of the matcher arguments must be true.

@Test
public void testAllOf() throws Exception {
    assertThat(Foo.hello(), is(allOf(notNullValue(), instanceOf(String.class), equalTo("Hello World"))));
}

any()

This matcher just checks that the actual result is a class of a certain type.

@Test
public void testAny() throws Exception {
    assertThat(Foo.hello(), is(any(String.class)));
}

anyOf()

If any of the matchers are true this assertion will pass.

@Test
public void testAnyOf() throws Exception {
    assertThat(Foo.hello(), is(anyOf(nullValue(), instanceOf(String.class), equalTo("Goodbye"))));
}

anything()

This matcher will always evaluates to true.

describedAs()

This allows you to override the default description for a matcher.

equalTo()

See above.

instanceOf()

Checks that you have an instance of a particular type.

is()

See above.

not()

Just reverses the outcome of a matcher.

@Test
public void testNot() throws Exception {
    assertThat(Foo.hello(), is(not("Bar")));
}

notNullValue()

Simply does a null check

nullValue()

Assert a null value.

sameInstance()

Assert two Objects are ==, i.e. NOT equals()

@Test
public void testSameInstance() throws Exception {
    assertThat(2 + 2, is(sameInstance(4)));
    assertThat(new Integer(2 + 2), is(not(sameInstance(new Integer(4)))));
}

JUnit testing CDI and EJB with Mockito, OpenEJB and Arquillian

Introduction

Unit testing EE have previously been hard, but since EE 6 and onward the EE specification has been improved on the test front.

Since EE 6 the following has been added:

• javax.ejb.embeddable.EJBContainer – A standardized test container for EJB.
• Portable JNDISyntax – Now all EE container must conform to the same JNDI registration and lookup name spaces. Before EE 6, there was no name standard and each EE container had each different name standard.

Do we need CDI and EJB injection for unit testing?

The general rule of thumb is to make unit tests fast, because if they are not, people will starts to disable test cases. And this will make unit tests contra productive.

Loading a CDI or EJB container takes time. Which is also true for Spring Application Context. So try to write your unit test without the need to load some kind of container.

JPA and JDBC can be tested with standard Java SE

See http://magnus-k-karlsson.blogspot.se/2015/02/hibernate-best-practise-jpa-20-and.html.

Logic and EJB can be tested with Mockito

When testing logic or EJB there is in most cases no need to create a EJB context. Try to use mockito, to mock your surroundings.

EJB Container Unit Testing

In some cases there is actually a need to create an EJB context. The two most common solutions are:

• OpenEJB
• Arquillian

OpenEJB

Pros:

• Is faster to start and stop than Arquillian
• More easier to learn
• Easy to use maven dependency

Cons:

• Only support standard EE (which is also a pro, since then you know your code will run in any EE container.).
• Does not support mere CDI, must use first an EJB to test pure CDI functionality. Can put EJB in test directory.

See

• https://tomee.apache.org/examples-trunk/

Arquillian

Pros:

• Can test EE container specific code.
• Support test of pure CDI code.

Cons:

• Is slower than OpenEJB.
• Its maven dependencies are long and complex.
• Is harder to learn compared with OpenEJB.

See:

• Maven dependency - https://github.com/arquillian/arquillian-examples/blob/master/arquillian-tutorial/pom.xml
• Arquillian Getting Started Guide - http://arquillian.org/guides/getting_started/

Why EJB?

Introduction

When people think of Java EE, they think in first hand of EJB, even though the EE specification contains much than EJB. For example Rod Johnson (founder of Spring) pointed out in the Expert One-on-One J2EE Development without EJB that JDBC, JTA and JMS are successful parts of EE.

But a lot has happen when this book was written and recent EE version starting with 5, but especially 6 and 7 had made EE development even easier than Spring development.

So if you are thinking of choosing EE, you might wonder what EJB contributes with:

1. Pooling
2. Transaction
3. Security
4. Monitoring and Management
5. Asynchronous

Pooling

This makes accessing a EJB faster. Typically usage with Spring is to lazy inject things when a Business Object is requested.

Pooling is also a DOS protection since you cannot request endless number of Business Object. When the pool is empty of non busy EJB, the request will fail fast. This handling is important, since accepting new request when the server is overloaded will only worsen the situation.

Further sizing and resizing the pool and other pool is typically something a application administrator want to do after the deployment in production and this is done from unified interface in the EE container. No need to go in each application and know its inner configuration.

Transaction

Every public method will default start a new transaction if no one exists.

Security

The EE specification contains a good security API and is easy to use and covers all the EE components.

Since the EE 6 the EAR archive is history. Now EJB can be bundle in standard WAR archive.

And standard web.xml security is quite straight forward.

Monitoring and Management

With the emerge of DevOps (Development Operation) the monitoring and management has grown in importance (even though it always has been important).

All modern EE container has today this capability of monitoring and manage EJB as a standard.

Asynchronous

javax.ejb.Asynchronous was introduced in EE 6 and is easy to use. Example to make a method asynchronous:

@Stateless
public class OrderProcessorBean {

    // synchronous method
    public boolean validate(Order order) throws PaymentException {
        ...
    }

    @Asynchronous
    public Future<string> processPayment(Order order) throws PaymentException {
        ...
    }
}

Use YubiKey for 2 Factor Authentication on Ubuntu 14.04

To increase the security for your Google Account you can use 2 factor authentication with SMS. But also you can use YubiKey. To get that to work on Ubuntu 14.04 follow the instruction here http://ihaveagadget.blogspot.se/2014/11/yubico-u2f-key-not-working-in-ubuntu.html.

How to Create a Bootable USB for Linux ISO

Introduction

The most generic way to create a bootable USB with a Linux ISO file, is to use the dd command.

Prerequisite

On RHEL/CentOS/Fedora

# yum install syslinux

On Ubuntu/Debian

$ sudo apt-get install syslinux

Steps

1. Download ISO file

2. Convert your normal ISO file to a hybrid ISO.

# isohybrid /home/magnus/Downloads/ubuntu-12.04.4-desktop-amd64.iso

3. Create a bootable USB.

$ dd if=/home/magnus/Downloads/ubuntu-12.04.4-desktop-amd64.iso of=/dev/sdf
1501184+0 records in
1501184+0 records out
768606208 bytes (769 MB) copied, 286.08 s, 2.7 MB/s

NOTE: the device name is without number, e.g. /dev/sdf1.

If you are unsure what the device name is you can e.g. use lsblk.

$ lsblk
NAME                        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0                          11:0    1  1024M  0 rom  
sda                           8:0    0 167.7G  0 disk 
├─sda1                        8:1    0   500M  0 part /boot
└─sda2                        8:2    0 146.5G  0 part 
  ├─vg_rhel6-lv_root (dm-0) 253:0    0   127G  0 lvm  /
  └─vg_rhel6-lv_swap (dm-1) 253:1    0  19.5G  0 lvm  [SWAP]
sdf                           8:80   0   3.7G  0 disk 

Good intro to AngularJS

Good intro to AngularJS, is built up with examples and short theory introduction.



https://github.com/curran/screencasts/tree/gh-pages/introToAngular

Crontab Expression

I do not write contab scheduling expression every day, so I tend to forget the expression syntax. Below is a good site for generating such expressions.

http://www.crontab-generator.org/

Puppet IDE Geppetto

When writing puppet modules I would recommend using an IDE when writing puppet code. The IDE from puppetlab is the most mature - Geppetto. Geppetto is fine, but one warning when installing it.

DO NOT install Geppetto as a plugin in an existing Eclipse. Instead I recommend downloading the bundle version http://puppetlabs.github.io/geppetto/download.html.

How to Install Tomcat 6 on RHEL 6

A good and accurate guide exists on http://oracle-base.com/articles/linux/apache-tomcat-installation-on-linux.php

Maven RPM Plugins

There are two major maven rpm plugins.

1. rpm-maven-plugin The Codehaus mojo plugin is good and holds good quality as almost all mojo plugins, but has one big disadvantage. It requires rpmbuild linux tool, which means it will not run without pain on Windows. You could of cource switch to a better os, but that is often easier said than done in bigger organizations.
2. Redline RPM The second tool is build on Redline, which is a pure java library, which means it will run on windows. The backdraw with this one, is that it exists no maven plugin for it, only ant is supported out of the box.

Hibernate Best Practise, JPA 2.0 and Second Level Cache with Infinispan

Introduction

The technique of using ORM has been becoming the defacto standard, when developing new application, but after passed the level of school books example, the usage of using ORM is not so simple. In this blog I will talk about minimizing code for simple CRUD actions, but also arguing about getting back to basic for complex side of ORM.

Before started, one must recognised that the number one implementation of ORM is hibernate and that is the implementation framework used here for the specification JPA 2.0 contained in EE 6.

ORM Best Practise

DRY AbstractDomain

All domain object contain some common characteristic, e.g. primary key, toString method, maybe created and last modified date time, etc. Put all those things in a abstract domain class

package se.magnuskkarlsson.example.jpa;

import java.lang.reflect.Field;

@javax.persistence.Cacheable(true)
@javax.persistence.MappedSuperclass
public abstract class AbstractDomain implements java.io.Serializable {

    private static final long serialVersionUID = 1L;

    @javax.persistence.Id
    @javax.persistence.GeneratedValue(strategy = javax.persistence.GenerationType.AUTO)
    private Long id;

    // ----------------------- Logic Methods -----------------------

    @Override
    public String toString() {
        StringBuilder sb = new StringBuilder();
        sb.append(this.getClass().getSimpleName());
        sb.append("@").append(hashCode());
        sb.append("[");
        toFieldString(sb, this.getClass().getSuperclass().getDeclaredFields());
        toFieldString(sb, this.getClass().getDeclaredFields());
        sb.append("]");
        return sb.toString();
    }

    // ----------------------- Helper Methods -----------------------

    protected void toFieldString(StringBuilder sb, Field[] fields) {
        for (Field field : fields) {
            if (field.isAnnotationPresent(javax.persistence.Id.class)
                    || field.isAnnotationPresent(javax.persistence.Column.class)) {

                try {
                    String name = field.getName();
                    field.setAccessible(true);
                    Object value = field.get(this);
                    sb.append(name).append("='").append(value).append("', ");
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }
    }

    // ----------------------- Get and Set Methods -----------------------

    public Long getId() {
        return id;
    }
}

Note about abstract domain class.

• Every domain class will be serializable.
• We add a toString method, which is realized with reflection. Reflection is not the fastest technique, which is OK for a toString method, since it should not be called often.
• We add javax.persistence.Cacheable(true), for preparing for entity second level cache.

Next is to create a concrete domain class.

package se.magnuskkarlsson.example.jpa;

import java.util.ArrayList;
import java.util.List;

@javax.persistence.Cacheable(true)
@javax.persistence.Entity
@javax.persistence.AttributeOverride(name = "id", column = @javax.persistence.Column(name = "companyId"))
public class Company extends AbstractDomain {

    private static final long serialVersionUID = 1L;

    @javax.persistence.Column(unique = false, nullable = false, length = 255)
    private String name;

    @javax.persistence.Transient
    private List<Employee> employees;

    // ----------------------- Logic Methods -----------------------

    // ----------------------- Helper Methods -----------------------

    // ----------------------- Get and Set Methods -----------------------

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public List<Employee> getEmployees() {
        if (employees == null) {
            employees = new ArrayList<Employee>();
        }
        return employees;
    }

    public void setEmployees(List<Employee> employees) {
        this.employees = employees;
    }
}

Note about concrete domain class.

• We make use of javax.persistence.AttributeOverride to specialize a primary key for concrete domain class.
• WE DO NOT MAP RELATIONSHIP WITH ORM. Se below.

KISS. Back to basic for relationship.

Not using ORM for our relationship, might sound crazy at first, but the fact is that managing relationship with ORM is hard and comes with several not obvious side effects.

So what went wrong? Lets start from the beginning. First we mapped all our relationship with ORM. Next we was challenged with how do we load all relationship. You might then started with loading all relationship eagerly. But after testing your implementation with more production like data volume, you realize that you are loading big chunk of your database in memory. This does not hold. OK, you comes to your senses and add lazy loading to all your relationship.

So what happened next? You started to optimize all your queries so you did not need to load each sub children separately. You probably now came across exception like org.hibernate.loader.MultipleBagFetchException: cannot simultaneously fetch multiple bags. And problem like loading loading N number of extra data. Or why do you need to load all children for inserting a new child to a parent?

When reaching this point you start to doubt the entire usage and justification of ORM. And that in fact is where I somewhere also landed. So lets get rid of all complexity it means of handling relationship with ORM and get back to basic - KISS.

DRY DomainDAO

We extract common CRUD task to a DAO base class.

package se.magnuskkarlsson.example.jpa;

import java.util.List;

import javax.persistence.TypedQuery;

public class DomainDAO<D extends AbstractDomain> {

    @javax.persistence.PersistenceContext
    protected javax.persistence.EntityManager em;

    // ----------------------- Logic Methods -----------------------

    public D create(D domain) {
        em.persist(domain);
        return domain;
    }

    public D get(Class<D> clazz, long id) {
        return em.find(clazz, id);
    }

    public List<D> getAll(Class<D> clazz) {
        String sql = "FROM " + clazz.getSimpleName();
        TypedQuery<D> query = em.createQuery(sql, clazz);
        // JPA 2.0 Cache Query hints
        // Use the object/data if already in the cache
        query.setHint("javax.persistence.cache.retrieveMode",
                javax.persistence.CacheRetrieveMode.USE);
        // Cache the objects/data returned from the query.
        query.setHint("javax.persistence.cache.storeMode",
                javax.persistence.CacheStoreMode.USE);
        return query.getResultList();
    }

    public D update(D domain) {
        return em.merge(domain);
    }

    public D delete(Class<D> clazz, long id) {
        D domain = em.find(clazz, id);
        if (domain != null) {
            em.remove(domain);
        }
        return domain;
    }

    public boolean contains(Class<D> clazz, long id) {
        return em.getEntityManagerFactory().getCache().contains(clazz, id);
    }

    public void evict(Class<D> clazz, long id) {
        em.getEntityManagerFactory().getCache().evict(clazz, id);
    }

    // ----------------------- Helper Methods -----------------------

    // ----------------------- Get and Set Methods -----------------------

}

Deployment Descriptor

src/main/resources/META-INF/persistence.xml

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
    version="2.0">

    <persistence-unit name="example-jpa" transaction-type="JTA">
        <provider>org.hibernate.ejb.HibernatePersistence</provider>
        <jta-data-source>java:jboss/datasources/ExampleJPADS</jta-data-source>
        <class>se.magnuskkarlsson.example.jpa.Company</class>
        <class>se.magnuskkarlsson.example.jpa.Employee</class>
        <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
        <validation-mode>CALLBACK</validation-mode>
        <properties>
            <!-- <property name="hibernate.dialect" value="org.hibernate.dialect.H2Dialect" /> -->
            <property name="hibernate.dialect" value="org.hibernate.dialect.MySQLInnoDBDialect" />
            <property name="hibernate.hbm2ddl.auto" value="create-drop" />

            <property name="hibernate.cache.use_second_level_cache" value="true" />
            <property name="hibernate.cache.use_query_cache" value="true" />
            <property name="hibernate.cache.region.factory_class" value="org.jboss.as.jpa.hibernate4.infinispan.InfinispanRegionFactory" />
            <property name="hibernate.cache.infinispan.cachemanager" value="java:jboss/infinispan/container/hibernate" />
            <property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.JBossTransactionManagerLookup" />

            <property name="hibernate.show_sql" value="false" />
            <property name="hibernate.format_sql" value="false" />
            <property name="hibernate.generate_statistics" value="true" />
            <property name="hibernate.cache.infinispan.statistics" value="true" />
        </properties>
    </persistence-unit>
</persistence>

Reference

Red Hat JBoss EAP Migration Guide

Infinispan User Guide

Wikibook Java Persistence

Test

src/test/resources/META-INF/persistence.xml

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
    version="2.0">

    <persistence-unit name="example-jpa-test" transaction-type="RESOURCE_LOCAL">
        <provider>org.hibernate.ejb.HibernatePersistence</provider>
        <class>se.magnuskkarlsson.example.jpa.Company</class>
        <class>se.magnuskkarlsson.example.jpa.Employee</class>
        <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
        <validation-mode>CALLBACK</validation-mode>
        <properties>
            <property name="hibernate.dialect" value="org.hibernate.dialect.H2Dialect" />
            <property name="hibernate.connection.driver_class" value="org.h2.Driver" />
            <property name="hibernate.connection.url" value="jdbc:h2:mem:" />
            <property name="hibernate.hbm2ddl.auto" value="create-drop" />

            <property name="hibernate.show_sql" value="false" />
            <property name="hibernate.format_sql" value="false" />
            <property name="hibernate.generate_statistics" value="true" />
        </properties>
    </persistence-unit>
</persistence>

src/test/resources/log4j.properties

# configure the root logger
log4j.rootLogger=INFO, STDOUT

# configure the console appender
log4j.appender.STDOUT=org.apache.log4j.ConsoleAppender
log4j.appender.STDOUT.Target=System.out
log4j.appender.STDOUT.layout=org.apache.log4j.PatternLayout
log4j.appender.STDOUT.layout.conversionPattern=%d{yyyy-MM-dd HH:mm:ss.SSS} [%p] %c:%L - %m%n

### https://docs.jboss.org/hibernate/orm/4.2/manual/en-US/html/ch03.html#configuration-logging
# Log all hibernate SQL DML statements as they are executed
log4j.logger.org.hibernate.SQL=TRACE
# Log all hibernate JDBC parameters
log4j.logger.org.hibernate.type=TRACE
# Log all hibernate second-level cache activity
log4j.logger.org.hibernate.cache=TRACE

src/test/java/se/magnuskkarlsson/example/jpa/DomainDAOTest.java

package se.magnuskkarlsson.example.jpa;

import javax.persistence.Persistence;

import junit.framework.Assert;

import org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup;
import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;

public class DomainDAOTest {

    private static final DomainDAO<Company> dao = new DomainDAO<Company>();

    private static final JBossStandaloneJTAManagerLookup lookup = new JBossStandaloneJTAManagerLookup();

    @BeforeClass
    public static void oneTimeSetUp() throws Exception {
        dao.em = Persistence.createEntityManagerFactory("example-jpa-test")
                .createEntityManager();
    }

    @After
    public void tearDown() throws Exception {
    }

    @Test
    public void testCRUD() throws Exception {
        dao.em.getTransaction().begin();

        Company company = new Company();
        company.setName("Foo");
        Company create = dao.create(company);
        Assert.assertNotNull(create);
        Assert.assertNotNull(create.getId());
        System.out.println("### " + create.toString());

        Company get1 = dao.get(Company.class, create.getId());
        Assert.assertNotNull(get1);
        Assert.assertEquals(create.getId(), get1.getId());
        Assert.assertEquals("Foo", get1.getName());

        get1.setName("Bar");
        Company update = dao.update(get1);
        Assert.assertNotNull(update);
        Assert.assertEquals(create.getId(), update.getId());
        Assert.assertEquals("Bar", update.getName());

        Company get2 = dao.get(Company.class, create.getId());
        Assert.assertNotNull(get2);
        Assert.assertEquals(create.getId(), get2.getId());
        Assert.assertEquals("Bar", get2.getName());

        Company delete = dao.delete(Company.class, create.getId());
        Assert.assertNotNull(delete);
        System.out.println("### " + delete);

        Company get3 = dao.get(Company.class, create.getId());
        Assert.assertNull(get3);

        dao.em.getTransaction().commit();
    }

    @Test
    public void testCache() throws Exception {
        dao.em.getTransaction().begin();

        Company company = new Company();
        company.setName("Cache");
        Company create = dao.create(company);
        Assert.assertNotNull(create);
        Assert.assertNotNull(create.getId());
        System.out.println("*** create " + create.toString());

        dao.em.getTransaction().commit();

        dao.em.getTransaction().begin();

        Company get1 = dao.get(Company.class, create.getId());
        Assert.assertNotNull(get1);
        Assert.assertEquals(create.getId(), get1.getId());
        Assert.assertEquals("Cache", get1.getName());
        System.out.println("*** get1 " + get1.toString());

        dao.em.getTransaction().commit();

        dao.em.getTransaction().begin();

        Company get2 = dao.get(Company.class, create.getId());
        Assert.assertNotNull(get2);
        Assert.assertEquals(create.getId(), get2.getId());
        Assert.assertEquals("Cache", get2.getName());
        System.out.println("*** get2 " + get2.toString());

        dao.em.getTransaction().commit();

        System.out.println("!! " + dao.contains(Company.class, create.getId()));
    }
}

Reference

Oracle Adam Bien Integration Testing for Java EE

OpenEJB Example

H2 Database Engine Cheat Sheet

Web Application Test

The testing of Second Level Cache with Infispan is not easily done outside the container. So lets write a simple REST service which we can simply test with e.g. REST client - Google Code

package se.magnuskkarlsson.example.jpa;

import java.util.List;

import org.apache.log4j.Logger;

@javax.ejb.Stateless
@javax.ws.rs.Path("/company")
public class CompanyBean {

    private static final Logger log = Logger.getLogger(CompanyBean.class);

    @javax.inject.Inject
    private DomainDAO<Company> dao;

    // String body: {"name":"Company FOO"}
    @javax.ws.rs.POST
    @javax.ws.rs.Consumes({ "application/json; charset=UTF-8" })
    @javax.ws.rs.Produces({ "application/json; charset=UTF-8" })
    public Company create(Company company) {
        log.info("Enter create(Company) " + company);
        Company entity = dao.create(company);
        log.info("Exit create(Company) " + entity);
        return entity;
    }

    @javax.ws.rs.GET
    @javax.ws.rs.Path("/{id}")
    @javax.ws.rs.Produces({ "application/json; charset=UTF-8" })
    public Company get(@javax.ws.rs.PathParam("id") long id) {
        log.info("Enter get(long) " + id);
        Company entity = dao.get(Company.class, id);
        log.info("Exit get(long) " + entity);
        return entity;
    }

    @javax.ws.rs.GET
    @javax.ws.rs.Produces({ "application/json; charset=UTF-8" })
    public List<Company> getAll() {
        log.info("Enter getAll() ");
        List<Company> entities = dao.getAll(Company.class);
        log.info("Exit get(long) " + entities);
        return entities;
    }

    @javax.ws.rs.GET
    @javax.ws.rs.Path("/contains/{id}")
    @javax.ws.rs.Produces({ "application/json; charset=UTF-8" })
    public boolean contains(@javax.ws.rs.PathParam("id") long id) {
        log.info("Enter contains(long) " + id);
        boolean rtn = dao.contains(Company.class, id);
        log.info("Exit contains(long) " + rtn);
        return rtn;
    }

    @javax.ws.rs.PUT
    @javax.ws.rs.Consumes({ "application/json; charset=UTF-8" })
    @javax.ws.rs.Produces({ "application/json; charset=UTF-8" })
    public Company update(Company company) {
        log.info("Enter update(Company) " + company);
        Company entity = dao.update(company);
        log.info("Exit update(Company) " + entity);
        return entity;
    }

    @javax.ws.rs.DELETE
    @javax.ws.rs.Path("/{id}")
    @javax.ws.rs.Produces({ "application/json; charset=UTF-8" })
    public Company delete(@javax.ws.rs.PathParam("id") long id) {
        log.info("Enter delete(Company) " + id);
        Company entity = dao.delete(Company.class, id);
        log.info("Exit delete(Company) " + entity);
        return entity;
    }
}

And the web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
    version="3.0">

    <display-name>Example JPA</display-name>

    <!-- Auto scan REST service -->
    <context-param>
        <param-name>resteasy.scan</param-name>
        <param-value>true</param-value>
    </context-param>

    <!-- this need same with resteasy servlet url-pattern -->
    <context-param>
        <param-name>resteasy.servlet.mapping.prefix</param-name>
        <param-value>/rest</param-value>
    </context-param>

    <listener>
        <listener-class>org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap</listener-class>
    </listener>

    <servlet>
        <servlet-name>resteasy-servlet</servlet-name>
        <servlet-class>org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>resteasy-servlet</servlet-name>
        <url-pattern>/rest/*</url-pattern>
    </servlet-mapping>
</web-app>

To make this work on JBoss 7 and later we need to add src/main/webapp/WEB-INF/jboss-deployment-structure.xml.

<?xml version="1.0" encoding="UTF-8"?>
<jboss-deployment-structure>
    <deployment>
        <dependencies>
            <module name="org.apache.log4j" />
            <module name="org.hibernate" />
            <module name="org.infinispan" />
        </dependencies>
    </deployment>
</jboss-deployment-structure>

And to make CDI work we add src/main/webapp/WEB-INF/beans.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/beans_1_0.xsd">

</beans>

Finally deploy it on JBoss and play around and watch log with below extra logging in JBoss.

            <!-- Log all hibernate SQL DML statements as they are executed -->
            <logger category="org.hibernate.SQL">
                <level name="TRACE"/>
            </logger>
            <!-- Log all hibernate JDBC parameters -->
            <logger category="org.hibernate.type.descriptor.sql">
                <level name="TRACE"/>
            </logger>
            <!-- Log all hibernate second-level cache activity -->
            <logger category="org.hibernate.cache">
                <level name="TRACE"/>
            </logger>

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>se.magnuskkarlsson.examples</groupId>
    <artifactId>example-jpa</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>war</packaging>
    <name>Example JPA</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
        <hibernate-version>4.2.14.Final</hibernate-version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-api</artifactId>
            <version>7.0</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
            <scope>provided</scope>
        </dependency>

        <!-- Test Support -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.185</version>
            <scope>test</scope>
        </dependency>

        <!-- <dependency> <groupId>org.apache.derby</groupId> <artifactId>derbyclient</artifactId> 
            <version>10.7.1.1</version> <scope>test</scope> </dependency> -->

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.34</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-entitymanager</artifactId>
            <version>${hibernate-version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-infinispan</artifactId>
            <version>${hibernate-version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-validator</artifactId>
            <version>4.3.1.Final</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                    <debug>true</debug>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-ejb-plugin</artifactId>
                <version>2.5</version>
                <configuration>
                    <ejbVersion>3.1</ejbVersion>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.5</version>
            </plugin>
        </plugins>
    </build>
</project>

Java EE 6 Deployment Descriptors

Contexts and Dependency Injection for Java (CDI) 1.0

[META-INF/beans.xml|WEB-INF/beans.xml]

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/beans_1_0.xsd">

</beans>

Java Persistence API (JPA) 2.0

META-INF/persistence.xml

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
    version="2.0">

</persistence>

Enterprise JavaBeans (EJB) 3.1

META-INF/ejb-jar.xml

<?xml version="1.0" encoding="UTF-8"?>
<ejb-jar xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/ejb-jar_3_1.xsd"
    version="3.1">

</ejb-jar>

JBoss Specific Deployment Descriptor

[META-INF/jboss-ejb3.xml|WEB-INF/jboss-ejb3.xml]

<?xml version="1.0" encoding="UTF-8"?>
<jboss:jboss xmlns="http://java.sun.com/xml/ns/javaee" xmlns:jboss="http://www.jboss.com/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:s="urn:security:1.1"
    version="3.1" impl-version="2.0">

</jboss:jboss>

Reference

WildFly Securing EJBs

Java Servlet 3.0

WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
    version="3.0">

</web-app>

JBoss Specific Deployment Descriptor

WEB-INF/jboss-web

<?xml version="1.0" encoding="UTF-8"?>
<jboss-web xmlns="http://www.jboss.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.jboss.com/xml/ns/javaee http://www.jboss.org/j2ee/schema/jboss-web_7_0.xsd"
    version="7.0">

</jboss-web>

Reference

Wikipedia Java EE version history

Oracle Java EE: XML Schemas for Java EE Deployment Descriptors

Gang of Four (GoF) Design Pattern

The old book of Gang of Four (GoF) Design Pattern by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides never get to old. And if you are not familiar with it. Read it!

Here is a good site with some of the most important pattern, with good illustration and example.

http://idiotechie.com/tag/gang-of-four/

Eclipse Top Download Plugins

Eclipse is maybe the most used IDE for java developers, but you also need some plugins to be even more productive. Here are the most downloaded.

1. Subversive / Subclipse - SVN plugins! Who installs it anyway?
2. EGit – Git has won, no wonder the plugin is popular.
3. Eclipse color theme / Moonlight UI – woohoo, we all like things that look pretty, don’t we?
4. Maven integration – Maven is used by 64% of Java developers, so perhaps it could be added into the bundle?
5. Gradle IDE pack – Gradle might very well rule the world eventually. Nice to know it gets traction and the tooling catches up.
6. Android development tools – Eclipse is still the official IDE for Android development.
7. PyDev – Python is flexible, dynamic and installed everywhere by default.
8. Spring Tool Suite (STS) – Spring Framework is an umbrella project for tons of useful libraries and making your IDE aware of them is a smart step–you’ll notice that STS is also in use by 4% of the survey respondents from the introduction, so it’s not easy to ignore.
9. Vaadin framework – Vaadin is an interesting web framework with pure Java components, beautiful widgets, flexibility.
10. JBoss Tools (both Luna and Kepler) – umbrella project to work with all things Red Hat, including JBoss, which is considered by some to be the best Java Application Server there is.
11. GlassFish Tools for Luna – Oracle has cut commercial support of the GlassFish, but it still is the Reference Implementation of Java EE server.
12. EclEmma – a very well-known code coverage tool for Java.
13. FindBugs – a very popular open source, static code analysis tool.
14. TestNG – JUnit is certainly used more than TestNG, but it doesn’t mean that other testing frameworks cannot top it in terms of quality, usability or features.
15. CheckStyle – code quality analysis tool focused on the looks of code. Make your team comply with a chosen code standard and enjoy more readable diffs.
16. JadClipse – a well-known Java Bytecode decompiler.
17. JRebel – a developer productivity tool, allows you to view code changes instantly, which enables developers to get more done in the same amount of time. Become 17% more productive immediately. More effective than a double espresso in the morning.

CSRF and Character Encoding Filter in Tomcat 7

In Tomcat 7 there are several interesting filter, which are ready to be used:

• CsrfPreventionFilter (Cross-Site Request Forgery), for details about CSRF, see OWASP CSRF.
• SetCharacterEncodingFilter, for details see How to Handle Character Encoding in JSP and Servlets.
• ExpiresFilter

There are more out-of-the-box Filter, see FilterBase.

Also check out the Combined Realm org.apache.catalina.realm.LockOutRealm, which can be used to mitigate user password brute force attacks.

Enable JMX Remote in Tomcat 7

Introduction

Remote monitoring is essential in a IT infrastructure. Sadly is the standard Java JMX based on port range. Here I will show you to overcome that in Tomcat 7.

Installation

Download catalina-jmx-remote.jar to $CATALINA_HOME/lib

Configuration

$ vi $CATALINA_HOME/conf/server.xml

<Server port="8005" shutdown="SHUTDOWN">
...
  <Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
          rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002" />
...
</Server>

# vi $CATALINA_HOME/bin/setenv.sh

export JAVA_OPTS="-Dcom.sun.management.jmxremote=true \
                  -Dcom.sun.management.jmxremote.ssl=false \
                  -Dcom.sun.management.jmxremote.authenticate=false"

Test

Start jvisualvm and connect.

Reference

http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html

http://blog.markshead.com/1129/connecting-visual-vm-to-tomcat-7/

https://www.zabbix.com/documentation/2.0/manual/config/items/itemtypes/jmx_monitoring

http://docs.oracle.com/javase/1.5.0/docs/guide/management/agent.html

Use Log4J in Tomcat 7

Introduction

Tomcat uses default Java SDK Logging, which is in mine opinion, generates default log files that are hard to read and it is also quite silly to reinvent the wheel, because there is already a de-facto standard logging framework - log4j.

So lets replace the default logging framework with log4j in Tomcat.

Configuration

1. Create and configure Log4J in $CATALINA_HOME/lib/log4j.properties.
2. Download Log4J 1.2 to $CATALINA_HOME/lib.
3. Download tomcat-juli-adapters.jar to $CATALINA_HOME/lib.
4. Download tomcat-juli.jar and RELPACE existing $CATALINA_HOME/bin/tomcat-juli.jar.
5. Delete $CATALINA_HOME/conf/logging.properties.

$CATALINA_HOME/lib/log4j.properties

I do not find having separate Tomcat log convenient so I let all loggers write to the same file.

log4j.rootLogger = INFO, CATALINA

# Define all the appenders
log4j.appender.CATALINA = org.apache.log4j.DailyRollingFileAppender
log4j.appender.CATALINA.File = ${catalina.base}/logs/catalina.out
log4j.appender.CATALINA.Append = true
log4j.appender.CATALINA.Encoding = UTF-8
log4j.appender.CATALINA.DatePattern = '.'yyyy-MM-dd
log4j.appender.CATALINA.layout = org.apache.log4j.PatternLayout
log4j.appender.CATALINA.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss.SSS} [%p] %c:%L - %m%n

log4j.appender.LOCALHOST = org.apache.log4j.DailyRollingFileAppender
log4j.appender.LOCALHOST.File = ${catalina.base}/logs/catalina.out
log4j.appender.LOCALHOST.Append = true
log4j.appender.LOCALHOST.Encoding = UTF-8
log4j.appender.LOCALHOST.DatePattern = '.'yyyy-MM-dd
log4j.appender.LOCALHOST.layout = org.apache.log4j.PatternLayout
log4j.appender.LOCALHOST.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss.SSS} [%p] %c:%L - %m%n

log4j.appender.MANAGER = org.apache.log4j.DailyRollingFileAppender
log4j.appender.MANAGER.File = ${catalina.base}/logs/catalina.out
log4j.appender.MANAGER.Append = true
log4j.appender.MANAGER.Encoding = UTF-8
log4j.appender.MANAGER.DatePattern = '.'yyyy-MM-dd
log4j.appender.MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.MANAGER.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss.SSS} [%p] %c:%L - %m%n

log4j.appender.HOST-MANAGER = org.apache.log4j.DailyRollingFileAppender
log4j.appender.HOST-MANAGER.File = ${catalina.base}/logs/catalina.out
log4j.appender.HOST-MANAGER.Append = true
log4j.appender.HOST-MANAGER.Encoding = UTF-8
log4j.appender.HOST-MANAGER.DatePattern = '.'yyyy-MM-dd
log4j.appender.HOST-MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.HOST-MANAGER.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss.SSS} [%p] %c:%L - %m%n

log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Encoding = UTF-8
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss.SSS} [%p] %c:%L - %m%n

# Configure which loggers log to which appenders
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost] = INFO, LOCALHOST
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager] = INFO, MANAGER
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager] = INFO, HOST-MANAGER

log4j.logger.org.apache.catalina.realm = TRACE

LDAP Authentication in Tomcat 7

Introduction Tomcat Configuration

Before starting we need to understand the Tomcat configuration (Context) hierarchy. Tomcat configuration can be placed in three places.

    - In $CATALINA_BASE/conf/server.xml.

    - In application /META-INF/context.xml.
    
    - In $CATALINA_BASE/conf/[enginename]/[hostname]/[appname].xml. The default 
    enginename is Catalina. The default hostname is localhost. Which resolves 
    above path to $CATALINA_BASE/conf/Catalina/localhost/[appname].xml.

    [http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Defining_a_context]

The recommended alternative is the last, externally the application, but not intrusive the Tomcat server.

This seperation makes also automated configuration more easily:

1. One package with standardised Tomcat configuration.
2. And another package for each application and their seperated configuration.

Next step is to do the actual Authentication configuration which is done by a Realm component.

    "A Catalina container (Engine, Host, or Context) may contain no more than ONE Realm element" 

    [http://tomcat.apache.org/tomcat-7.0-doc/config/realm.html#Introduction]

So a Realm is kind of like a singletone, but it also have scoope, depending where we place it.

    - Inside an <Engine> element - This Realm will be shared across ALL web 
    applications on ALL virtual hosts, UNLESS it is overridden by a Realm element 
    nested inside a subordinate <Host> or <Context> element.
    
    - Inside a <Host> element - This Realm will be shared across ALL web 
    applications for THIS virtual host, UNLESS it is overridden by a Realm element 
    nested inside a subordinate <Context> element.
    
    - Inside a <Context> element - This Realm will be used ONLY for THIS web application.

    [http://tomcat.apache.org/tomcat-7.0-doc/realm-howto.html#Configuring_a_Realm]

Configuration

Tomcat comes with several authentication modules (Realm) out of the boxes. Here we will use the LDAP authentication module org.apache.catalina.realm.JNDIRealm.

JNDI Directory Realm - org.apache.catalina.realm.JNDIRealm

JNDIRealm

$ vi $CATALINA_BASE/conf/Catalina/localhost/example-ldap.xml

<?xml version="1.0" encoding="UTF-8"?>
<Context>
    <Realm className="org.apache.catalina.realm.JNDIRealm" 
        connectionURL="ldap://ldap.magnuskkarlsson.com:389"
        userPattern="uid={0},ou=People,dc=magnuskkarlsson,dc=com" 
        roleBase="ou=Group,dc=magnuskkarlsson,dc=com"
        roleName="cn" roleSearch="(memberUid={1})" />
</Context>

users.ldif

dn: uid=magkar,ou=People,dc=magnuskkarlsson,dc=com
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: posixAccount
objectClass: shadowAccount
objectClass: top
uid: magkar
givenName: Magnus
initials: K
sn: Karlsson
cn: Magnus K Karlsson
mail: magnus.k.karlsson@msc.se
uidNumber: 500
gidNumber: 500
homeDirectory: /home/student
userPassword: {crypt}$6$TgMwYtfv6Z0C8Peo$H7p8XjUYuBIspHgTmbo2KKSusssqFuBJFi58uRIcFVsObQJr72RHmBEq9Qz8JcBuA0tCbKvmDawxbQzz112/y0
shadowLastChange: 16467
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash

dn: cn=magkar,ou=Group,dc=magnuskkarlsson,dc=com
objectClass: top
objectClass: posixGroup
cn: magkar
userPassword: {crypt}x
gidNumber: 500

dn: cn=tomcat,ou=Group,dc=magnuskkarlsson,dc=com
objectClass: top
objectClass: posixGroup
cn: tomcat
userPassword: {crypt}x
gidNumber: 501
memberUid: magkar

Test

To test this we create a simple web application.

example-ldap.war/index.jsp

<html>
<body>
<p>Remote User : <%= request.getRemoteUser()  %></p>
</body>
</html>

example-ldap.war/WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
    version="3.0" metadata-complete="true">

    <security-constraint>
        <web-resource-collection>
            <url-pattern>/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>tomcat</role-name>
        </auth-constraint>
    </security-constraint>

    <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>Example LDAP Authentication</realm-name>
    </login-config>

    <security-role>
        <role-name>tomcat</role-name>
    </security-role>
</web-app>

Using LDAP as Address Book in Thunderbird

In my previous blog I described how to setup OpenLDAP. In this blog I will describe how to use it as address book in thunderbird.

1. Open Preferences by clicking on menu Edit -> Preferences.
2. Select Composition and Addressing. Click on Edit Directories...

3. From the LDAP Directory Servers window click Add and enter you LDAP.

4. Click OK on all open windows. From main thunderbird window click on tool button Address Book.

5. Select you LDAP server and search in the quick search tool bar textfield and press Enter.

Understanding LDAP and LDAP Authentication

Introduction

If you let an IT administrator pick a persistence storage technique, he would probably choose a LDAP directory. But if you asked a system developer, then he would probably choose a RDBMS (Relational Database Management System). Why is that? I would guess because of history.

So is the two different technique so much different? They both stores data. Right. But they also have some key differences.

• LDAP stores data in a tree (hierarchical database) and RDBMS stores data in tables with relationship between them via primary keys and foreign keys.
• LDAP uses path (Distinguished Name, DN) to make entries unique. RDBMS uses primary keys in tables.
• In a RDBMS you design your tables and columns. In LDAP you pick from predefined object class (RDBMS table), with predefined attributes (RDBMS column).
• In RDBMS you define the column data types, but in LDAP everything are strings.

So by that said lets start to discover LDAP, with OpenLDAP.

Installation

openldap A package containing the libraries necessary to run the OpenLDAP server and client applications.

openldap-clients A package containing the command-line utilities for viewing and modifying directories on an LDAP server.

openldap-servers A package containing both the services and utilities to configure and run an LDAP server. This includes the Standalone LDAP Daemon, slapd.

compat-openldap A package containing the OpenLDAP compatibility libraries.

Choosing a Suffix

The LDAP suffix is the global LDAP name space or the toop root of you LDAP tree.

There are two standard approaches:

1. The X.500 naming model, which is geographically and organizationally based.
   Example: o=Magnus K Karlsson,l=Stockholm,s=Stockholm,c=SE
2. The Internet domain naming model, i.e. the organizations's DNS domain.
   Example: dc=magnuskkarlsson,dc=com

Abbreviation:
dc, Domain Component
dn, Distinguished Name
cn, Common Name
sn, Surname (family name/last name)
uid, User ID
o, Organization
ou, Organization Unit
l, Location
s, State
c, Country
RDN, Relative Distinuished Name
OID, Object Identifier
DIT, Directory Information Tree
LDIF, LDAP Data Interchange Format

The preferred method is the organizations's DNS domain.

Password Storing Policy

Storing password securily is importand. The default way in OpenLDAP is SSHA-1, in RHEL 6 it is SSHA-512 with 8-bit salt, see crypt(3).

NOTES
   Glibc Notes
       The glibc2 version of this function supports additional encryption 
       algorithms.

       If salt is a character string starting with the characters "$id$" 
       followed by a string terminated by "$":

              $id$salt$encrypted

       then instead of using the DES machine, id identifies the encryption 
       method used and this then determines how the rest of the password string 
       is interpreted.  The following values of id are supported:

              ID  | Method
              ---------------------------------------------------------
              1   | MD5
              2a  | Blowfish (not in mainline glibc; added in some
                  | Linux distributions)
              5   | SHA-256 (since glibc 2.7)
              6   | SHA-512 (since glibc 2.7)

       So $5$salt$encrypted is an SHA-256 encoded password and $6$salt$encrypted 
       is an SHA-512 encoded one.

       "salt" stands for the up to 16 characters following "$id$" in the salt. 
       The encrypted part of the password string is the actual computed password.  
       The size of this string is fixed:

       MD5     | 22 characters
       SHA-256 | 43 characters
       SHA-512 | 86 characters

       The characters in "salt" and "encrypted" are drawn from the set 
       [a–zA–Z0–9./]. In the MD5 and SHA implementations the entire key is 
       significant (instead of only the first 8 bytes in DES).

The slappasswd tool can be given options to use crypt(3), see slappasswd(8).

       -c crypt-salt-format
              Specify the format of the salt passed to crypt(3) when generating 
              {CRYPT} passwords.  This string needs to be in sprintf(3) format 
              and may include one (and only one) %s conversion. This  conversion
              will be substituted with a string of random characters from 
              [A-Za-z0-9./]. For example, ’%.2s’ provides a two character salt 
              and ’$1$%.8s’ tells some versions of crypt(3) to use an MD5 
              algorithm and provides 8 random characters of salt. The default is 
              ’%s’, which provides 31 characters of salt.

The recommended way is to use the strongest option, i.e. SSHA-512 with 16 bit salt.

$ slappasswd -c '$6$%.16s'
New password: 
Re-enter new password: 
{CRYPT}$6$/rq5GLnmg5yVZ8XB$CKypmnBEwy1g09R/8k7w8ZCdyelz04DPagKwBS7MakGNqnKSOcftiqW48pg.LBdI0cxAT6yfb1C8sWeD/qeP01

Configuration

The configuration in newer OpenLDAP versions has been moved from a single configuration file (/etc/openldap/slapd.conf) to a configuration directory (/etc/openldap/slapd.d/), containing several configuration files.

For more novice users is the single configuration file more easier to understand and easier to get an overview of all configuration. Here we will use the single configuration file.

$ rm -rf /etc/openldap/slapd.d
$ cp /usr/share/openldap-servers/slapd.conf.obsolete /etc/openldap/slapd.conf

Now we are ready to configure LDAP suffix and username and password for LDAP root user.

$ vi /etc/openldap/slapd.conf
...
suffix          "dc=magnuskkarlsson,dc=com"
rootdn          "cn=admin,dc=magnuskkarlsson,dc=com"
rootpw          "{CRYPT}$6$/rq5GLnmg5yVZ8XB$CKypmnBEwy1g09R/8k7w8ZCdyelz04DPagKwBS7MakGNqnKSOcftiqW48pg.LBdI0cxAT6yfb1C8sWeD/qeP01"
...

Start

$ service slapd start

Test

To test we can run a simple search.

$ ldapsearch -x -b '' -s base '(objectclass=*)' namingContexts
...
dn:
namingContexts: dc=magnuskkarlsson,dc=com
...

Another test can be to print all configuration.

$ ldapsearch -Y EXTERNAL -H ldapi:/// -b cn=config
...
olcRootDN: cn=admin,dc=magnuskkarlsson,dc=com
olcRootPW: {SSHA}nZElz+cA+HvrxmNVKn3tONJhRWphVLTN
...

GUI Administration Tools

Apache Directory Studio Eclipse-based LDAP tools

For Bug on RHEL 6 and CentOS 6:

See resolution https://issues.apache.org/jira/browse/DIRSTUDIO-999

Designing the Name Space

Designing your LDAP tree or Directory Information Tree (DIT) is an important thing.

Flat Name Space

Example:
uid=john,dc=magnuskkarlsson,dc=com
uid=jim,dc=magnuskkarlsson,dc=com

Advantages:

• Names do not need to change when job roles change or the organization changes.
• Simple design avoids need to object categoratization by directory administrators.

Disadvanteges:

• Hard to partition the directory later if needed.
• May be hard to maintain unique DNs.

Deeper Name Space

Example:
uid=peter,ou=Development,ou=People,l=Europe,dc=magnuskkarlsson,dc=com
uid=maria,ou=Sales,ou=People,l=Asia,dc=magnuskkarlsson,dc=com

Advantages:

• Easier to delegate control to organizational units.
• May simplify later partitioning of directory service among several servers.

Disadvanteges:

• Names to tend to change more often: job transfer, organizational changes, etc.
• May require more work to correctly categorize entries, keep up to date.

Compromise Name Space

Example:
ou=People,dc=magnuskkarlsson,dc=com
ou=Group,dc=magnuskkarlsson,dc=com

base.ldif

dn: dc=magnuskkarlsson,dc=com
dc: magnuskkarlsson
objectClass: domain
objectClass: top

dn: ou=People,dc=magnuskkarlsson,dc=com
ou: People
objectClass: organizationalUnit
objectClass: top

dn: ou=Group,dc=magnuskkarlsson,dc=com
ou: Group
objectClass: organizationalUnit
objectClass: top

OpenLDAP Tool

The openldap-clients packages installes a number of tools, the most common used are:

• ldapsearch, tool to search the directory.
• ldapadd and ldapmodify, tool that use LDIF (LDAP Data Interchange Format) files to update the directory.
• ldapdelete, tool to delete entry.

Common options for these tools are:

• -H host
• -x Use simple, not SASL binds (login method)
• -D dn Bind using dn (username)
• -W prompt for simple bind password

Example:

$ ldapadd -x -D "cn=admin,dc=magnuskkarlsson,dc=com" -W -f base.ldif

Common options for ldapsearch:

• -b dn Base DN in tree to start search from.

Example:

$ ldapsearch -x -b 'dc=magnuskkarlsson,dc=com' '(objectclass=*)'

User Class

User Structural Class

After we have designed and created our LDAP tree structure, it's time to create the actual user entry. But before that we need to decide which base class we want to use for our user entry.

Note that an entry must have one and only one structural object class. Each object class have a defined set of attributes, some mandatory and other optional. You can extend or add one or more addition auxiliary object classes.

The two most frequently used structural classes are:

• account Is useful if you are only using LDAP as a NIS (Network Information Service) replacement.
• inetOrgPerson Is best if you are also using LDAP to provide contact information.

User Auxiliary Classes

If you are planning to use the LDAP directory for RHEL authentication, you need to add the following auxiliary classes.

posixAccount represents a line from /etc/passwd.

shadowAccount represents a line from /etc/shadow.

Group Structural Class

To complete a UNIX user/authentication you also needs groups.

posixGroup represents a line from /etc/group.

student.ldif

dn: uid=student,ou=People,dc=magnuskkarlsson,dc=com
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: posixAccount
objectClass: shadowAccount
objectClass: top
cn: student
sn: student
uid: student
uidNumber: 500
gidNumber: 500
homeDirectory: /home/student
userPassword: {crypt}$6$TgMwYtfv6Z0C8Peo$H7p8XjUYuBIspHgTmbo2KKSusssqFuBJFi58uRIcFVsObQJr72RHmBEq9Qz8JcBuA0tCbKvmDawxbQzz112/y0
shadowLastChange: 16467
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash

dn: cn=student,ou=Group,dc=magnuskkarlsson,dc=com
objectClass: top
objectClass: posixGroup
cn: student
userPassword: {crypt}x
gidNumber: 500

Access Control Instructions, ACI

Writing ACI is out of the scoop of this blog, so comment out every ACI in /etc/openldap/slapd.conf, which will give everyone read, but restricts updates to rootdn.

Installation Apache Web Server (httpd) and LDAP authentication

Lets test our new LDAP directory, by configure LDAP authentication against httpd manual pages.

$ sudo yum install httpd httpd-manual

The httpd ldap module is alreaddy by default installed.

$ grep authnz /etc/httpd/conf/httpd.conf 
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so

Lets configure httpd-manual authentication.

$ vi /etc/httpd/conf.d/manual.conf

<Directory "/var/www/manual">
    AuthType Basic
    AuthName "Restricted Resource"
    AuthBasicProvider ldap
    AuthLDAPURL "ldap://ldap.magnuskkarlsson.com:389/ou=People,dc=magnuskkarlsson,dc=com?uid"
    Require valid-user
    Order deny,allow
    Deny from all
    Allow from all
</Directory>

Restart httpd and test.

$ service httpd restart

RHEL 6 LDAP Authentication Migrations Tools

$ sudo yum install migrationtools

Create LDIF export of local user and its password.

$ /usr/share/migrationtools/migrate_passwd.pl /etc/passwd

Create LDIF export of local groups.

$ /usr/share/migrationtools/migrate_group.pl /etc/group

To export and import everything.

$ /usr/share/migrationtools/migrate_all_online.sh
Enter the X.500 naming context you wish to import into: [dc=padl,dc=com] dc=magnuskkarlsson,dc=com
Enter the hostname of your LDAP server [ldap]: ldap.magnuskkarlsson.com
Enter the manager DN: [cn=manager,dc=magnuskkarlsson,dc=com]: cn=admin,dc=magnuskkarlsson,dc=com
Enter the credentials to bind with: 
Do you wish to generate a DUAConfigProfile [yes|no]? no

References

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/ch-Directory_Servers.html

https://help.ubuntu.com/lts/serverguide/openldap-server.html

http://wiki.openiam.com/pages/viewpage.action?pageId=7635198

LDAP Schemas

Default installed LDAP schemas.

$ cat /etc/openldap/slapd.conf
...
include  /etc/openldap/schema/corba.schema
include  /etc/openldap/schema/core.schema
include  /etc/openldap/schema/cosine.schema
include  /etc/openldap/schema/duaconf.schema
include  /etc/openldap/schema/dyngroup.schema
include  /etc/openldap/schema/inetorgperson.schema
include  /etc/openldap/schema/java.schema
include  /etc/openldap/schema/misc.schema
include  /etc/openldap/schema/nis.schema
include  /etc/openldap/schema/openldap.schema
include  /etc/openldap/schema/ppolicy.schema
include  /etc/openldap/schema/collective.schema
...

/etc/openldap/schema/cosine.schema

objectclass ( 0.9.2342.19200300.100.4.5 NAME 'account'
        SUP top STRUCTURAL
        MUST userid
        MAY ( description $ seeAlso $ localityName $
                organizationName $ organizationalUnitName $ host )
        )

/etc/openldap/schema/nis.schema

objectclass ( 1.3.6.1.1.1.2.0 NAME 'posixAccount'
        DESC 'Abstraction of an account with POSIX attributes'
        SUP top AUXILIARY
        MUST ( cn $ uid $ uidNumber $ gidNumber $ homeDirectory )
        MAY ( userPassword $ loginShell $ gecos $ description ) )

objectclass ( 1.3.6.1.1.1.2.1 NAME 'shadowAccount'
        DESC 'Additional attributes for shadow passwords'
        SUP top AUXILIARY
        MUST uid
        MAY ( userPassword $ shadowLastChange $ shadowMin $
              shadowMax $ shadowWarning $ shadowInactive $
              shadowExpire $ shadowFlag $ description ) )

objectclass ( 1.3.6.1.1.1.2.2 NAME 'posixGroup'
        DESC 'Abstraction of a group of accounts'
        SUP top STRUCTURAL
        MUST ( cn $ gidNumber )
        MAY ( userPassword $ memberUid $ description ) )

/etc/openldap/schema/core.schema

objectclass ( 2.5.6.6 NAME 'person'
        DESC 'RFC2256: a person'
        SUP top STRUCTURAL
        MUST ( sn $ cn )
        MAY ( userPassword $ telephoneNumber $ seeAlso $ description ) )

objectclass ( 2.5.6.7 NAME 'organizationalPerson'
        DESC 'RFC2256: an organizational person'
        SUP person STRUCTURAL
        MAY ( title $ x121Address $ registeredAddress $ destinationIndicator $
                preferredDeliveryMethod $ telexNumber $ teletexTerminalIdentifier $
                telephoneNumber $ internationaliSDNNumber $ 
                facsimileTelephoneNumber $ street $ postOfficeBox $ postalCode $
                postalAddress $ physicalDeliveryOfficeName $ ou $ st $ l ) )

/etc/openldap/schema/inetorgperson.schema

objectclass     ( 2.16.840.1.113730.3.2.2
    NAME 'inetOrgPerson'
        DESC 'RFC2798: Internet Organizational Person'
    SUP organizationalPerson
    STRUCTURAL
        MAY (
                audio $ businessCategory $ carLicense $ departmentNumber $
                displayName $ employeeNumber $ employeeType $ givenName $
                homePhone $ homePostalAddress $ initials $ jpegPhoto $
                labeledURI $ mail $ manager $ mobile $ o $ pager $
                photo $ roomNumber $ secretary $ uid $ userCertificate $
                x500uniqueIdentifier $ preferredLanguage $
                userSMIMECertificate $ userPKCS12 )
        )