December 30, 2015

How do you import static in Eclipse?

When you press Ctrl+Shift+O new imports are added and organized. But that will not work for static imports.

But if you place the cursor on the static import method and press Ctrl+Shift+M, then it will be imported.

Example

Now place cursor on each static method and press Ctrl+Shift+M and it will look like.

UTF-8 Charset as Constant in Java 7

Since Java 7 UTF-8 as a constant has been introduced with java.nio.charset.StandardCharsets

Example

Composite Primary Key with Generated and Unique Value with JPA 2.0

Introduction

Sometime you need several column for you primary keys. The background for that can very. There are two solution for that in JPA:

  • @IdClass as inline columns
  • @EmbeddedId with extract column in separate class

Databas Design

  _____________
 |             |
 |   Student   |
 |_____________|
 |             |
 | *studentId  |
 | *groupId    |
 |  firstName  |
 |             |
 |_____________|

@IdClass

@EmbeddedId

The StudentPK class remains the same.

Conclusion

The two above option is very much the same and I cannot see any clear advantage for one solution over the other, so I would recommend use the solution that resembles the real world, i.e. SQL and use IdClass.

Automatic Generated Values

When having composite primary key you cannot use @GeneratedValue, but you can use @TableGenerator. Remember there are three strategies for automatic primary key generation values and the only database vendor neutral solution is TABLE.

  • @GeneratedValue(strategy = IDENTITY), with automatic generated value on column. Works only for MySQL and MS SQL.
  • @GeneratedValue(strategy = SEQUENCE), separate sequence. Works only for PostgreSQL and Oracle.
  • @GeneratedValue(strategy = TABLE), the ONLY database vendor neutral solution. ORM specific implementation, does not rely on any database specific technique. Uses separate table to persist increment value and handles increment by it self.

The StudentPK class remains the same.

Which will generate the following database.

    _____________        _____________
   |             |      |             |
   |   Student   |      |   SEQ_TABLE |
   |_____________|      |_____________|
   |             |      |             |
   | *studentId  |      |  SEQ_NAME   |
   | *groupId    |      |  SEQ_COUNT  |
   |  firstName  |      |             |
   |_____________|      |_____________|

NOTE You do not need to have automatic generated values for both columns.

NOTE You do not need to have unique constraint for id column, I just added it to show that you can, because sometimes you want to have it.

December 29, 2015

Mapping OneToMany with JPA 2.0

Introduction

Mapping one-to-many relationship with JPA 2.0, can be done in 2 ways:

  • @OneToMany, with optional @JoinColumn
  • @OneToMany with @JoinTable

Here I will look closer on option one, which is most common. @JoinTable is more common with @ManyToMany relationship/mapping.

Databas Design

    _____________        _____________
   |             |      |             |
   |   Employee  |      |   Phone     |
   |_____________|      |_____________|
   |             |      |             |
   | *employeeId | -->  | *phoneId    |
   |  firstName  |      | *employeeId |
   |_____________|      | areaCode    |
                        |             |
                        |_____________|

Bidrectional OneToMany Mapping

Now lets write some test code to test this.

And the debug output from Hibernate

As we can see the above create operation, generates 5 SQL statements (3 INSERTS and 2 UPDATES).

The same behavior with multiple INSERT followed by UPDATE, happens when you want to create a new Phone. Then you need to load the Employee class and then call addPhone(Phone). This is not optimal as you can see. Now lets consider another approach with unidirectional mapping.

First Attempt Unidirectional OneToMany Mapping

First we update our mapping annotations./p>

As we can see, we still need 5 SQL operations. But we can do better. Think how the underlying SQL is working. There is nothing magic when JPA or ORM. Lets first create parent Employee and then create children Phone and explicit set foreign key.

Second Attempt Unidirectional OneToMany Mapping

Mapping is the same, but "business logic" is changed.

As we can see the number of SQL statement is now reduced to the natural number that is sensible needed. And when we need to create a new child Phone we do not need to load and add it to parent Employee.

Is there any downside with this solution. Not really if we would like bidirectional behavior we could add a @Transient parameter and set it lazy, just as we would have done with if the mapping was bidirectional. One would maybe think of keeping bidirectional mapping eager, but that is not a good pattern/practice, since you could risk to load huge object graphs into memory, which would impact the performance negative.

The Rest of the Files

To make this complete here is the rest of the code.

src/test/resources/META-INF

pom.xml

September 6, 2015

Simple Login Module in JBoss EAP 6 for Testing

Introduction

When testing locally or in integration tests, it is convenient to use a simple login module. Source code.

Configuration

Test Web App

WEB-INF/web.xml

WEB-INF/jboss-web.xml

August 23, 2015

How to Set Default JVM on Ubuntu

If your system has more than one version of Java, configure which one your system uses by entering the following command in a terminal window

August 12, 2015

How to Log SQL Statements, Parameters and Cache in Hibernate?

Introduction

Current implementation of Hibernate are:

  • Hibernate 4.2 implements JPA 2.0 (EE 6)
  • Hibernate 4.3+ implements JPA 2.1 (EE 7)

Logging

Do not set logging level in persistence.xml (hibernate.show_sql, hibernate.format_sql), instead use Log4J for changing logging level.

You can put this in you src/test/resources/log4j.properties if you want more logging when writing or debugging your test cases.

Reference

August 11, 2015

What is Docker and Why is it Hot?

What is Docker?

It is the possible replacement for Virtualization.

What is the Difference between Virtualization and Docker?

Below is a good picture from https://www.docker.com/whatisdocker.

Docker Diagram Docker Diagram

So the difference between Virtualization and Docker is that everything runs locally on the same server, without the Virtualization overhead.

Example. Lets download a ready image from the Docker Hub (https://hub.docker.com/). You can either browse the Docker Hub via web browser or via command line.

Lets pick the official docker image. The first time you run a new Docker Image, it will download it and that will take a little while.

And now try it by open http://localhost:8888.

So what happened? We started a new process.

And a new tomcat process.

And for network. A new virtual bridge was created, which is basically a virtual router.

Why is Docker Better?

  • Lower licensing costs. You only need ONE license for the virtualized server for ALL its virtual guests.
  • More RAM and CPU. With Docker all process runs on the server directly. No virtualization layer that steals resources.
  • Less disks. A virtual guest is at least 20 GB and a Docker image is around 300 MB, since no duplication of OS and all its libraries are needed.

How Does Docker Work?

Docker makes use of proven mature Linux technologies:

  • CPU and RAM - cgroups. With cgroups you can restrict how much a process can use of the CPU and RAM, much like the same way CPU and RAM is handled in virtualization.
  • Disks - namespaces. With namespaces you can isolate local file system and divide it much like a network filesystem.
  • Network - virtual ethernet bridge.

Common Docker Commands

Installation on Ubuntu 14.04

Install docker.

Test the installation.

Reference

July 17, 2015

Getting Started with Mockito

Introduction

To create effective unit test, they should be fast. But also you might need to mock object out to be able to write unit tests. The most common library to use for mocking is mockito.

Example: Mock expected results


import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testIterator() throws Exception {
        Iterator mock = mock(Iterator.class);
        when(mock.next()).thenReturn("Hello").thenReturn("World");
        assertThat("Hello World", is(equalTo(mock.next() + " " + mock.next())));
    }

Example: Mock expected result with input


import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testArrayList() throws Exception {
        ArrayList mock = mock(ArrayList.class);
        when(mock.get(1)).thenReturn("Foo");
        assertThat("Foo", is(equalTo(mock.get(1))));
    }

Example: Mock expected results with any input


import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testArrayListAny() throws Exception {
        ArrayList mock = mock(ArrayList.class);
        when(mock.get(anyInt())).thenReturn("Foo");
        assertThat("Foo", is(equalTo(mock.get(1))));
    }

Example: Mock and expect exception


import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test(expected = IOException.class)
    public void testOutputStream() throws Exception {
        OutputStream mock = mock(OutputStream.class);
        doThrow(new IOException()).when(mock).close();
        mock.close();
    }

Example: Mock and verify that metods were invoked


import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
...
    @Test
    public void testOutputStreamVerify() throws Exception {
        OutputStream mock = mock(OutputStream.class);
        BufferedOutputStream fos = new BufferedOutputStream(mock);
        fos.close();
        verify(mock).close();
    }

Using Hamcrest with JUnit 4

Introduction

The latest JUnit 4 comes with a transitive dependency of hamcrest. What hamcrest brings to the table is to increase the readability of the test code. This is important because codes are often more read than written.

Example


@Test
public void testEqualTo() throws Exception {
    assertThat(Foo.hello(), is(equalTo("Hello World")));
}

Core

The starting point is org.junit.Assert.assertThat(T actual, Matcher matcher) and the matcher methods in org.hamcrest.CoreMatchers.*:

  • allOf()
  • any()
  • anyOf()
  • anything()
  • describedAs()
  • equalTo()
  • instanceOf()
  • is()
  • not()
  • notNullValue()
  • nullValue()
  • sameInstance()

allOf()

allOf() is a simple assertion that just says all of the matcher arguments must be true.


@Test
public void testAllOf() throws Exception {
    assertThat(Foo.hello(), is(allOf(notNullValue(), instanceOf(String.class), equalTo("Hello World"))));
}

any()

This matcher just checks that the actual result is a class of a certain type.


@Test
public void testAny() throws Exception {
    assertThat(Foo.hello(), is(any(String.class)));
}

anyOf()

If any of the matchers are true this assertion will pass.


@Test
public void testAnyOf() throws Exception {
    assertThat(Foo.hello(), is(anyOf(nullValue(), instanceOf(String.class), equalTo("Goodbye"))));
}

anything()

This matcher will always evaluates to true.

describedAs()

This allows you to override the default description for a matcher.

equalTo()

See above.

instanceOf()

Checks that you have an instance of a particular type.

is()

See above.

not()

Just reverses the outcome of a matcher.


@Test
public void testNot() throws Exception {
    assertThat(Foo.hello(), is(not("Bar")));
}

notNullValue()

Simply does a null check

nullValue()

Assert a null value.

sameInstance()

Assert two Objects are ==, i.e. NOT equals()


@Test
public void testSameInstance() throws Exception {
    assertThat(2 + 2, is(sameInstance(4)));
    assertThat(new Integer(2 + 2), is(not(sameInstance(new Integer(4)))));
}

JUnit testing CDI and EJB with Mockito, OpenEJB and Arquillian

Introduction

Unit testing EE have previously been hard, but since EE 6 and onward the EE specification has been improved on the test front.

Since EE 6 the following has been added:

  • javax.ejb.embeddable.EJBContainer – A standardized test container for EJB.
  • Portable JNDISyntax – Now all EE container must conform to the same JNDI registration and lookup name spaces. Before EE 6, there was no name standard and each EE container had each different name standard.

Do we need CDI and EJB injection for unit testing?

The general rule of thumb is to make unit tests fast, because if they are not, people will starts to disable test cases. And this will make unit tests contra productive.

Loading a CDI or EJB container takes time. Which is also true for Spring Application Context. So try to write your unit test without the need to load some kind of container.

JPA and JDBC can be tested with standard Java SE

See http://magnus-k-karlsson.blogspot.se/2015/02/hibernate-best-practise-jpa-20-and.html.

Logic and EJB can be tested with Mockito

When testing logic or EJB there is in most cases no need to create a EJB context. Try to use mockito, to mock your surroundings.

EJB Container Unit Testing

In some cases there is actually a need to create an EJB context. The two most common solutions are:

  • OpenEJB
  • Arquillian

OpenEJB

Pros:

  • Is faster to start and stop than Arquillian
  • More easier to learn
  • Easy to use maven dependency

Cons:

  • Only support standard EE (which is also a pro, since then you know your code will run in any EE container.).
  • Does not support mere CDI, must use first an EJB to test pure CDI functionality. Can put EJB in test directory.

See

Arquillian

Pros:

  • Can test EE container specific code.
  • Support test of pure CDI code.

Cons:

  • Is slower than OpenEJB.
  • Its maven dependencies are long and complex.
  • Is harder to learn compared with OpenEJB.

See:

July 15, 2015

Why EJB?

Introduction

When people think of Java EE, they think in first hand of EJB, even though the EE specification contains much than EJB. For example Rod Johnson (founder of Spring) pointed out in the Expert One-on-One J2EE Development without EJB that JDBC, JTA and JMS are successful parts of EE.

But a lot has happen when this book was written and recent EE version starting with 5, but especially 6 and 7 had made EE development even easier than Spring development.

So if you are thinking of choosing EE, you might wonder what EJB contributes with:

  1. Pooling
  2. Transaction
  3. Security
  4. Monitoring and Management
  5. Asynchronous

Pooling

This makes accessing a EJB faster. Typically usage with Spring is to lazy inject things when a Business Object is requested.

Pooling is also a DOS protection since you cannot request endless number of Business Object. When the pool is empty of non busy EJB, the request will fail fast. This handling is important, since accepting new request when the server is overloaded will only worsen the situation.

Further sizing and resizing the pool and other pool is typically something a application administrator want to do after the deployment in production and this is done from unified interface in the EE container. No need to go in each application and know its inner configuration.

Transaction

Every public method will default start a new transaction if no one exists.

Security

The EE specification contains a good security API and is easy to use and covers all the EE components.

Since the EE 6 the EAR archive is history. Now EJB can be bundle in standard WAR archive.

And standard web.xml security is quite straight forward.

Monitoring and Management

With the emerge of DevOps (Development Operation) the monitoring and management has grown in importance (even though it always has been important).

All modern EE container has today this capability of monitoring and manage EJB as a standard.

Asynchronous

javax.ejb.Asynchronous was introduced in EE 6 and is easy to use. Example to make a method asynchronous:


@Stateless
public class OrderProcessorBean {

    // synchronous method
    public boolean validate(Order order) throws PaymentException {
        ...
    }

    @Asynchronous
    public Future processPayment(Order order) throws PaymentException {
        ...
    }
}

June 11, 2015

Use YubiKey for 2 Factor Authentication on Ubuntu 14.04

To increase the security for your Google Account you can use 2 factor authentication with SMS. But also you can use YubiKey. To get that to work on Ubuntu 14.04 follow the instruction here http://ihaveagadget.blogspot.se/2014/11/yubico-u2f-key-not-working-in-ubuntu.html.

How to Create a Bootable USB for Linux ISO

Introduction

The most generic way to create a bootable USB with a Linux ISO file, is to use the dd command.

Prerequisite

On RHEL/CentOS/Fedora

# yum install syslinux

On Ubuntu/Debian

$ sudo apt-get install syslinux

Steps

1. Download ISO file

2. Convert your normal ISO file to a hybrid ISO.

# isohybrid /home/magnus/Downloads/ubuntu-12.04.4-desktop-amd64.iso

3. Create a bootable USB.


# dd if=/home/magnus/Downloads/ubuntu-12.04.4-desktop-amd64.iso of=/dev/sdf
1501184+0 records in
1501184+0 records out
768606208 bytes (769 MB) copied, 286.08 s, 2.7 MB/s

NOTE: the device name is without number, e.g. /dev/sdf1.

If you are unsure what the device name is you can e.g. use lsblk.


# lsblk
NAME                        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0                          11:0    1  1024M  0 rom  
sda                           8:0    0 167.7G  0 disk 
├─sda1                        8:1    0   500M  0 part /boot
└─sda2                        8:2    0 146.5G  0 part 
  ├─vg_rhel6-lv_root (dm-0) 253:0    0   127G  0 lvm  /
  └─vg_rhel6-lv_swap (dm-1) 253:1    0  19.5G  0 lvm  [SWAP]
sdf                           8:80   0   3.7G  0 disk 

March 6, 2015

Crontab Expression

I do not write contab scheduling expression every day, so I tend to forget the expression syntax. Below is a good site for generating such expressions.

http://www.crontab-generator.org/

March 3, 2015

Puppet IDE Geppetto

When writing puppet modules I would recommend using an IDE when writing puppet code. The IDE from puppetlab is the most mature - Geppetto. Geppetto is fine, but one warning when installing it.

DO NOT install Geppetto as a plugin in an existing Eclipse. Instead I recommend downloading the bundle version http://puppetlabs.github.io/geppetto/download.html.

March 2, 2015

How to Install Tomcat 6 on RHEL 6

A good and accurate guide exists on http://oracle-base.com/articles/linux/apache-tomcat-installation-on-linux.php

Maven RPM Plugins

There are two major maven rpm plugins.

  1. rpm-maven-plugin The Codehaus mojo plugin is good and holds good quality as almost all mojo plugins, but has one big disadvantage. It requires rpmbuild linux tool, which means it will not run without pain on Windows. You could of cource switch to a better os, but that is often easier said than done in bigger organizations.
  2. Redline RPM The second tool is build on Redline, which is a pure java library, which means it will run on windows. The backdraw with this one, is that it exists no maven plugin for it, only ant is supported out of the box.

February 18, 2015

Hibernate Best Practise, JPA 2.0 and Second Level Cache with Infinispan

Introduction

The technique of using ORM has been becoming the defacto standard, when developing new application, but after passed the level of school books example, the usage of using ORM is not so simple. In this blog I will talk about minimizing code for simple CRUD actions, but also arguing about getting back to basic for complex side of ORM.

Before started, one must recognised that the number one implementation of ORM is hibernate and that is the implementation framework used here for the specification JPA 2.0 contained in EE 6.

ORM Best Practise

DRY AbstractDomain

All domain object contain some common characteristic, e.g. primary key, toString method, maybe created and last modified date time, etc. Put all those things in a abstract domain class

Note about abstract domain class.

  • Every domain class will be serializable.
  • We add a toString method, which is realized with reflection. Reflection is not the fastest technique, which is OK for a toString method, since it should not be called often.
  • We add javax.persistence.Cacheable(true), for preparing for entity second level cache.

Next is to create a concrete domain class.

Note about concrete domain class.

  • We make use of javax.persistence.AttributeOverride to specialize a primary key for concrete domain class.
  • WE DO NOT MAP RELATIONSHIP WITH ORM. Se below.

KISS. Back to basic for relationship.

Not using ORM for our relationship, might sound crazy at first, but the fact is that managing relationship with ORM is hard and comes with several not obvious side effects.

So what went wrong? Lets start from the beginning. First we mapped all our relationship with ORM. Next we was challenged with how do we load all relationship. You might then started with loading all relationship eagerly. But after testing your implementation with more production like data volume, you realize that you are loading big chunk of your database in memory. This does not hold. OK, you comes to your senses and add lazy loading to all your relationship.

So what happened next? You started to optimize all your queries so you did not need to load each sub children separately. You probably now came across exception like org.hibernate.loader.MultipleBagFetchException: cannot simultaneously fetch multiple bags. And problem like loading loading N number of extra data. Or why do you need to load all children for inserting a new child to a parent?

When reaching this point you start to doubt the entire usage and justification of ORM. And that in fact is where I somewhere also landed. So lets get rid of all complexity it means of handling relationship with ORM and get back to basic - KISS.

DRY DomainDAO

We extract common CRUD task to a DAO base class.

Deployment Descriptor

src/main/resources/META-INF/persistence.xml

Reference

Red Hat JBoss EAP Migration Guide

Infinispan User Guide

Wikibook Java Persistence

Test

src/test/resources/META-INF/persistence.xml

src/test/resources/log4j.properties

src/test/java/se/magnuskkarlsson/example/jpa/DomainDAOTest.java

Reference

Oracle Adam Bien Integration Testing for Java EE

OpenEJB Example

H2 Database Engine Cheat Sheet

Web Application Test

The testing of Second Level Cache with Infispan is not easily done outside the container. So lets write a simple REST service which we can simply test with e.g. REST client - Google Code

And the web.xml

To make this work on JBoss 7 and later we need to add src/main/webapp/WEB-INF/jboss-deployment-structure.xml.

And to make CDI work we add src/main/webapp/WEB-INF/beans.xml

Finally deploy it on JBoss and play around and watch log with below extra logging in JBoss.

pom.xml

February 13, 2015

Java EE 6 Deployment Descriptors

Contexts and Dependency Injection for Java (CDI) 1.0

[META-INF/beans.xml|WEB-INF/beans.xml]

Java Persistence API (JPA) 2.0

META-INF/persistence.xml

Enterprise JavaBeans (EJB) 3.1

META-INF/ejb-jar.xml

JBoss Specific Deployment Descriptor

[META-INF/jboss-ejb3.xml|WEB-INF/jboss-ejb3.xml]

Reference

WildFly Securing EJBs

Java Servlet 3.0

WEB-INF/web.xml

JBoss Specific Deployment Descriptor

WEB-INF/jboss-web

Reference

Wikipedia Java EE version history

Oracle Java EE: XML Schemas for Java EE Deployment Descriptors

February 10, 2015

Gang of Four (GoF) Design Pattern

The old book of Gang of Four (GoF) Design Pattern by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides never get to old. And if you are not familiar with it. Read it!

Here is a good site with some of the most important pattern, with good illustration and example.

http://idiotechie.com/tag/gang-of-four/

Eclipse Top Download Plugins

Eclipse is maybe the most used IDE for java developers, but you also need some plugins to be even more productive. Here are the most downloaded.

  1. Subversive / Subclipse - SVN plugins! Who installs it anyway?
  2. EGit – Git has won, no wonder the plugin is popular.
  3. Eclipse color theme / Moonlight UI – woohoo, we all like things that look pretty, don’t we?
  4. Maven integration – Maven is used by 64% of Java developers, so perhaps it could be added into the bundle?
  5. Gradle IDE pack – Gradle might very well rule the world eventually. Nice to know it gets traction and the tooling catches up.
  6. Android development tools – Eclipse is still the official IDE for Android development.
  7. PyDev – Python is flexible, dynamic and installed everywhere by default.
  8. Spring Tool Suite (STS) – Spring Framework is an umbrella project for tons of useful libraries and making your IDE aware of them is a smart step–you’ll notice that STS is also in use by 4% of the survey respondents from the introduction, so it’s not easy to ignore.
  9. Vaadin framework – Vaadin is an interesting web framework with pure Java components, beautiful widgets, flexibility.
  10. JBoss Tools (both Luna and Kepler) – umbrella project to work with all things Red Hat, including JBoss, which is considered by some to be the best Java Application Server there is.
  11. GlassFish Tools for Luna – Oracle has cut commercial support of the GlassFish, but it still is the Reference Implementation of Java EE server.
  12. EclEmma – a very well-known code coverage tool for Java.
  13. FindBugs – a very popular open source, static code analysis tool.
  14. TestNG – JUnit is certainly used more than TestNG, but it doesn’t mean that other testing frameworks cannot top it in terms of quality, usability or features.
  15. CheckStyle – code quality analysis tool focused on the looks of code. Make your team comply with a chosen code standard and enjoy more readable diffs.
  16. JadClipse – a well-known Java Bytecode decompiler.
  17. JRebel – a developer productivity tool, allows you to view code changes instantly, which enables developers to get more done in the same amount of time. Become 17% more productive immediately. More effective than a double espresso in the morning.

CSRF and Character Encoding Filter in Tomcat 7

In Tomcat 7 there are several interesting filter, which are ready to be used:

There are more out-of-the-box Filter, see FilterBase.

Also check out the Combined Realm org.apache.catalina.realm.LockOutRealm, which can be used to mitigate user password brute force attacks.

February 9, 2015

Enable JMX Remote in Tomcat 7

Introduction

Remote monitoring is essential in a IT infrastructure. Sadly is the standard Java JMX based on port range. Here I will show you to overcome that in Tomcat 7.

Installation

Download catalina-jmx-remote.jar to $CATALINA_HOME/lib

Configuration

Test

Start jvisualvm and connect.

Reference

http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html

http://blog.markshead.com/1129/connecting-visual-vm-to-tomcat-7/

https://www.zabbix.com/documentation/2.0/manual/config/items/itemtypes/jmx_monitoring

http://docs.oracle.com/javase/1.5.0/docs/guide/management/agent.html

Use Log4J in Tomcat 7

Introduction

Tomcat uses default Java SDK Logging, which is in mine opinion, generates default log files that are hard to read and it is also quite silly to reinvent the wheel, because there is already a de-facto standard logging framework - log4j.

So lets replace the default logging framework with log4j in Tomcat.

Configuration

  1. Create and configure Log4J in $CATALINA_HOME/lib/log4j.properties.
  2. Download Log4J 1.2 to $CATALINA_HOME/lib.
  3. Download tomcat-juli-adapters.jar to $CATALINA_HOME/lib.
  4. Download tomcat-juli.jar and RELPACE existing $CATALINA_HOME/bin/tomcat-juli.jar.
  5. Delete $CATALINA_HOME/conf/logging.properties.

$CATALINA_HOME/lib/log4j.properties

I do not find having separate Tomcat log convenient so I let all loggers write to the same file.

LDAP Authentication in Tomcat 7

Introduction Tomcat Configuration

Before starting we need to understand the Tomcat configuration (Context) hierarchy. Tomcat configuration can be placed in three places.


    - In $CATALINA_BASE/conf/server.xml.

    - In application /META-INF/context.xml.
    
    - In $CATALINA_BASE/conf/[enginename]/[hostname]/[appname].xml. The default 
    enginename is Catalina. The default hostname is localhost. Which resolves 
    above path to $CATALINA_BASE/conf/Catalina/localhost/[appname].xml.

    [http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Defining_a_context]

The recommended alternative is the last, externally the application, but not intrusive the Tomcat server.

This seperation makes also automated configuration more easily:

  1. One package with standardised Tomcat configuration.
  2. And another package for each application and their seperated configuration.

Next step is to do the actual Authentication configuration which is done by a Realm component.


    "A Catalina container (Engine, Host, or Context) may contain no more than ONE Realm element" 

    [http://tomcat.apache.org/tomcat-7.0-doc/config/realm.html#Introduction]

So a Realm is kind of like a singletone, but it also have scoope, depending where we place it.


    - Inside an  element - This Realm will be shared across ALL web 
    applications on ALL virtual hosts, UNLESS it is overridden by a Realm element 
    nested inside a subordinate  or  element.
    
    - Inside a  element - This Realm will be shared across ALL web 
    applications for THIS virtual host, UNLESS it is overridden by a Realm element 
    nested inside a subordinate  element.
    
    - Inside a  element - This Realm will be used ONLY for THIS web application.

    [http://tomcat.apache.org/tomcat-7.0-doc/realm-howto.html#Configuring_a_Realm]

Configuration

Tomcat comes with several authentication modules (Realm) out of the boxes. Here we will use the LDAP authentication module org.apache.catalina.realm.JNDIRealm.

JNDI Directory Realm - org.apache.catalina.realm.JNDIRealm

JNDIRealm

users.ldif

Test

To test this we create a simple web application.

example-ldap.war/index.jsp

example-ldap.war/WEB-INF/web.xml

February 8, 2015

Using LDAP as Address Book in Thunderbird

In my previous blog I described how to setup OpenLDAP. In this blog I will describe how to use it as address book in thunderbird.

  1. Open Preferences by clicking on menu Edit -> Preferences.
  2. Select Composition and Addressing. Click on Edit Directories...
  3. From the LDAP Directory Servers window click Add and enter you LDAP.
  4. Click OK on all open windows. From main thunderbird window click on tool button Address Book.
  5. Select you LDAP server and search in the quick search tool bar textfield and press Enter.

Understanding LDAP and LDAP Authentication

Introduction

If you let an IT administrator pick a persistence storage technique, he would probably choose a LDAP directory. But if you asked a system developer, then he would probably choose a RDBMS (Relational Database Management System). Why is that? I would guess because of history.

So is the two different technique so much different? They both stores data. Right. But they also have some key differences.

  • LDAP stores data in a tree (hierarchical database) and RDBMS stores data in tables with relationship between them via primary keys and foreign keys.
  • LDAP uses path (Distinguished Name, DN) to make entries unique. RDBMS uses primary keys in tables.
  • In a RDBMS you design your tables and columns. In LDAP you pick from predefined object class (RDBMS table), with predefined attributes (RDBMS column).
  • In RDBMS you define the column data types, but in LDAP everything are strings.

So by that said lets start to discover LDAP, with OpenLDAP.

Installation

openldap A package containing the libraries necessary to run the OpenLDAP server and client applications.

openldap-clients A package containing the command-line utilities for viewing and modifying directories on an LDAP server.

openldap-servers A package containing both the services and utilities to configure and run an LDAP server. This includes the Standalone LDAP Daemon, slapd.

compat-openldap A package containing the OpenLDAP compatibility libraries.

Choosing a Suffix

The LDAP suffix is the global LDAP name space or the toop root of you LDAP tree.

There are two standard approaches:

  1. The X.500 naming model, which is geographically and organizationally based.
    Example: o=Magnus K Karlsson,l=Stockholm,s=Stockholm,c=SE
  2. The Internet domain naming model, i.e. the organizations's DNS domain.
    Example: dc=magnuskkarlsson,dc=com

Abbreviation:
dc, Domain Component
dn, Distinguished Name
cn, Common Name
sn, Surname (family name/last name)
uid, User ID
o, Organization
ou, Organization Unit
l, Location
s, State
c, Country
RDN, Relative Distinuished Name
OID, Object Identifier
DIT, Directory Information Tree
LDIF, LDAP Data Interchange Format

The preferred method is the organizations's DNS domain.

Password Storing Policy

Storing password securily is importand. The default way in OpenLDAP is SSHA-1, in RHEL 6 it is SSHA-512 with 8-bit salt, see crypt(3).


NOTES
   Glibc Notes
       The glibc2 version of this function supports additional encryption 
       algorithms.

       If salt is a character string starting with the characters "$id$" 
       followed by a string terminated by "$":

              $id$salt$encrypted

       then instead of using the DES machine, id identifies the encryption 
       method used and this then determines how the rest of the password string 
       is interpreted.  The following values of id are supported:

              ID  | Method
              ---------------------------------------------------------
              1   | MD5
              2a  | Blowfish (not in mainline glibc; added in some
                  | Linux distributions)
              5   | SHA-256 (since glibc 2.7)
              6   | SHA-512 (since glibc 2.7)

       So $5$salt$encrypted is an SHA-256 encoded password and $6$salt$encrypted 
       is an SHA-512 encoded one.

       "salt" stands for the up to 16 characters following "$id$" in the salt. 
       The encrypted part of the password string is the actual computed password.  
       The size of this string is fixed:

       MD5     | 22 characters
       SHA-256 | 43 characters
       SHA-512 | 86 characters

       The characters in "salt" and "encrypted" are drawn from the set 
       [a–zA–Z0–9./]. In the MD5 and SHA implementations the entire key is 
       significant (instead of only the first 8 bytes in DES).

The slappasswd tool can be given options to use crypt(3), see slappasswd(8).


       -c crypt-salt-format
              Specify the format of the salt passed to crypt(3) when generating 
              {CRYPT} passwords.  This string needs to be in sprintf(3) format 
              and may include one (and only one) %s conversion. This  conversion
              will be substituted with a string of random characters from 
              [A-Za-z0-9./]. For example, ’%.2s’ provides a two character salt 
              and ’$1$%.8s’ tells some versions of crypt(3) to use an MD5 
              algorithm and provides 8 random characters of salt. The default is 
              ’%s’, which provides 31 characters of salt.

The recommended way is to use the strongest option, i.e. SSHA-512 with 16 bit salt.

Configuration

The configuration in newer OpenLDAP versions has been moved from a single configuration file (/etc/openldap/slapd.conf) to a configuration directory (/etc/openldap/slapd.d/), containing several configuration files.

For more novice users is the single configuration file more easier to understand and easier to get an overview of all configuration. Here we will use the single configuration file.

Now we are ready to configure LDAP suffix and username and password for LDAP root user.

Start

Test

To test we can run a simple search.

Another test can be to print all configuration.

GUI Administration Tools

Apache Directory Studio Eclipse-based LDAP tools

For Bug on RHEL 6 and CentOS 6:

See resolution https://issues.apache.org/jira/browse/DIRSTUDIO-999

Designing the Name Space

Designing your LDAP tree or Directory Information Tree (DIT) is an important thing.

Flat Name Space

Example:
uid=john,dc=magnuskkarlsson,dc=com
uid=jim,dc=magnuskkarlsson,dc=com

Advantages:

  • Names do not need to change when job roles change or the organization changes.
  • Simple design avoids need to object categoratization by directory administrators.

Disadvanteges:

  • Hard to partition the directory later if needed.
  • May be hard to maintain unique DNs.

Deeper Name Space

Example:
uid=peter,ou=Development,ou=People,l=Europe,dc=magnuskkarlsson,dc=com
uid=maria,ou=Sales,ou=People,l=Asia,dc=magnuskkarlsson,dc=com

Advantages:

  • Easier to delegate control to organizational units.
  • May simplify later partitioning of directory service among several servers.

Disadvanteges:

  • Names to tend to change more often: job transfer, organizational changes, etc.
  • May require more work to correctly categorize entries, keep up to date.

Compromise Name Space

Example:
ou=People,dc=magnuskkarlsson,dc=com
ou=Group,dc=magnuskkarlsson,dc=com

base.ldif

OpenLDAP Tool

The openldap-clients packages installes a number of tools, the most common used are:

  • ldapsearch, tool to search the directory.
  • ldapadd and ldapmodify, tool that use LDIF (LDAP Data Interchange Format) files to update the directory.
  • ldapdelete, tool to delete entry.

Common options for these tools are:

  • -H host
  • -x Use simple, not SASL binds (login method)
  • -D dn Bind using dn (username)
  • -W prompt for simple bind password

Example:

Common options for ldapsearch:

  • -b dn Base DN in tree to start search from.

Example:

User Class

User Structural Class

After we have designed and created our LDAP tree structure, it's time to create the actual user entry. But before that we need to decide which base class we want to use for our user entry.

Note that an entry must have one and only one structural object class. Each object class have a defined set of attributes, some mandatory and other optional. You can extend or add one or more addition auxiliary object classes.

The two most frequently used structural classes are:

  • account Is useful if you are only using LDAP as a NIS (Network Information Service) replacement.
  • inetOrgPerson Is best if you are also using LDAP to provide contact information.

User Auxiliary Classes

If you are planning to use the LDAP directory for RHEL authentication, you need to add the following auxiliary classes.

posixAccount represents a line from /etc/passwd.

shadowAccount represents a line from /etc/shadow.

Group Structural Class

To complete a UNIX user/authentication you also needs groups.

posixGroup represents a line from /etc/group.

student.ldif

Access Control Instructions, ACI

Writing ACI is out of the scoop of this blog, so comment out every ACI in /etc/openldap/slapd.conf, which will give everyone read, but restricts updates to rootdn.

Installation Apache Web Server (httpd) and LDAP authentication

Lets test our new LDAP directory, by configure LDAP authentication against httpd manual pages.

The httpd ldap module is alreaddy by default installed.

Lets configure httpd-manual authentication.

Restart httpd and test.

RHEL 6 LDAP Authentication Migrations Tools

Create LDIF export of local user and its password.

Create LDIF export of local groups.

To export and import everything.

References

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/ch-Directory_Servers.html

https://help.ubuntu.com/lts/serverguide/openldap-server.html

http://wiki.openiam.com/pages/viewpage.action?pageId=7635198

LDAP Schemas

Default installed LDAP schemas.

/etc/openldap/schema/cosine.schema


objectclass ( 0.9.2342.19200300.100.4.5 NAME 'account'
        SUP top STRUCTURAL
        MUST userid
        MAY ( description $ seeAlso $ localityName $
                organizationName $ organizationalUnitName $ host )
        )

/etc/openldap/schema/nis.schema


objectclass ( 1.3.6.1.1.1.2.0 NAME 'posixAccount'
        DESC 'Abstraction of an account with POSIX attributes'
        SUP top AUXILIARY
        MUST ( cn $ uid $ uidNumber $ gidNumber $ homeDirectory )
        MAY ( userPassword $ loginShell $ gecos $ description ) )

objectclass ( 1.3.6.1.1.1.2.1 NAME 'shadowAccount'
        DESC 'Additional attributes for shadow passwords'
        SUP top AUXILIARY
        MUST uid
        MAY ( userPassword $ shadowLastChange $ shadowMin $
              shadowMax $ shadowWarning $ shadowInactive $
              shadowExpire $ shadowFlag $ description ) )

objectclass ( 1.3.6.1.1.1.2.2 NAME 'posixGroup'
        DESC 'Abstraction of a group of accounts'
        SUP top STRUCTURAL
        MUST ( cn $ gidNumber )
        MAY ( userPassword $ memberUid $ description ) )

/etc/openldap/schema/core.schema


objectclass ( 2.5.6.6 NAME 'person'
        DESC 'RFC2256: a person'
        SUP top STRUCTURAL
        MUST ( sn $ cn )
        MAY ( userPassword $ telephoneNumber $ seeAlso $ description ) )

objectclass ( 2.5.6.7 NAME 'organizationalPerson'
        DESC 'RFC2256: an organizational person'
        SUP person STRUCTURAL
        MAY ( title $ x121Address $ registeredAddress $ destinationIndicator $
                preferredDeliveryMethod $ telexNumber $ teletexTerminalIdentifier $
                telephoneNumber $ internationaliSDNNumber $ 
                facsimileTelephoneNumber $ street $ postOfficeBox $ postalCode $
                postalAddress $ physicalDeliveryOfficeName $ ou $ st $ l ) )

/etc/openldap/schema/inetorgperson.schema


objectclass     ( 2.16.840.1.113730.3.2.2
    NAME 'inetOrgPerson'
        DESC 'RFC2798: Internet Organizational Person'
    SUP organizationalPerson
    STRUCTURAL
        MAY (
                audio $ businessCategory $ carLicense $ departmentNumber $
                displayName $ employeeNumber $ employeeType $ givenName $
                homePhone $ homePostalAddress $ initials $ jpegPhoto $
                labeledURI $ mail $ manager $ mobile $ o $ pager $
                photo $ roomNumber $ secretary $ uid $ userCertificate $
                x500uniqueIdentifier $ preferredLanguage $
                userSMIMECertificate $ userPKCS12 )
        )