February 18, 2015

Hibernate Best Practise, JPA 2.0 and Second Level Cache with Infinispan

Introduction

The technique of using ORM has been becoming the defacto standard, when developing new application, but after passed the level of school books example, the usage of using ORM is not so simple. In this blog I will talk about minimizing code for simple CRUD actions, but also arguing about getting back to basic for complex side of ORM.

Before started, one must recognised that the number one implementation of ORM is hibernate and that is the implementation framework used here for the specification JPA 2.0 contained in EE 6.

ORM Best Practise

DRY AbstractDomain

All domain object contain some common characteristic, e.g. primary key, toString method, maybe created and last modified date time, etc. Put all those things in a abstract domain class

Note about abstract domain class.

  • Every domain class will be serializable.
  • We add a toString method, which is realized with reflection. Reflection is not the fastest technique, which is OK for a toString method, since it should not be called often.
  • We add javax.persistence.Cacheable(true), for preparing for entity second level cache.

Next is to create a concrete domain class.

Note about concrete domain class.

  • We make use of javax.persistence.AttributeOverride to specialize a primary key for concrete domain class.
  • WE DO NOT MAP RELATIONSHIP WITH ORM. Se below.

KISS. Back to basic for relationship.

Not using ORM for our relationship, might sound crazy at first, but the fact is that managing relationship with ORM is hard and comes with several not obvious side effects.

So what went wrong? Lets start from the beginning. First we mapped all our relationship with ORM. Next we was challenged with how do we load all relationship. You might then started with loading all relationship eagerly. But after testing your implementation with more production like data volume, you realize that you are loading big chunk of your database in memory. This does not hold. OK, you comes to your senses and add lazy loading to all your relationship.

So what happened next? You started to optimize all your queries so you did not need to load each sub children separately. You probably now came across exception like org.hibernate.loader.MultipleBagFetchException: cannot simultaneously fetch multiple bags. And problem like loading loading N number of extra data. Or why do you need to load all children for inserting a new child to a parent?

When reaching this point you start to doubt the entire usage and justification of ORM. And that in fact is where I somewhere also landed. So lets get rid of all complexity it means of handling relationship with ORM and get back to basic - KISS.

DRY DomainDAO

We extract common CRUD task to a DAO base class.

Deployment Descriptor

src/main/resources/META-INF/persistence.xml

Reference

Red Hat JBoss EAP Migration Guide

Infinispan User Guide

Wikibook Java Persistence

Test

src/test/resources/META-INF/persistence.xml

src/test/resources/log4j.properties

src/test/java/se/magnuskkarlsson/example/jpa/DomainDAOTest.java

Reference

Oracle Adam Bien Integration Testing for Java EE

OpenEJB Example

H2 Database Engine Cheat Sheet

Web Application Test

The testing of Second Level Cache with Infispan is not easily done outside the container. So lets write a simple REST service which we can simply test with e.g. REST client - Google Code

And the web.xml

To make this work on JBoss 7 and later we need to add src/main/webapp/WEB-INF/jboss-deployment-structure.xml.

And to make CDI work we add src/main/webapp/WEB-INF/beans.xml

Finally deploy it on JBoss and play around and watch log with below extra logging in JBoss.

pom.xml

February 13, 2015

Java EE 6 Deployment Descriptors

Contexts and Dependency Injection for Java (CDI) 1.0

[META-INF/beans.xml|WEB-INF/beans.xml]

Java Persistence API (JPA) 2.0

META-INF/persistence.xml

Enterprise JavaBeans (EJB) 3.1

META-INF/ejb-jar.xml

JBoss Specific Deployment Descriptor

[META-INF/jboss-ejb3.xml|WEB-INF/jboss-ejb3.xml]

Reference

WildFly Securing EJBs

Java Servlet 3.0

WEB-INF/web.xml

JBoss Specific Deployment Descriptor

WEB-INF/jboss-web

Reference

Wikipedia Java EE version history

Oracle Java EE: XML Schemas for Java EE Deployment Descriptors

February 10, 2015

Gang of Four (GoF) Design Pattern

The old book of Gang of Four (GoF) Design Pattern by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides never get to old. And if you are not familiar with it. Read it!

Here is a good site with some of the most important pattern, with good illustration and example.

http://idiotechie.com/tag/gang-of-four/

Eclipse Top Download Plugins

Eclipse is maybe the most used IDE for java developers, but you also need some plugins to be even more productive. Here are the most downloaded.

  1. Subversive / Subclipse - SVN plugins! Who installs it anyway?
  2. EGit – Git has won, no wonder the plugin is popular.
  3. Eclipse color theme / Moonlight UI – woohoo, we all like things that look pretty, don’t we?
  4. Maven integration – Maven is used by 64% of Java developers, so perhaps it could be added into the bundle?
  5. Gradle IDE pack – Gradle might very well rule the world eventually. Nice to know it gets traction and the tooling catches up.
  6. Android development tools – Eclipse is still the official IDE for Android development.
  7. PyDev – Python is flexible, dynamic and installed everywhere by default.
  8. Spring Tool Suite (STS) – Spring Framework is an umbrella project for tons of useful libraries and making your IDE aware of them is a smart step–you’ll notice that STS is also in use by 4% of the survey respondents from the introduction, so it’s not easy to ignore.
  9. Vaadin framework – Vaadin is an interesting web framework with pure Java components, beautiful widgets, flexibility.
  10. JBoss Tools (both Luna and Kepler) – umbrella project to work with all things Red Hat, including JBoss, which is considered by some to be the best Java Application Server there is.
  11. GlassFish Tools for Luna – Oracle has cut commercial support of the GlassFish, but it still is the Reference Implementation of Java EE server.
  12. EclEmma – a very well-known code coverage tool for Java.
  13. FindBugs – a very popular open source, static code analysis tool.
  14. TestNG – JUnit is certainly used more than TestNG, but it doesn’t mean that other testing frameworks cannot top it in terms of quality, usability or features.
  15. CheckStyle – code quality analysis tool focused on the looks of code. Make your team comply with a chosen code standard and enjoy more readable diffs.
  16. JadClipse – a well-known Java Bytecode decompiler.
  17. JRebel – a developer productivity tool, allows you to view code changes instantly, which enables developers to get more done in the same amount of time. Become 17% more productive immediately. More effective than a double espresso in the morning.

CSRF and Character Encoding Filter in Tomcat 7

In Tomcat 7 there are several interesting filter, which are ready to be used:

There are more out-of-the-box Filter, see FilterBase.

Also check out the Combined Realm org.apache.catalina.realm.LockOutRealm, which can be used to mitigate user password brute force attacks.

February 9, 2015

Enable JMX Remote in Tomcat 7

Introduction

Remote monitoring is essential in a IT infrastructure. Sadly is the standard Java JMX based on port range. Here I will show you to overcome that in Tomcat 7.

Installation

Download catalina-jmx-remote.jar to $CATALINA_HOME/lib

Configuration

Test

Start jvisualvm and connect.

Reference

http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html

http://blog.markshead.com/1129/connecting-visual-vm-to-tomcat-7/

https://www.zabbix.com/documentation/2.0/manual/config/items/itemtypes/jmx_monitoring

http://docs.oracle.com/javase/1.5.0/docs/guide/management/agent.html

Use Log4J in Tomcat 7

Introduction

Tomcat uses default Java SDK Logging, which is in mine opinion, generates default log files that are hard to read and it is also quite silly to reinvent the wheel, because there is already a de-facto standard logging framework - log4j.

So lets replace the default logging framework with log4j in Tomcat.

Configuration

  1. Create and configure Log4J in $CATALINA_HOME/lib/log4j.properties.
  2. Download Log4J 1.2 to $CATALINA_HOME/lib.
  3. Download tomcat-juli-adapters.jar to $CATALINA_HOME/lib.
  4. Download tomcat-juli.jar and RELPACE existing $CATALINA_HOME/bin/tomcat-juli.jar.
  5. Delete $CATALINA_HOME/conf/logging.properties.

$CATALINA_HOME/lib/log4j.properties

I do not find having separate Tomcat log convenient so I let all loggers write to the same file.

LDAP Authentication in Tomcat 7

Introduction Tomcat Configuration

Before starting we need to understand the Tomcat configuration (Context) hierarchy. Tomcat configuration can be placed in three places.


    - In $CATALINA_BASE/conf/server.xml.

    - In application /META-INF/context.xml.
    
    - In $CATALINA_BASE/conf/[enginename]/[hostname]/[appname].xml. The default 
    enginename is Catalina. The default hostname is localhost. Which resolves 
    above path to $CATALINA_BASE/conf/Catalina/localhost/[appname].xml.

    [http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Defining_a_context]

The recommended alternative is the last, externally the application, but not intrusive the Tomcat server.

This seperation makes also automated configuration more easily:

  1. One package with standardised Tomcat configuration.
  2. And another package for each application and their seperated configuration.

Next step is to do the actual Authentication configuration which is done by a Realm component.


    "A Catalina container (Engine, Host, or Context) may contain no more than ONE Realm element" 

    [http://tomcat.apache.org/tomcat-7.0-doc/config/realm.html#Introduction]

So a Realm is kind of like a singletone, but it also have scoope, depending where we place it.


    - Inside an  element - This Realm will be shared across ALL web 
    applications on ALL virtual hosts, UNLESS it is overridden by a Realm element 
    nested inside a subordinate  or  element.
    
    - Inside a  element - This Realm will be shared across ALL web 
    applications for THIS virtual host, UNLESS it is overridden by a Realm element 
    nested inside a subordinate  element.
    
    - Inside a  element - This Realm will be used ONLY for THIS web application.

    [http://tomcat.apache.org/tomcat-7.0-doc/realm-howto.html#Configuring_a_Realm]

Configuration

Tomcat comes with several authentication modules (Realm) out of the boxes. Here we will use the LDAP authentication module org.apache.catalina.realm.JNDIRealm.

JNDI Directory Realm - org.apache.catalina.realm.JNDIRealm

JNDIRealm

users.ldif

Test

To test this we create a simple web application.

example-ldap.war/index.jsp

example-ldap.war/WEB-INF/web.xml

February 8, 2015

Using LDAP as Address Book in Thunderbird

In my previous blog I described how to setup OpenLDAP. In this blog I will describe how to use it as address book in thunderbird.

  1. Open Preferences by clicking on menu Edit -> Preferences.
  2. Select Composition and Addressing. Click on Edit Directories...
  3. From the LDAP Directory Servers window click Add and enter you LDAP.
  4. Click OK on all open windows. From main thunderbird window click on tool button Address Book.
  5. Select you LDAP server and search in the quick search tool bar textfield and press Enter.

Understanding LDAP and LDAP Authentication

Introduction

If you let an IT administrator pick a persistence storage technique, he would probably choose a LDAP directory. But if you asked a system developer, then he would probably choose a RDBMS (Relational Database Management System). Why is that? I would guess because of history.

So is the two different technique so much different? They both stores data. Right. But they also have some key differences.

  • LDAP stores data in a tree (hierarchical database) and RDBMS stores data in tables with relationship between them via primary keys and foreign keys.
  • LDAP uses path (Distinguished Name, DN) to make entries unique. RDBMS uses primary keys in tables.
  • In a RDBMS you design your tables and columns. In LDAP you pick from predefined object class (RDBMS table), with predefined attributes (RDBMS column).
  • In RDBMS you define the column data types, but in LDAP everything are strings.

So by that said lets start to discover LDAP, with OpenLDAP.

Installation

openldap A package containing the libraries necessary to run the OpenLDAP server and client applications.

openldap-clients A package containing the command-line utilities for viewing and modifying directories on an LDAP server.

openldap-servers A package containing both the services and utilities to configure and run an LDAP server. This includes the Standalone LDAP Daemon, slapd.

compat-openldap A package containing the OpenLDAP compatibility libraries.

Choosing a Suffix

The LDAP suffix is the global LDAP name space or the toop root of you LDAP tree.

There are two standard approaches:

  1. The X.500 naming model, which is geographically and organizationally based.
    Example: o=Magnus K Karlsson,l=Stockholm,s=Stockholm,c=SE
  2. The Internet domain naming model, i.e. the organizations's DNS domain.
    Example: dc=magnuskkarlsson,dc=com

Abbreviation:
dc, Domain Component
dn, Distinguished Name
cn, Common Name
sn, Surname (family name/last name)
uid, User ID
o, Organization
ou, Organization Unit
l, Location
s, State
c, Country
RDN, Relative Distinuished Name
OID, Object Identifier
DIT, Directory Information Tree
LDIF, LDAP Data Interchange Format

The preferred method is the organizations's DNS domain.

Password Storing Policy

Storing password securily is importand. The default way in OpenLDAP is SSHA-1, in RHEL 6 it is SSHA-512 with 8-bit salt, see crypt(3).


NOTES
   Glibc Notes
       The glibc2 version of this function supports additional encryption 
       algorithms.

       If salt is a character string starting with the characters "$id$" 
       followed by a string terminated by "$":

              $id$salt$encrypted

       then instead of using the DES machine, id identifies the encryption 
       method used and this then determines how the rest of the password string 
       is interpreted.  The following values of id are supported:

              ID  | Method
              ---------------------------------------------------------
              1   | MD5
              2a  | Blowfish (not in mainline glibc; added in some
                  | Linux distributions)
              5   | SHA-256 (since glibc 2.7)
              6   | SHA-512 (since glibc 2.7)

       So $5$salt$encrypted is an SHA-256 encoded password and $6$salt$encrypted 
       is an SHA-512 encoded one.

       "salt" stands for the up to 16 characters following "$id$" in the salt. 
       The encrypted part of the password string is the actual computed password.  
       The size of this string is fixed:

       MD5     | 22 characters
       SHA-256 | 43 characters
       SHA-512 | 86 characters

       The characters in "salt" and "encrypted" are drawn from the set 
       [a–zA–Z0–9./]. In the MD5 and SHA implementations the entire key is 
       significant (instead of only the first 8 bytes in DES).

The slappasswd tool can be given options to use crypt(3), see slappasswd(8).


       -c crypt-salt-format
              Specify the format of the salt passed to crypt(3) when generating 
              {CRYPT} passwords.  This string needs to be in sprintf(3) format 
              and may include one (and only one) %s conversion. This  conversion
              will be substituted with a string of random characters from 
              [A-Za-z0-9./]. For example, ’%.2s’ provides a two character salt 
              and ’$1$%.8s’ tells some versions of crypt(3) to use an MD5 
              algorithm and provides 8 random characters of salt. The default is 
              ’%s’, which provides 31 characters of salt.

The recommended way is to use the strongest option, i.e. SSHA-512 with 16 bit salt.

Configuration

The configuration in newer OpenLDAP versions has been moved from a single configuration file (/etc/openldap/slapd.conf) to a configuration directory (/etc/openldap/slapd.d/), containing several configuration files.

For more novice users is the single configuration file more easier to understand and easier to get an overview of all configuration. Here we will use the single configuration file.

Now we are ready to configure LDAP suffix and username and password for LDAP root user.

Start

Test

To test we can run a simple search.

Another test can be to print all configuration.

GUI Administration Tools

Apache Directory Studio Eclipse-based LDAP tools

For Bug on RHEL 6 and CentOS 6:

See resolution https://issues.apache.org/jira/browse/DIRSTUDIO-999

Designing the Name Space

Designing your LDAP tree or Directory Information Tree (DIT) is an important thing.

Flat Name Space

Example:
uid=john,dc=magnuskkarlsson,dc=com
uid=jim,dc=magnuskkarlsson,dc=com

Advantages:

  • Names do not need to change when job roles change or the organization changes.
  • Simple design avoids need to object categoratization by directory administrators.

Disadvanteges:

  • Hard to partition the directory later if needed.
  • May be hard to maintain unique DNs.

Deeper Name Space

Example:
uid=peter,ou=Development,ou=People,l=Europe,dc=magnuskkarlsson,dc=com
uid=maria,ou=Sales,ou=People,l=Asia,dc=magnuskkarlsson,dc=com

Advantages:

  • Easier to delegate control to organizational units.
  • May simplify later partitioning of directory service among several servers.

Disadvanteges:

  • Names to tend to change more often: job transfer, organizational changes, etc.
  • May require more work to correctly categorize entries, keep up to date.

Compromise Name Space

Example:
ou=People,dc=magnuskkarlsson,dc=com
ou=Group,dc=magnuskkarlsson,dc=com

base.ldif

OpenLDAP Tool

The openldap-clients packages installes a number of tools, the most common used are:

  • ldapsearch, tool to search the directory.
  • ldapadd and ldapmodify, tool that use LDIF (LDAP Data Interchange Format) files to update the directory.
  • ldapdelete, tool to delete entry.

Common options for these tools are:

  • -H host
  • -x Use simple, not SASL binds (login method)
  • -D dn Bind using dn (username)
  • -W prompt for simple bind password

Example:

Common options for ldapsearch:

  • -b dn Base DN in tree to start search from.

Example:

User Class

User Structural Class

After we have designed and created our LDAP tree structure, it's time to create the actual user entry. But before that we need to decide which base class we want to use for our user entry.

Note that an entry must have one and only one structural object class. Each object class have a defined set of attributes, some mandatory and other optional. You can extend or add one or more addition auxiliary object classes.

The two most frequently used structural classes are:

  • account Is useful if you are only using LDAP as a NIS (Network Information Service) replacement.
  • inetOrgPerson Is best if you are also using LDAP to provide contact information.

User Auxiliary Classes

If you are planning to use the LDAP directory for RHEL authentication, you need to add the following auxiliary classes.

posixAccount represents a line from /etc/passwd.

shadowAccount represents a line from /etc/shadow.

Group Structural Class

To complete a UNIX user/authentication you also needs groups.

posixGroup represents a line from /etc/group.

student.ldif

Access Control Instructions, ACI

Writing ACI is out of the scoop of this blog, so comment out every ACI in /etc/openldap/slapd.conf, which will give everyone read, but restricts updates to rootdn.

Installation Apache Web Server (httpd) and LDAP authentication

Lets test our new LDAP directory, by configure LDAP authentication against httpd manual pages.

The httpd ldap module is alreaddy by default installed.

Lets configure httpd-manual authentication.

Restart httpd and test.

RHEL 6 LDAP Authentication Migrations Tools

Create LDIF export of local user and its password.

Create LDIF export of local groups.

To export and import everything.

References

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/ch-Directory_Servers.html

https://help.ubuntu.com/lts/serverguide/openldap-server.html

http://wiki.openiam.com/pages/viewpage.action?pageId=7635198

LDAP Schemas

Default installed LDAP schemas.

/etc/openldap/schema/cosine.schema


objectclass ( 0.9.2342.19200300.100.4.5 NAME 'account'
        SUP top STRUCTURAL
        MUST userid
        MAY ( description $ seeAlso $ localityName $
                organizationName $ organizationalUnitName $ host )
        )

/etc/openldap/schema/nis.schema


objectclass ( 1.3.6.1.1.1.2.0 NAME 'posixAccount'
        DESC 'Abstraction of an account with POSIX attributes'
        SUP top AUXILIARY
        MUST ( cn $ uid $ uidNumber $ gidNumber $ homeDirectory )
        MAY ( userPassword $ loginShell $ gecos $ description ) )

objectclass ( 1.3.6.1.1.1.2.1 NAME 'shadowAccount'
        DESC 'Additional attributes for shadow passwords'
        SUP top AUXILIARY
        MUST uid
        MAY ( userPassword $ shadowLastChange $ shadowMin $
              shadowMax $ shadowWarning $ shadowInactive $
              shadowExpire $ shadowFlag $ description ) )

objectclass ( 1.3.6.1.1.1.2.2 NAME 'posixGroup'
        DESC 'Abstraction of a group of accounts'
        SUP top STRUCTURAL
        MUST ( cn $ gidNumber )
        MAY ( userPassword $ memberUid $ description ) )

/etc/openldap/schema/core.schema


objectclass ( 2.5.6.6 NAME 'person'
        DESC 'RFC2256: a person'
        SUP top STRUCTURAL
        MUST ( sn $ cn )
        MAY ( userPassword $ telephoneNumber $ seeAlso $ description ) )

objectclass ( 2.5.6.7 NAME 'organizationalPerson'
        DESC 'RFC2256: an organizational person'
        SUP person STRUCTURAL
        MAY ( title $ x121Address $ registeredAddress $ destinationIndicator $
                preferredDeliveryMethod $ telexNumber $ teletexTerminalIdentifier $
                telephoneNumber $ internationaliSDNNumber $ 
                facsimileTelephoneNumber $ street $ postOfficeBox $ postalCode $
                postalAddress $ physicalDeliveryOfficeName $ ou $ st $ l ) )

/etc/openldap/schema/inetorgperson.schema


objectclass     ( 2.16.840.1.113730.3.2.2
    NAME 'inetOrgPerson'
        DESC 'RFC2798: Internet Organizational Person'
    SUP organizationalPerson
    STRUCTURAL
        MAY (
                audio $ businessCategory $ carLicense $ departmentNumber $
                displayName $ employeeNumber $ employeeType $ givenName $
                homePhone $ homePostalAddress $ initials $ jpegPhoto $
                labeledURI $ mail $ manager $ mobile $ o $ pager $
                photo $ roomNumber $ secretary $ uid $ userCertificate $
                x500uniqueIdentifier $ preferredLanguage $
                userSMIMECertificate $ userPKCS12 )
        )