Syslog client in Java that implements RELP, TCP, SSL/TLS, Caching and Resending

In one of my previous project we had the need of logging message in a secure way, we also needed to be sure that each message was really transmitted and if not sent successfully, should that really be detected.

When I started I thought there must be someone out there that have solved something similar or there exist a product that did this. Be after searching the web I came up with only this:

Log4J do support Syslog (org.apache.log4j.net.SyslogAppender) but is based on java.net.DatagramSocket which uses UDP, which is not a reliable transport protocol. I also looked at other common logging libraries, but they also do not support a TCP based Syslog Appender.

So I started of extended existing SocketAppender to talk Syslog instead. This worked, but then I realized that just having a client talking TCP does not make it reliable. The java.net.Socket.getOutputStream().write() will not throw Exception even if you stop the Syslog after successfully opening a Socket. Socket write operation is not a blocking operation and will not wait for any success message. You might now argue that a socket.getOutputStream().flush() will solve this problem, but as the Javadoc API points out, the flush operation does not guarantee that actual operating system socket will be flushed. “... it does not guarantee that they are actually written to a physical device such as a disk drive. ” [http://docs.oracle.com/javase/7/docs/api/java/io/OutputStream.html#flush%28%29]

So this was the background when I started to look at RELP, Reliable Event Logging Protocol [http://www.librelp.com/relp.html. Using the RELP protocol on top of the Syslog protocol really seemed to solve my problems. The only problem was that I could not find any Java implementation. [http://lists.adiscon.net/pipermail/relp/2010-October/000026.html] The closest project I could find was http://syslog4j.org/ but it does not support RELP.

After happily got my first Syslog Appender working using TCP and RELP. The next problem occurred when we also needed to protect the transport data. Fine I thought, just change my implementation to use a javax.net.ssl.SSLSocket instead of java.net.Socket. But the problem is the current librelp implementation does not support SSL/TLS. “The RELP transport can currently not be protected by TLS.” [http://www.rsyslog.com/doc/rsyslog_tls.html] And the only solution provided was to use a stunnel. Which works.

The last problem to solve was to make my Syslog TCP RELP Appender cache instead of sending Exception up, when communication fails. This was quite straightforward and the only challenge was to make my implementation threadsafe.

There has been discussion of making the solution open source, but the main problem right now is the lack of time.