Spring Boot, Logback and Logstash

When working in a Microservices architecture where you have many services all doing small tasks and features, it can be difficult to troubleshoot and debug in upper environments.  This is why logging and especially centralized logging becomes critical.

ELK

There are several frameworks and systems to make this work, but the ELK stack is a very common approach.  ELK stands for Elasticsearch, Logstash and Kibana These tools are the backbone of our infrastructure

• Logstash – a powerful filtering, mapping and data transformation layer for our logs
• Elasticsearch – a popular NoSQL database that provides a powerful searching engine on top of it
• Kibana – a front end and visualization tool for viewing your transformed logs

Docker

I’m not going to go into too much here, but Docker and Microservices go together very well.  And when running a Java service in Docker, having a log shipper like Filebeats seems to break the idea of single responsibility of the container.  In the specific instance I recently ran across, my workloads were deployed in AWS’ Fargate which doesn’t give us access to the underlying host and any of the powerful log drivers built into AWS’ EC2 deployment type in ECS.

Logback

Enter Logback.  If you’ve done Spring Boot, you’ve probably used it before (it ships as default).   Logback is an implementation of the SLF4J and with some basic configuration makes logging in your application super simple

All I’ve done is defined a Logging instance and then used it to log one info message.  Super simple.  Underneath this is the Logback configuration.  There are hundreds of articles that articulate how to configure Logback for writing to console, file and a bunch of different appenders.  The key to understand is that you have Appenders and Encoders.  You specify the where (Appender) and then the how/format in the Encoder.

Back to the original problem, we have these services, running in containers and don’t want to have a separate process off service shipping logs.   Enter a TCP Appender.  So instead of writing to file or console, Logback is writing to a Ring Buffer that handles the shipping of the written logs off to a TCP endpoint of our choosing.  And it just so happens, that Logstash can listen on a TCP endpoint for such data.

The Setup

Your logback-spring.xml

As you can see, I’ve defined a LogstashTcpSocketAppender and then provided a LogstashEncoder.   This tells Logback how and what to log.  The definition of the Destination tag lets me point to a specific location (Logstash).  For more detail on the power of this library, check Logstash Encoder

Logstash configuration for example purposes looks like this

Again, very simple example but it takes input on port 5000 and dumps the output to the console.  What you’ll find is that the Logback Encoder goes on ahead and converts the output to JSON so that you have less grok’ing and processing to do with unstructured data and more working with your structured JSON payloads

Wrap Up

This was a really small example of how to get started with with centralized logging in Spring Boot using ELK and Docker.  Hopefully this has been helpful!