May 3, 2018

Getting Started with Elasticsearch, Kibana, X-Pack and SSL/TLS

Introduction Elasticsearch

Why Elasticsearch? "Elasticsearch is a Near Realtime (NRT) search platform." [1]

Elasticsearch stores only JSON Object (Document). A JSON Object is built up by name and value pairs. Value can be one of the following data types:

  • a string
  • a number
  • an object (JSON object)
  • an array
  • a boolean
  • null

Elasticsearch supports more data types, than standard JSON, such as date and geo-point. [2]

Elasticsearch is accessed primarly with JSON over HTTP.

Several of the same documents type are mapped to Types. Which can be thought of as Tables in a RDBM. Different Types are stored in an Indices (plural of Index) which can be thought of a Database.

MySQL => Databases => Tables => Columns/Rows
Elasticsearch => Indices => Types => Documents with Properties

Searching and querying takes the format of: http://localhost:9200/[index]/[type]/[id], i.e. GET localhost:9200/accounts/person/1

An Index can potentially grow and be very large, to help that, Elasticsearch splits Index to multiple Shards. Both Index and Shard are stored in a server Node.

"Sharding is important for two primary reasons:

  • It allows you to horizontally split/scale your content volume
  • It allows you to distribute and parallelize operations across shards (potentially on multiple nodes) thus increasing performance/throughput"

"... Elasticsearch allows you to make one or more copies of your index’s shards into what are called replica shards, or replicas for short."

"Replication is important for two primary reasons:

  • It provides high availability in case a shard/node fails. For this reason, it is important to note that a replica shard is never allocated on the same node as the original/primary shard that it was copied from.
  • It allows you to scale out your search volume/throughput since searches can be executed on all replicas in parallel."

Multiple Nodes forms a Cluster. When you first start Elasticsearch, you start a Node in a Cluster called "elasticsearch", i.e. you have created a Cluster with one Node.

Reference

Elasticsearch

Download


$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.tar.gz
$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.tar.gz.sha512
$ shasum -a 512 -c elasticsearch-6.2.4.tar.gz.sha512 
elasticsearch-6.2.4.tar.gz: OK

ZIP Installation


$ tar -xzf elasticsearch-6.2.4.tar.gz
$ cd elasticsearch-6.2.4/ 

Run Interactively


$ bin/elasticsearch

Test


$ curl http://localhost:9200/
{
  "name" : "hCDxdzG",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "npmGFI2nRcmXke0YLW9tXQ",
  "version" : {
    "number" : "6.2.4",
    "build_hash" : "ccec39f",
    "build_date" : "2018-04-12T20:37:28.497551Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Or open browser 'http://localhost:9200/'.

Reference

Kibana

Download


$ wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-linux-x86_64.tar.gz
$ wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-linux-x86_64.tar.gz.sha512
$ shasum -a 512 -c kibana-6.2.4-linux-x86_64.tar.gz.sha512 
kibana-6.2.4-linux-x86_64.tar.gz: OK

ZIP Installation


$ tar -xzf kibana-6.2.4-linux-x86_64.tar.gz
$ cd kibana-6.2.4-linux-x86_64/ 

Run Interactively


$ bin/kibana

Test

Or open browser 'http://localhost:5601/'.

Reference

X-Pack

"X-Pack is an Elastic Stack extension that bundles security (*), alerting, monitoring, reporting, and graph capabilities into one easy-to-install package."

(*) Authentication, Authorization and Audit Log.

Without X-Pack, Elasticsearch is wide open and everyone can read everything.

Download


$ wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.2.4.zip
$ wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.2.4.zip.sha512
$ shasum -a 512 -c x-pack-6.2.4.zip.sha512
x-pack-6.2.4.zip: OK

X-Pack Elasticsearch

Installation


$ bin/elasticsearch-plugin install [x-pack|file:///path/to/file/x-pack-6.2.4.zip]

$ bin/elasticsearch-plugin install file:///home/magnus/bin/x-pack-6.2.4.zip
-> Downloading file:///home/magnus/bin/x-pack-6.2.4.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \\.\pipe\* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@        WARNING: plugin forks a native controller        @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.

Continue with installation? [y/N]y
Elasticsearch keystore is required by plugin [x-pack-security], creating...
-> Installed x-pack with: x-pack-monitoring,x-pack-ml,x-pack-logstash,x-pack-graph,x-pack-core,x-pack-upgrade,x-pack-deprecation,x-pack-security,x-pack-watcher

Then restart elasticsearch and continue with changing admin passwords.


$ bin/x-pack/setup-passwords [auto|interactive]

$ bin/x-pack/setup-passwords auto
Initiating the setup of passwords for reserved users elastic,kibana,logstash_system.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y

Changed password for user kibana
PASSWORD kibana = PFUPZoLlTdIqECtoS0Qs

Changed password for user logstash_system
PASSWORD logstash_system = sw9XNc2fDskZstgiHdaJ

Changed password for user elastic
PASSWORD elastic = 8fAYzzJ2ZlGxdynE2zqf

Reference

X-Pack Kibana

Installation


$ bin/kibana-plugin install [x-pack|file:///path/to/file/x-pack-6.2.4.zip]

$ bin/kibana-plugin install file:///home/magnus/bin/x-pack-6.2.4.zip
Attempting to transfer from file:///home/magnus/bin/x-pack-6.2.4.zip
Transferring 309419696 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation complete

Configuration


$ vi config/kibana.yml
...
elasticsearch.username: "kibana"
elasticsearch.password: "PFUPZoLlTdIqECtoS0Qs"
...

Restart Kibana and open browser http://localhost:5601


Username: elastic
Password: 8fAYzzJ2ZlGxdynE2zqf 

Using Kibana

Open http://localhost:5601 in your web browser and select DevTools. Now we will use examples from

https://www.elastic.co/pdf/getting-started-webinar-dev-console-commands.pdf

SSL/TLS Elasticsearch

Encrypting HTTP Client Communications: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/configuring-tls.html#tls-http

Create a root, intermediate and localhost certificate according to http://magnus-k-karlsson.blogspot.se/2016/09/openssl-certificate-authority-ca.html

Server Certificate:

  • /home/magnus/bin/ca/intermediate/certs/localhost.cert.pem
  • /home/magnus/bin/ca/intermediate/private/localhost.key.pem-NOPWD

Server Certificate Chain:

  • /home/magnus/bin/ca/intermediate/certs/intermediate.cert.pem
  • /home/magnus/bin/ca/certs/ca.cert.pem

Elasticsearch do not have a seperate settings for server certificate chain, so you need to add the server certificate chain to the server certificate pem file.


$ cp localhost.cert.pem localhost.cert.pem+ca-chain.cert.pem
$ cat intermediate.cert.pem >> localhost.cert.pem+ca-chain.cert.pem
$ cat ca.cert.pem >> localhost.cert.pem+ca-chain.cert.pem

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key:  certs/localhost.key.pem-NOPWD
xpack.security.http.ssl.certificate: certs/localhost.cert.pem+ca-chain.cert.pem

A note about settings 'xpack.security.http.ssl.certificate_authorities', which one might think is server certificate chain, but is NOT. It is for client certificate authentication and is the trusted client certificate CA that are allowed to login. To enable two way SSL you also need to set 'xpack.ssl.client_authentication: required'.

Elasticsearch also supports PKCS#12 format, to use it.


$ cp intermediate.cert.pem ca-chain.cert.pem
$ cat ca.cert.pem >> ca-chain.cert.pem
openssl pkcs12 -export -out localhost.p12 -inkey localhost.key.pem-NOPWD -in localhost.cert.pem -certfile ca-chain.cert.pem

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/localhost.p12

Created elasticsearch keystore in /home/magnus/bin/elasticsearch-6.2.4/config


$ bin/elasticsearch-keystore create

Enter value for xpack.security.http.ssl.keystore.secure_password:


$ bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password

This setups transport crypto (HTTPS) and if you want client certificate authentication also, you need to set the truststore property. But using PKCS#12 format for client certificate CA is not a smooth way, specially when client certificate have a different CA than the server certificate. So I would recommend using the PEM settings.

To setup transport crypto for the internal traffic inside elasticsearch nodes, see Encrypting Communications Between Nodes in a Cluster.

To verify your settings you can either open 'localhost:9200' in a web browser and or use 'openssl s_client', to fully verify your SSL/TLS settings.


$ openssl s_client -connect localhost:9200 -showcerts

NOTE When enable SSL/TLS is HTTP disabled.

SSL/TLS Hardening Elasticsearch

The only secure SSL protocol version is TLSv1.2


xpack.ssl.supported_protocols: TLSv1.2

To hardening you cipher suite you need to install Java Cryptography Extension (JCE), if you are running OpenJDK it is already installed and if you are running Oracle JDK, you need to install it.

Here we will use OWASP TLS/SSL Cheat Sheet

https://www.owasp.org/index.php/Transport_Layer_Protection_Cheat_Sheet#Rule_-_Only_Support_Strong_Cryptographic_Ciphers


$ openssl ciphers -v "EDH+aRSA+AESGCM:EDH+aRSA+AES:EECDH+aRSA+AESGCM:EECDH+aRSA+AES:-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:RSA+AESGCM:RSA+AES+SHA256:RSA+AES+SHA:DES-CBC3-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA"
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD

ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES256-SHA    SSLv3 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA1
ECDHE-RSA-AES128-SHA    SSLv3 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA1
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1
DES-CBC3-SHA            SSLv3 Kx=RSA      Au=RSA  Enc=3DES(168) Mac=SHA1
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1

Then we filter out:

  • only TLSv1.2 ciphers
  • only encryption AESGCM
  • only key exchange DH or ECDH

Which leaves us with only these four ciphers


DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD

ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD

Now we need to map these ciphers to Java: https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider


TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

xpack.ssl.cipher_suites: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Then restart and verify


$ openssl s_client -connect localhost:9200 -showcerts
...
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : DHE-RSA-AES256-GCM-SHA384
...

Reference

SSL/TLS Kibana

When setting up SSL/TLS for Kibana you first need to set https in the URL to Elasticsearch.


elasticsearch.url: "https://localhost:9200"

Then we need to add a truststore in Kibanan, so it trust the Elasticsearch SSL/TLS. A few notes about the certificate management in Kibana:

  • All files needs to be inside kibana directory, due to X-Pack Java Security Manager.
  • You need to use absulte path, compared with elasticsearch.

elasticsearch.ssl.certificateAuthorities: [ "/home/magnus/bin/kibana-6.2.4-linux-x86_64/config/certs/intermediate.cert.pem", "/home/magnus/bin/kibana-6.2.4-linux-x86_64/config/certs/ca.cert.pem" ]

Then restart Kibana and test that you can access http://localhost:5601.


Username: elastic
Password: 8fAYzzJ2ZlGxdynE2zqf 

When OK, then lets continue with setting up SSL/TLS for Kibana.


server.ssl.enabled: true
server.ssl.certificate: /home/magnus/bin/kibana-6.2.4-linux-x86_64/config/certs/localhost+ca-chain.cert.pem
server.ssl.key: /home/magnus/bin/kibana-6.2.4-linux-x86_64/config/certs/localhost.key.pem-NOPWD

And finally test it: https://localhost:5601/

Reference

No comments: