Recomendations while installing ELK+ X-Pack

Elastic Stack Elasticsearch
1

Hello,

I've setup an ElasticSearch instance with 1 node + an app once already but now I'm about to setup a cluster.

What I'm thinking about is installing the entire ELK stack + x-pack using a basic Elastic license on ubuntu 18.04 LTS and then snapshotting and duplicating the VM into a cluster.

The reason why I'm posting this in the X-Pack section is because I'm wondering whether or not I need to install X-Pack. I might get a gold or platinum membership in the future and I like to future proof. Appart from the upgrade assistant, and future proofing what's th benefit of installing X-Pack ?

I can simply setup ELK on this base server then install X-Pack later when I need it right ?

Thanks

2

If you install X-Pack you get everything that's listed under the Basic license here - https://www.elastic.co/subscriptions. That includes things like the Monitoring functionality that are super useful, as well as access to the additional APM dashboards.

Then if you do upgrade you don't need to restart nodes to install the plugin, you just upgrade the license :slight_smile:

1 Like
3

Our company will host hundreds of apps in a container cluster (kubernetes) that will connect to an ElasticSearch service.

I don't really know how to design my cluster.

Here's a representation of how I think it could work, master nodes being hosted in docker containers within kubernetes. Application talk to an NGINX proxy that routes their requests to master nodes using a round robin balancer. The master node then routes it's request through nginx to the dedicated server data nodes.

In this design the master nodes are small VMs with small ammount of dedicated resources.
I'm still brainstorming at this point but, in a design like that where do you install tools such as kibana, logstash and x-pack ?
Does such a design makes sense ? My reasoning behind this design is :

  1. Master nodes each require their own servers which are easily virtualized and managed in kubernetes
  2. Data nodes require more processing power so they have dedicated machines
  3. All these machines are in the same datacenter and/or region so the latency will be minimal.

Is it possible to install logstash/kibana to talk directly to data nodes or do they need to interact through the master nodes ?

4

Make sure they have persistent state! Otherwise you may run into problems.

Nope, master only nodes should do nothing other than manage the cluster. Don't send requests to them. If you want nodes for that use client nodes.

1 Like
5

Okay so I did some reading and figured out what I now understand made no sense.

This is how I think it'll work now, everything being ran in stateful containers inside kubernetes. I wasn't super comfortable at first but found a github project that already did all the configs so I'm gonna give it a try.

I think this is how the diagram should look, also I've got no idea where X-Pack has to be setup now. It goes along with kibana right ?

6

In current versions of the Elastic Stack, X-Pack is a plugin that is available for Elasticsearch, Kibana and Logstash.
Generally speaking, you should install the plugin into each of the products that you use.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.