Hi!
I deployed 1 ELK stack on premise.
I have only 1 Elastic node.
I am going to upgrade my license from Free to Platinum (still on premise) and delete the ELK free version cluster. I am going to buy only 1 Elastic node
I wonder if the my Platinum Elastic node goes down. I would like to have Elastic node (free version) as a backup. Will it be possible to do that?
Welcome!
I'm not sure that the sales team is selling one node only licenses. Did you contact them?
You need to contact the sales team, but if I'm not wrong the minimum that you can buy is 5 nodes, I don't think is possible to buy the license for one node only, a single node cluster is not a deployment recommended for production, so it makes no sense for elastic to sell something that is not production ready.
Regardless of the license this is not possible, You cannot have a different cluster to act as a backup for another cluster.
The closer you can get is to have 2 different clusters running and use Cross-Cluster Replication, but in this case you need a license for both clusters.
I contacted the sales team. They offered me a Platinum license for 3 nodes. But I have replied back that I want only one.
The only ways I think to get this is by running on cloud. You can choose there to have only one node.
The cloud is not a solution for me. I must use self-managed stack.
I'm afraid you have no other choice then.
What is your use-case? I'm curious about the "why" you don't want a real production cluster (3 nodes).
The cost of a self managed 3 nodes cluster is too high. And I currently use the one Elastic node and it works fine.
But what is your use case? Logs? Observability? Security?
What you require from the platinum features? You can do a lot with the basic license.
A single node is not a production cluster, it may work in many cases, but it as you use it you can face many issues that you won't be able to solve without adding more nodes.
At my workplace we currently use the basic license.
We deployed the ELK stack but with one Elastic node only. It works fine this way.
We want to use the watcher feature only available with Platinum.
We asked the sales department but the price for the 3 Elastic nodes is too high, therefore we go for only one. Thus I was wondering if it is possible to combine a Elastic Platinum version node to have the watcher feature available with an other Elastic free version node as a backup in case the Platinum version goes down.
Yeah, I don't think Elastic sell license for single-node cluster because single-node cluster are not production ready clusters.
You also do not combine clusters, even paid ones, what you may do is use Cross-Cluster Replication to keep 2 or more clusters in sync, but this is also a paid feature and both cluster needs to be licensed.
But what exactly you want to do with Watcher? My personal opinion is that Watcher alone does not justify a license because its feature can be replicated by custom applications, like python scripts, or third party applications, like ElastAlert2.
The description for Watcher is this:
Watcher is an Elasticsearch feature that you can use to create actions based on conditions, which are periodically evaluated using queries on your data.
This can be easily replicated in many forms, Kibana Alerts, which is also part of the Basic License, is one of them, ElastAlert2 is free third-party tool that also allows you to create alerts based on queries, and you also can build your own code to query ir data and performactions.
Here is my case. When I have a 500 http error in my application, it sends it to the ELK. However, the application sends to ELK another error message (the complete Stacktraces), which arrives right after the http 500 error code in the Kibana time frame.
My goal: Each time I receive the http 500 error, I want to send an email which contains the http 500 error with its metadata, AND all stacktraces that arrived right after and which are related to this particular http 500 error.
My current achievement: I tried with ElastAlert2 but I could only send the http 500 error and its metadata but not the stacktraces that came after, because I couldn't identify the link between the http 500 error and the stacktrace.
My ElastAlert2 script the condition was "When HTTP 500 occurs" -> "Send an email".
As there is no "HTTP 500 error" in stacktraces, my script couldn't catch the stacktraces and send them over email to me.
Could you share 2 typical lines of logs when this is happening?
I'm not sure if Watcher would help you to solve that. Did you try yet?
When I filter HTTP 500 code I have only 3 hits.
Here is what I get when I catch HTTP 500 code, as you can see in the message, the value is related to HTTP protocol and not the stacktrace of the application:
{
"_index": "k8s-test-app-admin-2024.02",
"_id": "x8Aix40BRPoEsDWM",
"_version": 1,
"_score": 1,
"_ignored": [
"message.keyword"
],
"_source": {
"container": {
"image": {
"name": "gitlab.app.002.fr:5005/ln/nto:ln_admin_nto"
},
"id": "b2d2bcc5ab8a9ef64aecbf41a8c758e998217e5c22",
"runtime": "docker"
},
"tags": [
"beats_input_codec_plain_applied"
],
"method": "POST",
"kubernetes": {
"labels": {
"workloadID_ingress-3c032470f6d1a567dcd068ce4889bf3b": "true",
"logme": "true",
"workloadID_ingress-ea457564d91bc9b1d4997822ea1e97ac": "true",
"workloadID_ingress-9de728feae5c3ffa112d6d840a43c4d0": "true"
},
"namespace_labels": {
"cattle_io/creator": "norman"
},
"replicaset": {
"name": "app-test-admin-77bd68f65d"
},
"pod": {
"ip": "10.42.3.30",
"name": "app-test-admin-77bd68f65d-9wsb6"
},
"container": {
"name": "app-test-admin"
},
"deployment": {
"name": "app-test-admin"
},
"namespace": "app-test",
"node": {
"labels": {},
"name": "von-wor-01",
"hostname": "von-wor-01"
}
},
"request_size": "1825",
"request": "/secured/honorary-consul/declaration/17/edit",
"app_access_full_request": "POST /secured/honorary-consul/declaration/17/edit HTTP/1.1",
"host": {
"name": "von-wor-01"
},
"@version": "1",
"client_ip": "0.0.1.187",
"app_access_timestamp": "20/Feb/2024:15:29:15 +0000",
"status_code": "500",
"client_identity": "-",
"user_id": "-",
"@timestamp": "2024-02-20T15:29:15Z",
"ecs": {
"version": "8.0.0"
},
"input": {
"type": "container"
},
"agent": {
"version": "8.0.1",
"name": "von-wor-01",
"type": "filebeat",
"id": "e28625b4-85dc-489b-9ffd-f39d2a4a23f7",
"ephemeral_id": "eea74980-db4e-4ccb-b4e6-e48b4e782d00"
},
"referer": "https://ln-protocole-qual.intranet.ste.fr/secured/honorary-consul/declaration/17/edit",
"http_version": "1.1",
"event": {},
"stream": "stdout",
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0",
"log": {
"file": {
"path": "/var/log/containers/app-test-admin-77bd68f65d-9wsb6_app-test_app-test-admin-b2d2bcc0f92ac283255ab8a9ef5866c83f94764aecbf41a8c758e998217e5c22.log"
},
"offset": 8766689
},
"message": "0.0.1.187 - - [20/Feb/2024:15:29:15 +0000] \"POST /secured/honorary-consul/declaration/17/edit HTTP/1.1\" 500 1825 \"https://ln-protocole-qual.intranet.ste.fr/secured/honorary-consul/declaration/17/edit\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0\""
}
}
But actually if I remove the filter status_code:500, I have 51 hits. Among them there are the logs of the application:
Here is one of the hit log. In this one THERE IS NO CODE HTTP 500, but there is the a part of the stack trace in the value of message key
{
"_index": "k8s-app-test-app-test-admin-2024.02",
"_id": "1sAix40BRPoEE3WhsDWO",
"_version": 1,
"_score": 1,
"_source": {
"container": {
"image": {
"name": "gitlab.app.ste.lu:5005/ind/app:mae_admin_oidc"
},
"id": "b2d2bcc0f92ac283255ab8a9ef5866c83f94764aecbf41a8c758e998217e5c22",
"runtime": "docker"
},
"host": {
"name": "von-wor-01"
},
"@version": "1",
"tags": [
"beats_input_codec_plain_applied",
"_grokparsefailure"
],
"kubernetes": {
"labels": {
"logme": "true",
"workloadID_ingress-3c032470f6d1a567dcd068ce4889bf3b": "true",
"workloadID_ingress-ea457564d91bc9b1d4997822ea1e97ac": "true",
"workloadID_ingress-9de728feae5c3ffa112d6d840a43c4d0": "true"
},
"namespace_labels": {
"cattle_io/creator": "norman"
},
"replicaset": {
"name": "app-test-admin-77bd68f65d"
},
"pod": {
"ip": "10.42.3.30",
"name": "app-test-admin-77bd68f65d-9wsb6"
},
"container": {
"name": "app-test-admin"
},
"deployment": {
"name": "app-test-admin"
},
"namespace": "app-test",
"node": {
"labels": {},
"name": "von-wor-01",
"hostname": "von-wor-01"
}
},
"@timestamp": "2024-02-20T15:29:15.673Z",
"ecs": {
"version": "8.0.0"
},
"input": {
"type": "container"
},
"agent": {
"version": "8.0.1",
"name": "von-wor-01",
"type": "filebeat",
"id": "e28625b4-85dc-489b-9ffd-f39d2a4a23f7",
"ephemeral_id": "eea74980-db4e-4ccb-b4e6-e48b4e782d00"
},
"event": {},
"stream": "stdout",
"log": {
"offset": 8764433,
"file": {
"path": "/var/log/containers/app-test-admin-77bd68f65d-9wsb6_app-test_app-test-admin-b2d2bcc0f92ac283255ab8a9ef5866c83f94764aecbf41a8c758e998217e5c22.log"
}
},
"message": "#8 /var/www/html/vendor/symfony/form/Extension/HttpFoundation/HttpFoundationRequestHandler.php(110): Symfony\\\\Component\\\\Form\\\\Form->submit()"
},
By the way I have just understood that all my stack traces logs are split each time ELK encounters "\n".
So my questions now are different.
As when I look at a status_code:500 log, the value of the key "message" doesn't show any messages related to the stack traces and the logs which are related to the stack traces don't include the status_code:500 key/value pair. The only link that I see is that both type of logs are in the same time frame.
Yes. I think this is the right question to ask.
I think you should look at Manage multiline messages | Filebeat Reference [8.12] | Elastic
I believe that the initial discussion you opened is now "solved" as you got the answer.
I'd suggest that you open a new discussion about this specific problem in Beats using the filebeat tag (and you might have suggestions from the past about it).
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.