I'm very new here and to the ELK stack in general. I setup an Ubuntu VM in my Azure resource group and installed the latest Elasticsearch,LogStash, and Kibana. I have basic user authentication setup for kibana, and no SSL in http or transport. Essentially my goal is to have 3 of my production VMS sending different log sources to this VM's logstash, process, and display in the dashboard. I wanted to started out simple with winlogbeat, but I've been running into a memory issue I believe? When I just have Elasticsearch and kibana running, it works fine...but when I start up logstash, it practically crashes the whole system, and my elasticsearch ends up exiting because it gets killed by OOM. I'm not sure what steps I should take to further debug this? The VM has 4GB of RAM, which should be plenty for this application no? I'm sorry I'm not really sure what information I need to give for y'all to be able to help me out, but I'll be glad to provide anything you need.
Hello Anthony,
4GB seems a bit low for your setup:
By default, Logstash always takes 1GB, but you can force it to a certain value by using -Xms and -Xmx parameters in the jvm.options.
Elasticsearch detects the required memory dynamically based on your memory and the roles assigned, but you can force it to a certain value by using -Xms and -Xmx parameters in the jvm.options.
Kibana works the same as Elasticsearch and scales with the available memory, but can be limited using --max-old-space-size in the node.options.
So, what you can try to do is using the above settings to limit the memory usage and see if the processes are running, but I would say that you need at least 5GB (2GB Elasticsearch, 1GB Logstash, 1GB Kibana, 1GB OS) of RAM to have it running.
Best regards
Wolfram
Also you don't need LS for Winlogbeat, and can limit LS to 512 MB.
Thanks for your reply! I did a combination of that...I expanding to 8GB of RAM (next available VM size in azure) and I also created an .options file in the jvm.options.d folder as elasticsearch recommends. All that stuff seems to be coming through smoothly! I'm not sure if I did the setup wrong with winlogbeat now...but its like the original setup commands never happened upon first startup. I believe I just did the MSI install on windows. But no template was imported to elasticsearch, and no dashboards were imported either. I get an error when trying to import the dashboards. Should I reinstall it a different way on my windows VMS?
Very true...I only went with LogStash because in the end I will be using FileBeat, WinLogBeat, and another one for SQL on 3 different VMS. Thought it would just be better to route everything through logstash. If that is not the case I'm willing to scrap that and do the ingestion nodes directly through elasticsearch/kibana.
I get an error when trying to import the dashboards. Should I reinstall it a different way on my windows VMS?
Which error do you get?
I get
Exiting: Error connecting to Kibana: fail to get the Kibana version: fail to parse kibana version(): passed version is not semvar:
When I visit http /api/status in my browser on the box I see
{"status":{"overall":{"level":"available"}}}
Can you please check which versions of the products you use? Maybe there is an incompatibility between the Kibana and the Beat version: Support Matrix | Elastic
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.