Hi,
please restart the filebeat and capture the messages logs.
And please confirm still you are getting same cipher suit error??
Regards,
Hi @harshbajaj16,
I have restarted filebeat and /var/log/messages shows below status
2018-07-17T11:37:42.306561+05:30 esbapp systemd[1]: Stopping filebeat...
2018-07-17T11:37:42.315497+05:30 esbapp systemd[1]: Stopped filebeat.
2018-07-17T11:37:42.326265+05:30 esbapp systemd[1]: Started filebeat.
Yes, getting same errors of cipher suit.
https://pastebin.com/raw/HaRKjKzd
Thank you.
HI,
it showing your logstash working properly and listen on 5045 and that cipher logs are debug logs.
Please check your elasticsearh weather its creating index or not.
Also check that your log file mentioned generating logs or not.
Regards,
Hi @harshbajaj16,
Elasticsearch is not creating an index which I am trying to create.
I have checked elasticsearch created index by using the following command
curl -XGET http://x.x.x.x:9200/_cat/indices
Log file path which is mention in logstash.conf file is continuously generating logs. My logstash log is continously showing one error as below , what it meant for?
[2018-07-17T12:28:46,262][DEBUG][logstash.pipeline ] Pushing flush onto pipeline
Hi,
Please confirm below thing,
As you are using stdout { codec => rubydebug } in logstash conf it will show the standard o/p on screen. r u able to see that.
please also let me know how you are starting logstash service.
Is there any reason to use logstash even you can send directly data from filebeat to elasticsearch?
Regards,
1 Like
-
I am not able to see any o/p on terminal.
-
For starting logstash server I uses
systemctl start/restart logstash -
I am using logstash because to create custom index. If will use elasticsearch to sending log then will it use default .filebeat index or will I able to create my own index?
Hi @mamta,
Yes you can create your index with specific name please refer below example for the same.
> # Optional index name. The default is "filebeat" plus date
> # and generates [filebeat-]YYYY.MM.DD keys.
> index: "yourpattern-%{+yyyy.MM.dd}"
Regards,
Hi @harshbajaj16,
I think will have to upgrade filebeat version. Currently, my version is 5.6.8 and in this version, custom index option is not available.
Thank you.
Hi @mamta,
i'm using 5.4.3 and its working fine here. please try and refer filebeat.full.yml file for filebeat configuration reference. it is the reference file only.
Regards,
Hi @harshbajaj16,
I have seen those option available only in filebeat.full.yml. I am not able to see in filebeat.yml
In which file will have to configure custom index?
Thank you.
Hi,
FIlebeat.full.yml is only for reference. you can check and copy the same configuration in your filebeat.yml file.
you need to disable logstash o/p and enable elasticsearch u/p and define your index name and restart the service.
Regards,
Hi @harshbajaj16,
I have copy all the contents of filebeat.full.yml in filebeat.yml file and output with elasticsearch, But still not able to create an index in elasticsearch and on kibana. I am also using metricbeat which is running on port 5044. Where can I set the port for filebeat or it is sending the logs to elasticsearch so no need to set the port for filebeat?
Sorry for all the basics questions.
Thank you so much for all your informative help.
HI @mamta,
You don't need to set port just give the elasticsearch host/ip address in filebeat.yml file and restart the services. It will start sending data to elasticsearch directly.
Regards,
Hi @harshbajaj16,
Thank you so much.
Please forgive me for all my basics questions but I am not able to identify whats wrong with my configuration. I have set elasticsearch as an output and set the IP in filebeat.yml but still not able to create an index on kibana nor elasticsearch showing any index.
Please look into below link of filebeat.yml
Hi @mamta,
That's ok.
You file seems ok and please check filebeat and elasticsearch logs. are u getting any error.
If there is no error then your index should be create.
please confirm below,
- how you are checking elasticsearch index?
- Are you getting metrics in filebeat logs that your have has been harvested etc.?
Regards,
Hi @harshbajaj16,
I have checked elasticsearch and filebeat logs but there is no such errors.
- Command for checking index
curl -XGET http://x.x.x.x:9200/_cat/indices - No
Elasticsearch Logs
https://pastebin.com/raw/gN48N2EU
Filebeat Logs
https://pastebin.com/raw/B3spvj7Z
Thank you.
Hi @mamta,
Could you please delete filebeat registry file from /var/lib/filebeat/ and restart filebeat service as it seems your file had already been harvested and state saved in registry file.
OR
Please try to place new file in filebeat and check in logs that file is harvesting or not.
Please take backup before delete.
Regards,
Hi @harshbajaj16,
Thank you so much for replying.
I have deleted the /var/lib/filebeat/registry file and restarted filebeat.
I have also given a new path of the log file. But still not able to create an index in kibana nor in elasticsearch. I also compared old registry data with newly generated registry data both are same and showing old data.
Hi,
Please share the o/p of below curl command.
curl -X GET "x.x.x.x:9200/_cat/indices"
Regards,
Hi @harshbajaj16,
Thank you.
Output
yellow open metricbeat-6.2.3-2018.07.18 Y5Iqab45Scqw1wcBe-irjw 1 1 396276 0 142.7mb 142.7mb
yellow open metricbeat-6.2.3-2018.07.17 EjJInAvATLan1m04vXKWdw 1 1 391252 0 137.8mb 137.8mb
yellow open metricbeat-6.2.3-2018.07.03 IBLWfk0_T_-JNz6ThmWfRA 1 1 8280 0 2.6mb 2.6mb
yellow open metricbeat-6.2.3-2018.07.15 1b4nOceERyyHW7OWConhzw 1 1 388465 0 141.2mb 141.2mb
yellow open metricbeat-6.2.3-2018.07.12 p-JMoPSLQJWIwF-l6JoxIw 1 1 118012 0 43.4mb 43.4mb
yellow open metricbeat-6.2.3-2018.07.16 DsXl1vuPRvS2wj5YrwNZ8Q 1 1 388384 0 138.2mb 138.2mb
yellow open metricbeat-6.2.3-2018.07.19 YhbCgwz9RC2djb67KebncA 1 1 264823 0 95mb 95mb
yellow open metricbeat-6.2.3-2018.07.14 3Ggcsl_2R9a4is4e-Vgiew 1 1 388649 0 140.4mb 140.4mb
yellow open .kibana VJyAhMwIQnKOGgDR_XFfvA 1 1 152 130 499.8kb 499.8kb
yellow open metricbeat-6.2.3-2018.07.13 3imOBUpxQEqDlR7MnHbyng 1 1 388439 0 140.6mb 140.6mb