May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.145-0400 INFO [license] licenser/es_callback.go:51 Elasticsearch license: Basic
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.145-0400 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.146-0400 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.146-0400 INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.146-0400 INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{filebeat-7.12.1 {now/d}-000001}' as ILM is enabled.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.146-0400 INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'filebeat-7.12.1-*' as ILM is enabled.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.146-0400 INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.12.1 {now/d}-000001} as ILM is enabled.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.146-0400 INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_s
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.147-0400 INFO template/load.go:97 Template filebeat-7.12.1 already exists and will not be overwritten.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.147-0400 INFO [index-management] idxmgmt/std.go:298 Loaded index template.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.148-0400 INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated.
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.158-0400 INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to backoff(elasticsearch(http://192.168.0.3:9200)) established
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.166-0400 ERROR [elasticsearch] elasticsearch/client.go:224 failed to perform any bulk index operations: 500 Internal Server Error: {"error":{"root_cause":[{"type":"illegal_state_except
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.167-0400 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
May 18 10:50:36 debian filebeat[5663]: 2021-05-18T10:50:36.168-0400 INFO [publisher] pipeline/retry.go:223 done
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.800-0400 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: 500 Internal Server Error: {"error":{"root_cause":[{"type":"illegal_state_exception","reason
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.800-0400 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(elasticsearch(http://192.168.0.3:9200))
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.800-0400 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.800-0400 INFO [publisher] pipeline/retry.go:223 done
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.801-0400 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.12.1
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.816-0400 INFO [license] licenser/es_callback.go:51 Elasticsearch license: Basic
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.818-0400 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{filebeat-7.12.1 {now/d}-000001}' as ILM is enabled.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'filebeat-7.12.1-*' as ILM is enabled.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.12.1 {now/d}-000001} as ILM is enabled.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_s
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO template/load.go:97 Template filebeat-7.12.1 already exists and will not be overwritten.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.819-0400 INFO [index-management] idxmgmt/std.go:298 Loaded index template.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.820-0400 INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated.
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.820-0400 INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to backoff(elasticsearch(http://192.168.0.3:9200)) established
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.823-0400 ERROR [elasticsearch] elasticsearch/client.go:224 failed to perform any bulk index operations: 500 Internal Server Error: {"error":{"root_cause":[{"type":"illegal_state_except
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.823-0400 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
May 18 10:50:37 debian filebeat[5663]: 2021-05-18T10:50:37.823-0400 INFO [publisher] pipeline/retry.go:223 done
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.482-0400 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: 500 Internal Server Error: {"error":{"root_cause":[{"type":"illegal_state_exception","reason
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.482-0400 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(elasticsearch(http://192.168.0.3:9200))
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.482-0400 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.482-0400 INFO [publisher] pipeline/retry.go:223 done
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.483-0400 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.12.1
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.498-0400 INFO [license] licenser/es_callback.go:51 Elasticsearch license: Basic
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.498-0400 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{filebeat-7.12.1 {now/d}-000001}' as ILM is enabled.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'filebeat-7.12.1-*' as ILM is enabled.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.12.1 {now/d}-000001} as ILM is enabled.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_s
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO template/load.go:97 Template filebeat-7.12.1 already exists and will not be overwritten.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.499-0400 INFO [index-management] idxmgmt/std.go:298 Loaded index template.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.500-0400 INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated.
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.503-0400 INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to backoff(elasticsearch(http://192.168.0.3:9200)) established
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.517-0400 ERROR [elasticsearch] elasticsearch/client.go:224 failed to perform any bulk index operations: 500 Internal Server Error: {"error":{"root_cause":[{"type":"illegal_state_except
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.517-0400 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
May 18 10:50:39 debian filebeat[5663]: 2021-05-18T10:50:39.517-0400 INFO [publisher] pipeline/retry.go:223 done
Welcome to our community! 
Please don't just post your log with no other information, it's a little rude.
What version of the Elastic Stack are you on?
What is the current state of Elasticsearch?
Hi mark,
This is my first time writing to any forums,so pardon my ettiquette,
So i am using bitnami elastic stack ,and the version is 7.12.1-1
My elastic search is setupusing bitnami vm and my filebeats is installed on another vm with apache installed on it ,and i am trying to forward my logs from apache to kibana ,i can succesfully see kibana i can also see a index filebeat-* created but there is nothing in the index ,and when i checked the journalctl -u filebeat.service i got these logs
Ok, Elasticsearch doesn't look healthy, so you need to check the logs for it and make sure everything is ok.
If you can post them, we can help with that.
this is the current health
curl 192.168.0.3:9200/_cat/health
1621546769 21:39:29 bnCluster yellow 1 1 9 9 0 0 1 0 - 90.0%
and result of GET /_cluster/allocation/explain
{
"index" : "filebeat-7.12.1-2021.05.16-000001",
"shard" : 0,
"primary" : false,
"current_state" : "unassigned",
"unassigned_info" : {
"reason" : "CLUSTER_RECOVERED",
"at" : "2021-05-20T21:28:51.972Z",
"last_allocation_status" : "no_attempt"
},
"can_allocate" : "no",
"allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
"node_allocation_decisions" : [
{
"node_id" : "4uhwxTw2QP-6gNanEk5-3g",
"node_name" : "debian",
"transport_address" : "127.0.0.1:9300",
"node_attributes" : {
"xpack.installed" : "true",
"transform.node" : "true"
},
"node_decision" : "no",
"deciders" : [
{
"decider" : "same_shard",
"decision" : "NO",
"explanation" : "a copy of this shard is already allocated to this node [[filebeat-7.12.1-2021.05.16-000001][0], node[4uhwxTw2QP-6gNanEk5-3g], [P], s[STARTED], a[id=iyK3JBbyQmq6MhrkJtCQ0g]]"
}
]
}
]
}
and the only thing unique in logs i see is Illegal state exception: there are no ingest nodes in this cluster unable to forward request to an ingest node
ok guys stopped and restarted services in order of
elasticsearch
kibana
filebeat
logstash fixed it
but now i have my index when i want to visualize it ,
yellow open filebeat-7.12.1-2021.05.16-000001 AeD0qkH9TmSJRWhkrnNIbg 1 1 0 0 208b 208b
have just started there is not much data in there but i want to be sure its working because,
it says no data recieved
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.