message:
[2022-09-12T11:02:46,381][WARN ][logstash.outputs.elasticsearch][main][2a206be8e9b0598adfe625bef432d5a7f49b90230ff74f12fb3baf9c5024173f] Could not index event to Elasticsearch. {:status=>400, :action=
cat 02-beats-input.conf
input {
beats {
port => 5044
}
}
cat 30-elasticsearch-output.conf
output {
if [@metadata][pipeline] {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
pipeline => "%{[@metadata][pipeline]}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
}
Can anyone share your knowledge on my above post please. so it will be useful for me..
Welcome to our community! 
Please share the entire error.
Thanks for your reply
ELK Version :
{
"name" : "elk.hyperbig.com",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "h4hGPTKrTxmbZOuSD8C92w",
"version" : {
"number" : "7.17.6",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "f65e9d338dc1d07b642e14a27f338990148ee5b6",
"build_date" : "2022-08-23T11:08:48.893373482Z",
"build_snapshot" : false,
"lucene_version" : "8.11.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Logstash Version
Using bundled JDK: /usr/share/logstash/jdk
logstash 7.17.6
root@elk:/etc/logstash/conf.d# cat 02-beats-input.conf
input {
beats {
port => 5044
}
}
root@elk:/etc/logstash/conf.d# cat 30-elasticsearch-output.conf
output {
if [@metadata][pipeline] {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
pipeline => "%{[@metadata][pipeline]}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
}
root@elk:/etc/logstash/conf.d#
I m getting logs like this
message:
[2022-09-13T07:53:30,403][WARN ][logstash.outputs.elasticsearch][main][2a206be8e9b0598adfe625bef432d5a7f49b90230ff74f12fb3baf9c5024173f] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-7.15.1-2022.09.13", :routing=>nil, :pipeline=>"filebeat-7.15.1-apache-access-pipeline"}, {"agent"=>{"hostname"=>"target.server", "type"=>"filebeat", "name"=>"target.server", "id"=>"767014ec-beb9-4fed-99ab-e62af52a8336", "version"=>"7.15.1", "ephemeral_id"=>"34a50a37-38f0-40bf-854c-332353815e22"}, "log"=>{"file"=>{"path"=>"/var/log/sysadmin.requests.log"}, "offset"=>131684}, "host"=>{"hostname"=>"target.server", "architecture"=>"x86_64", "name"=>"target.server", "containerized"=>false, "id"=>"a0d63ac56c5f4acc84a3c4d4d2b892e4", "mac"=>
Could not index event to Elasticsearch
please give me the solution
I need remote server apache logs and graph has to show in dashboard but not showing that is my problem
2022-09-13T13:47:29.511+0530 WARN beater/filebeat.go:178 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2022-09-13T13:47:29.511+0530 ERROR instance/beat.go:989 Exiting: Index management requested but the Elasticsearch output is not configured/enabled
Exiting: Index management requested but the Elasticsearch output is not configured/enabled
Can you give me any update please
Please give me apache filter file inorder to configure from our end
Please use </> for code. It will be easier to analyze.
Please share your filebeat.yml.
------------------------------ Logstash Output -------------------------------
output.logstash:
The Logstash hosts
hosts: ["103.77.232.85:5044"]
Optional SSL. By default is off.
List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
================================= Processors =================================
processors:
- add_host_metadata:
when.not.contains.tags: forwarded - add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
Did you read my post? 
Sorry i didn't get it what you are asking
<Hi Can i get an update please
[2022-09-13T11:14:25,373][WARN ][logstash.outputs.elasticsearch][main][f5ace9c97b8565874cb73e7b856d6e96dbf0a0d5b5059053b6c522c64890b7f5] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-7.17.6-2022.09.13", :routing=>nil, :pipeline=>"filebeat-7.17.6-apache-access-pipeline"}, {"@timestamp"=>2022-09-13T11:11:11.130Z, "tags"=>["beats_input_codec_plain_applied"], "service"=>{"type"=>"apache"}, "input"=>{"type"=>"log"}, "fileset"=>{"name"=>"access"}, "type"=>"random_logs", "log"=>{"file"=>{"path"=>"/var/log/sysadminaccess.log"}, "offset"=>184186}, "@version"=>"1", "event"=>{"module"=>"apache", "dataset"=>"apache.access"}, "agent"=>{"hostname"=>"target.server", "id"=>"767014ec-beb9-4fed-99ab-e62af52a8336", "type"=>"filebeat", "ephemeral_id"=>"3d97261f-863b-495e-8f7b-94e15204976e", "name"=>"target.server", "version"=>"7.17.6"}, "host"=>{"hostname"=>"target.server", "id"=>"a0d63ac56c5f4acc84a3c4d4d2b892e4", "mac"=>["00:0c:29:5f:1b:d5"], "architecture"=>"x86_64", "os"=>{"kernel"=>"3.10.0-1160.71.1.el7.x86_64", "type"=>"linux", "platform"=>"centos", "family"=>"redhat", "version"=>"7 (Core)", "name"=>"CentOS Linux", "codename"=>"Core"}, "name"=>"target.server", "containerized"=>false, "ip"=>["173.208.192.93", "fe80::3ec8:c6bd:d570:ee56"]}, "ecs"=>{"version"=>"1.12.0"}, "message"=>"201.150.180.250 - - [13/Sep/2022:16:41:03 +0530] "GET / HTTP/1.1" 200 160 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.7 (KHTML, like Gecko) Version/9.1.2 Safari/601.7.7""}], :response=>{"index"=>{"_index"=>"filebeat-7.17.6-2022.09.13", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"pipeline with id [filebeat-7.17.6-apache-access-pipeline] does not exist"}}}}
reason"=>"pipeline with id [filebeat-7.17.6-apache-access-pipeline] does not exist"}}}}
please give me solution for this
/>
Hello,
People in this forum are volunteers, do not keep bumping your post or pinging people, just post your question and wait, people will answer when they have the time to do it, there is no SLA in this forum.
Also, you need to help people help you, do not post your configuration or logs unformatted, it will make things very hard to understand, post your config, select the entire text and click in the </> button, this will correct the format.
Your question is really confusing at this time because you shared a lot of things without any context.
You have issues with both Filebeat and Logstash, what is your current issue now since you probably change somethings?
Please share your entire filebeat.yml file using the format as mentioned above and your logstash configuration using the format, the </> button.
reason"=>"pipeline with id [filebeat-7.17.6-apache-access-pipeline] does not exist"}}}}
Did you run filebeat setup before anything else? If you are going to use a filebeat module you need to run filebeat setup, to do that you will need to temporarily change your filebeat output to elasticsearch, please read this part of documentation.
2 Likes
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.