# Apache Access Logs with Filebeats-\>Logstash-\>EL\<-Kibana: Throws errors at default dashboard **URL:** \ **Category:** Logstash\ **Created:** [September 2, 2022, 8:47pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585 "2022-09-02T20:47:34Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![SirStephanikus](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@SirStephanikus](https://discuss.elastic.co/u/SirStephanikus)\ **Post date:** [September 2, 2022, 8:47pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585/1 "2022-09-02T20:47:34Z")
Hello Everyone While I'm trying to deep dive into ELK, I followed a video course that is unfortunately outdated... My ELK version is 8.3.3 However, the lab goal is this: Use filebeat to parse some apache access logs, send it to Logstash ...use Logstash to learn everything about filtering+ECS and from there ship it to Elasticsearch. Then, use filebeats default dashboards in Kibana for some basic visualizations of the apache access logs. Doing so _WITHOUT_ logstash, works fine....but with Logstash, I get a couple of sharding errors when clicking on the default dashbord. ![Bildschirmfoto 2022-09-02 um 22.32.12](https://us1.discourse-cdn.com/elastic/original/3X/d/4/d4792a1336e227b35730790f91b3d1954a9b8b64.jpeg) Error Message -\> Title: 1 of 3 shards failed The data you are seeing might be incomplete or wrong. Error Message Reason: ```auto { "took": 1, "timed_out": false, "_shards": { "total": 3, "successful": 2, "skipped": 2, "failed": 1, "failures": [ { "shard": 0, "index": "filebeat-8.3.3-2017-09-20", "node": "eToI6phpReyc43TQRqyOpg", "reason": { "type": "illegal_argument_exception", "reason": "**Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [user_agent.name] in order to load field data by uninverting the inverted index.** Note that this can use significant memory." } } ] }, "hits": { "total": 0, "max_score": 0, "hits": [] } } ``` There are multiple pop-ups which complain all about the same problem just for dfferent fields. What I did: filebeat setup, yes including all the steps from the manual (multiple times). Except (because it wont work): ```auto filebeat setup --pipelines throws: Exiting: module apache is configured but has no enabled filesets ``` and I did not use the metadata condition to use the ingest pipeline based on this metadata: ```auto output { if [@metadata][pipeline] { elasticsearch { hosts => "https://061ab24010a2482e9d64729fdb0fd93a.us-east-1.aws.found.io:9243" manage_template => false index => "%{[@metadata][beat]}-%{[@metadata][version]}" action => "create" pipeline => "%{[@metadata][pipeline]}" user => "elastic" password => "secret" } } else { elasticsearch { hosts => "https://061ab24010a2482e9d64729fdb0fd93a.us-east-1.aws.found.io:9243" manage_template => false index => "%{[@metadata][beat]}-%{[@metadata][version]}" action => "create" user => "elastic" password => "secret" } } } ``` Deleting and re-creating (automatically) the indices. What I guess: I guess the mapping of the Filebeat ingest pipeline and the dashboard fields do not match when uing my Logstash filter (which is based on an outdated video course...). As far as I understand Filebeat on its own set everything up...even the Index and mapping part...but Logstash ship the data with different datatypes (like text only). Question: Is there a way to fix it??? Perhaps a working Logstash filter? ```auto input { beats { port => 5044 host => "0.0.0.0" } } filter { if [event][dataset] != "apache.access" { drop { } } grok { match => { "[event][original]" => '%{HTTPD_COMBINEDLOG}' } } date { match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"] } grok { "match" => { "[source][address]" => "^(%{IP:[source][ip]}|%{HOSTNAME:[source][domain]})" } } #if "_grokparsefailure" in [tags] { # drop { } #} mutate { remove_field => ["log", "input", "service", "host", "ecs", "@version"] } mutate { add_field => { "[event][created]" => "%{@timestamp}" } } useragent { source => "[user_agent][original]" target => "[user_agent]" } geoip { source => "[source][ip]" target => "[source][geo]" } } output { elasticsearch { hosts => "localhost" manage_template => false index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY-MM-dd}" } stdout { codec => rubydebug { metadata => false } } } ``` ---
**Author:** ![SirStephanikus](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@SirStephanikus](https://discuss.elastic.co/u/SirStephanikus)\ **Post date:** [September 6, 2022, 6:22pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585/2 "2022-09-06T18:22:41Z")
Update: I've have this issue now with the filebeat system module too, which send its stream to logstash. The pipeline configuration is plain simple: ```auto input { beats { port => 5044 host => "0.0.0.0" } } output { if [@metadata][pipeline] { elasticsearch { hosts => "localhost:9200" manage_template => false index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY-MM-dd}" action => "create" pipeline => "%{[@metadata][pipeline]}" } stdout { codec => rubydebug { metadata => true } } } } ``` And again, the default dashboards thow errors: ```auto "reason": { "type": "illegal_argument_exception", "reason": "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [host.hostname] in order to load field data by uninverting the inverted index. Note that this can use significant memory." } ``` What must I do, to fix this issue? I doubt that the solution is to edit the default dashboards. Shouldn't the filebeat pipeline be like "fire and forget" ? The mappings of the filebeat index for the hostname looks like this: ```auto "hostname": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } ``` ---
**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Post date:** [September 6, 2022, 6:36pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585/3 "2022-09-06T18:36:58Z")
The easiest fix is don't use logstash. Can you just let filebeat send direct to elasticsearch? If not, what I've done is setup a "sham" filebeat that appears to send to elasticsearch (I just use my work station). It's only purpose is to create the template, data streams and Kibana dashboards. You will need to do this for each version of beats you're going to use. Then your logstash logic to call the appropriate ingest pipeline should work. In a brief read, it looks similar to mine. ---
**Author:** ![SirStephanikus](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@SirStephanikus](https://discuss.elastic.co/u/SirStephanikus)\ **Post date:** [September 6, 2022, 6:47pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585/4 "2022-09-06T18:47:38Z")
Thanks for your quick response @ruegenl I'm not sure what you mean with a "sham" filebeat. It is a lab environment and it is mandatory to use filebeat AND logstash for various learning reasons. On the VM is the whole ELK stack. Without logstash, the dashboards work as intended. When I edit the various dashboard widgets by hand and choose the "right" field like this host.name --\> host.name.keyword ... the error message for this particular field is gone. However...I highly assume that this is a bug. Even the manual (correct me if I'm wrong) does not mention anything in this regard. I found a [chinese blog](https://blog.csdn.net/weixin_43549350/article/details/116352671) that had this problem too but this guy had his own indice. Perhaps it explains the topic a bit better (I used a translator plugin in my browser to translate it into english). ---
**Author:** ![SirStephanikus](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@SirStephanikus](https://discuss.elastic.co/u/SirStephanikus)\ **Post date:** [September 6, 2022, 6:51pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585/5 "2022-09-06T18:51:05Z")
Update to the ELK [manual](https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html#fielddata-mapping-param): However...the manual explains the problem, but nowhere do I find a quick workaround for the dozens of default dashboard. ---
**Author:** ![SirStephanikus](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@SirStephanikus](https://discuss.elastic.co/u/SirStephanikus)\ **Post date:** [September 6, 2022, 6:57pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585/6 "2022-09-06T18:57:14Z")
Another Update: [winlogbeat had this issue in ELK 7.10.x ...looks like a bug to me:](https://github.com/elastic/beats/issues/21725) Yep, a bug --\> [see this.](https://discuss.elastic.co/t/elastic-7-9-1-security-siem-your-visualization-has-error-s-illegal-argument-exception/250265/14) Will test the recommended solution and will give my feedback ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 6, 2022, 7:26pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585/7 "2022-09-06T19:26:36Z")
@SirStephanikus Nope that is not your issue in 8.3.3 I just ran setup with `setup.ilm.check_exists: false` `./filebeat setup -e` and the mappings are correct ```auto GET _cat/indices?v health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open .ds-filebeat-8.3.3-2022.09.06-000001 u3YyG9i_Sh2XduYtAaVloQ 1 1 0 0 225b 225b GET .ds-filebeat-8.3.3-2022.09.06-000001/ { ".ds-filebeat-8.3.3-2022.09.06-000001": { "aliases": {}, "mappings": { "_meta": { "beat": "filebeat", "version": "8.3.3" }, "_data_stream_timestamp": { "enabled": true }, ........ "host": { "properties": { "architecture": { "type": "keyword", "ignore_above": 1024 }, .... "hostname": { "type": "keyword",