Visualize: Fielddata is disabled on text fields by default. Set fielddata=true on [apache2.access.remote_ip] in order to load fielddata in memory

Please help me to fix this error.

versions used
filebeat version 6.3.2
logstash version 6.3.2
elasticsearch version 6.3.2
kibana version 6.3.2

Apache log format is
LogFormat ""%{True-Client-IP}i" %t %>s %b "%r" "%{Referer}i" "%{X-Forwarded-For}i" "%{User-Agent}i" "%{X-JSESSIONID}o" %u %q %T/%D %h" custom

filebeat.yml

filebeat.inputs:

  • paths:
    • /path/to/logs/access*
      tags: [apache_access]

output.logstash:
hosts: ["127.0.0.1:5044"]
setup.kibana:
host: "http://127.0.0.1:5601"

logstash config

input {
beats {
port => 5044
}
}

filter {
if "apache_access" in [tags] {
grok {
# You'll need to customize the pattern for your log format.
match => { "message" => [
""%{IPORHOST:apache2.access.client_ip}" [%{HTTPDATE:apache2.access.time}] %{NUMBER:apache2.access.response_code} %{NUMBER:apache2.access.body_sent.bytes} "%{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}" "%{DATA:apache2.access.rawrequest}" "%{DATA:apache2.access.referrer}" "%{DATA:apache2.access.agent}" "%{DATA:apache2.access.jsessionid}" - %{DATA:apache2.access.extra} %{NUMBER:apache2.access.seconds}/%{NUMBER:apache2.access.microseconds} %{IPORHOST:apache2.access.remote_ip}",
""%{IPORHOST:apache2.access.client_ip}" [%{HTTPDATE:apache2.access.time}] %{NUMBER:apache2.access.response_code} - "%{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}" "%{DATA:apache2.access.rawrequest}" "%{DATA:apache2.access.referrer}" "%{DATA:apache2.access.agent}" "%{DATA:apache2.access.jsessionid}" - %{NUMBER:apache2.access.seconds}/%{NUMBER:apache2.access.microseconds} %{IPORHOST:apache2.access.remote_ip}",
""%{IPORHOST:apache2.access.client_ip}" [%{HTTPDATE:apache2.access.time}] %{NUMBER:apache2.access.response_code} %{NUMBER:bytes} "%{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}" "%{DATA:apache2.access.rawrequest}" "%{DATA:apache2.access.referrer}" "%{DATA:apache2.access.agent}" "%{DATA:apache2.access.jsessionid}" - %{NUMBER:apache2.access.seconds}/%{NUMBER:apache2.access.microseconds} %{IPORHOST:apache2.access.remote_ip}"
]
}
}
}
}

output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

I am OK to do screen share. I am available from 1:30 PM IST to 9:30 PM IST

Thanks
Satish

Your mappings are not properly set up. If you want to use fielddata on text fields, you set fielddata=true for those fields. See https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html

Archana how do I set fielddata = true ? What is the procedure ?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.