Visualize: Fielddata is disabled on text fields by default. Set fielddata=true on [apache2.access.remote_ip] in order to load fielddata in memory

[Satish_Chhatpar]

Please help me to fix this error.

versions used
filebeat version 6.3.2
logstash version 6.3.2
elasticsearch version 6.3.2
kibana version 6.3.2

Apache log format is
LogFormat ""%{True-Client-IP}i" %t %>s %b "%r" "%{Referer}i" "%{X-Forwarded-For}i" "%{User-Agent}i" "%{X-JSESSIONID}o" %u %q %T/%D %h" custom

filebeat.yml

filebeat.inputs:

• paths:
  • /path/to/logs/access*
    tags: [apache_access]

output.logstash:
hosts: ["127.0.0.1:5044"]
setup.kibana:
host: "http://127.0.0.1:5601"

logstash config

input {
beats {
port => 5044
}
}

filter {
if "apache_access" in [tags] {
grok {
# You'll need to customize the pattern for your log format.
match => { "message" => [
""%{IPORHOST:apache2.access.client_ip}" [%{HTTPDATE:apache2.access.time}] %{NUMBER:apache2.access.response_code} %{NUMBER:apache2.access.body_sent.bytes} "%{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}" "%{DATA:apache2.access.rawrequest}" "%{DATA:apache2.access.referrer}" "%{DATA:apache2.access.agent}" "%{DATA:apache2.access.jsessionid}" - %{DATA:apache2.access.extra} %{NUMBER:apache2.access.seconds}/%{NUMBER:apache2.access.microseconds} %{IPORHOST:apache2.access.remote_ip}",
""%{IPORHOST:apache2.access.client_ip}" [%{HTTPDATE:apache2.access.time}] %{NUMBER:apache2.access.response_code} - "%{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}" "%{DATA:apache2.access.rawrequest}" "%{DATA:apache2.access.referrer}" "%{DATA:apache2.access.agent}" "%{DATA:apache2.access.jsessionid}" - %{NUMBER:apache2.access.seconds}/%{NUMBER:apache2.access.microseconds} %{IPORHOST:apache2.access.remote_ip}",
""%{IPORHOST:apache2.access.client_ip}" [%{HTTPDATE:apache2.access.time}] %{NUMBER:apache2.access.response_code} %{NUMBER:bytes} "%{WORD:apache2.access.method} %{DATA:apache2.access.url} HTTP/%{NUMBER:apache2.access.http_version}" "%{DATA:apache2.access.rawrequest}" "%{DATA:apache2.access.referrer}" "%{DATA:apache2.access.agent}" "%{DATA:apache2.access.jsessionid}" - %{NUMBER:apache2.access.seconds}/%{NUMBER:apache2.access.microseconds} %{IPORHOST:apache2.access.remote_ip}"
]
}
}
}
}

output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

I am OK to do screen share. I am available from 1:30 PM IST to 9:30 PM IST

Thanks
Satish

[a5a (Archana)]

Your mappings are not properly set up. If you want to use fielddata on text fields, you set fielddata=true for those fields. See https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html

[Satish_Chhatpar]

Archana how do I set fielddata = true ? What is the procedure ?