Field data loading is forbidden on [message]

Elastic Stack Beats
1

I am using winlogbeat to monitor windows log event. However, when I tried to use message field to do some visulaization, it pops out the message as shown in the topic.

Is there any ways to solve it. I am using winlogbeat to send data to logstash in order to gather the raw data field. Other field generates raw field except message field. Is this the reason?

2

Creating a visualization based on an analyzed field is not a good idea because it can use lots of heap memory. And this error is a safeguard put in place to prevent this. The index template provided by Logstash disables fielddata for analyzed fields. The index template also does not setup a multi-field for message so you do not get a raw field for message. You can customize this behavior by providing your own index templates.

What are you trying to visualize using the message field? There may be alternative means of accomplishing it. Can you describe what you are trying to accomplish?

3

I am trying to make a table to map the event_id, message, count. However, when I add message field into the visulization, the words within the message will be break into different rows

4

The relationship between event_id and message is not one-to-one, it's one-to-many since the message is usually populated with event specific details. So I don't think you can really visualize that in a data table.

You might be better off with just an event_id and count table. Then you can select a particular event_id from the table, pin the query, then visualize the messages over on the discover tab.

If you want to make message a raw field you need to change the index template. Here's the Logstash template if that's what you are using: https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/elasticsearch-template.json

5

Is that the same as the dafault es output template for logstash? Because it seems the same and I am using the default template for logstash for packetbeat. I meant except message field other field are able to generate a raw field. Thus, it seems wierd

6

Yeah, that link points to the default ES index template shipped with Logstash.

If you examine the template you will see that the message field is handled separately from other string fields. The message field does not have the fields: { "raw": ... } section which explains why you don't have a raw message field.

7

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.