How to find JSON-Properties from LogFile as Fields in Elasticsearch?

(Josh) #1


I have to create an ELK-Showcase for my team.

The task is to take Log(files) from an ASP.NET-Application with Filebeat, forward them via Logstash to Elasticsearch and create a few sample-dashboards in Kibana.

I have a basic setup up and runnig now. But I am not really happy about what I´m seeing in elasticsearch. Right now I find basically the whole Log-message in the message-field in ES. But I want to see all JSON-Properties separated, to query them easily. Example:

I have created a new ASP.NET Core WebApplication, with Serilog as a logger. One Log-entry can look like this:

"MessageTemplate":"{ActionName} has been invoked. Output is {Output}",

I want to see something similar to this JSON in ES, to find e.g. every Message from 2017/12 with above message-template. What is the best way to achieve that?

Right now I have installed ES with MSI-Installer (without any Extras), Logstash is configured like this:
input {
beats {
port => "5044"

output {
elasticsearch {
hosts => [ "localhost:9200" ]

And here is the filebeat configuration:

hosts: ["localhost:5044"]


  • type: log
    • C:\myPath*.log

I´ve read something about logstashs JSON-Filter but I don´t really understand if this is what I am looking for and if so, how to configure it correctly.

Does anyone have an idea what I´m missing, doing wrong etc.?

Thanks in advance,

(Magnus Bäck) #2

One Log-entry can look like this:

On one line or spread across multiple lines as in your example?

(Josh) #3

On one line.. Just formatted it like this for better readability.

(Josh) #4

Hi @magnusbaeck,

I´ve just seen that my initial answer is not correct: If I log any Microsoft-ExceptionMessages, the message contains line break characters \r\n. Does this have any implications regarding your question?

(Josh) #5

After a long day I´ve found out, that logstash was not the right place for my configuration. My answer was here: Parse JSON data with filebeat

This configuration for filebeat.prospectors finally solved my problem.

@magnusbaeck nevertheless thank you for your answer!

(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.