# How to find JSON-Properties from LogFile as Fields in Elasticsearch?

**URL:** https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266
**Category:** Logstash
**Created:** [January 5, 2018, 10:00am UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266 "2018-01-05T10:00:43Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![awaitJosh](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@awaitJosh](https://discuss.elastic.co/u/awaitJosh)
#### Post date: [January 5, 2018, 10:00am UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266/1 "2018-01-05T10:00:43Z")

</div>

Hi,

I have to create an ELK-Showcase for my team.

The task is to take Log(files) from an ASP.NET-Application with Filebeat, forward them via Logstash to Elasticsearch and create a few sample-dashboards in Kibana.

I have a basic setup up and runnig now. But I am not really happy about what I´m seeing in elasticsearch. Right now I find basically the whole Log-message in the message-field in ES. But I want to see all JSON-Properties separated, to query them easily. Example:

I have created a new [ASP.NET](http://ASP.NET) Core WebApplication, with Serilog as a logger. One Log-entry can look like this:

{  
"Timestamp":"2018-01-02T17:15:37.8033208+01:00",  
"Level":"Information",  
"MessageTemplate":"{ActionName} has been invoked. Output is {Output}",  
"Properties":  
{  
"ActionName":"GetAction",  
"Output":["value1","value2"],  
"SourceContext":"ValuesLogger",  
"ActionId":"459f66df-299e-44e7-813e-45749f4f8d48",  
"RequestId":"0HLAHVG7IOG5M:00000001",  
"RequestPath":"/api/values",  
"Scope":["GetValues"]  
}  
}

I want to see something similar to this JSON in ES, to find e.g. every Message from 2017/12 with above message-template. What is the best way to achieve that?

Right now I have installed ES with MSI-Installer (without any Extras), Logstash is configured like this:  
input {  
beats {  
port =\> "5044"  
}  
}

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
}  
}

And here is the filebeat configuration:

output.logstash:  
hosts: ["localhost:5044"]

filebeat.prospectors:

- type: log  
paths:
  - C:\myPath\*.log

I´ve read something about logstashs JSON-Filter but I don´t really understand if this is what I am looking for and if so, how to configure it correctly.

Does anyone have an idea what I´m missing, doing wrong etc.?

Thanks in advance,  
Josh

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 5, 2018, 12:03pm UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266/2 "2018-01-05T12:03:38Z")

</div>

> One Log-entry can look like this:

On one line or spread across multiple lines as in your example?

---

<div class="post-metadata">

### Author: ![awaitJosh](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@awaitJosh](https://discuss.elastic.co/u/awaitJosh)
#### Post date: [January 5, 2018, 12:21pm UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266/3 "2018-01-05T12:21:51Z")

</div>

On one line.. Just formatted it like this for better readability.

---

<div class="post-metadata">

### Author: ![awaitJosh](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@awaitJosh](https://discuss.elastic.co/u/awaitJosh)
#### Post date: [January 5, 2018, 3:47pm UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266/4 "2018-01-05T15:47:05Z")

</div>

Hi @magnusbaeck,

I´ve just seen that my initial answer is not correct: If I log any Microsoft-ExceptionMessages, the message contains line break characters \r\n. Does this have any implications regarding your question?

---

<div class="post-metadata">

### Author: ![awaitJosh](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@awaitJosh](https://discuss.elastic.co/u/awaitJosh)
#### Post date: [January 5, 2018, 6:25pm UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266/5 "2018-01-05T18:25:39Z")

</div>

After a long day I´ve found out, that logstash was not the right place for my configuration. My answer was here: [Parse JSON data with filebeat - #3 by calmandniceperson](https://discuss.elastic.co/t/parse-json-data-with-filebeat/80008/3)

This configuration for filebeat.prospectors finally solved my problem.

> [@Parse JSON data with filebeat](https://discuss.elastic.co/t/parse-json-data-with-filebeat/80008/4):
>
> filebeat.prospectors:
> 
> - paths:
> - test.json  
> input\_type: log  
> json.keys\_under\_root: true  
> json.add\_error\_key: true

@magnusbaeck nevertheless thank you for your answer!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 2, 2018, 6:25pm UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266/6 "2018-02-02T18:25:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
