# How to write filter part in my conf file for json file as input

**URL:** <https://discuss.elastic.co/t/how-to-write-filter-part-in-my-conf-file-for-json-file-as-input/86075>\
**Category:** Logstash\
**Created:** [May 17, 2017, 9:46am UTC](https://discuss.elastic.co/t/how-to-write-filter-part-in-my-conf-file-for-json-file-as-input/86075 "2017-05-17T09:46:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gopi\_K1](https://avatars.discourse-cdn.com/v4/letter/g/9e8a1a/32.png) [@Gopi\_K1](https://discuss.elastic.co/u/Gopi_K1)\
**Post date:** [May 17, 2017, 9:46am UTC](https://discuss.elastic.co/t/how-to-write-filter-part-in-my-conf-file-for-json-file-as-input/86075/1 "2017-05-17T09:46:33Z")

</div>

Here is my conf file look like:

input  
{  
file  
{  
codec =\> "json"  
path =\> "/ci/data/cirrus/report/\*/status.json"  
start\_position =\> "beginning"  
sincedb\_path =\> "/ci/data/cirrus/sincedb\_path/status.db"  
type =\> "STATUS"  
ignore\_older =\> 0  
}  
}

filter {  
json {  
source =\> "path"  
add\_field =\>  
{  
"status" =\> "%{field1}"  
"cirrus\_id" =\> "%{field2}"  
}  
}  
}

output {  
stdout { codec =\> rubydebug }  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "scan"  
}  
}

what's wrong with conf file?? and what should i write inorder to get certain fields from my json file.

my json file look like:

{  
"status": "abort",  
"time\_start": 1494845194.7839701,  
"agent\_pid": 16466,  
"id": "ci-000c2969deb0\_administrator\_1",  
"time\_queued": 1494845187.88799,  
"time\_end": 1494845194.8816111,  
"message": "catch exception [[Errno 2] No such file or directory: u'/ci/data/cirrus/jobs/ci-000c2969deb0\_administrator\_1/input/job.json\_from\_(no suts defined)'];1 stage(s) failed",  
"stages": {  
"run\_sys\_cmd\_sut": {  
"status": "fail",  
"message": "failed to connect to sut, unknown SUT OS type"  
}  
}  
}

i want to store get "status", "id" , "time\_queued " from the above json file as an output. for this what should i write in my filter part.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [May 17, 2017, 11:24am UTC](https://discuss.elastic.co/t/how-to-write-filter-part-in-my-conf-file-for-json-file-as-input/86075/2 "2017-05-17T11:24:52Z")

</div>

your json file is "pretty printed"  
In the file input with a json codec, each line is expected to be a full json object per line.  
e.g.

```auto
{"field1":"value1","field2":"value2","field3":"value3","field4":"value4"}
{"field1":"value1","field2":"value2","field3":"value3","field4":"value4"}

```

You should consider using filebeat to do the multiline on the CI box and send the data to a Logstash box.

Failing that, consider using a script that reads files from `/ci/data/cirrus/report/*/status.json`, removes the newline characters and appends the result text as one line to a new file that you "tail" in the file input.

As a last resort you can try using the multiline codec in the file input but its not easy see [stackoverflow](http://stackoverflow.com/a/25625187/5349531)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 14, 2017, 11:25am UTC](https://discuss.elastic.co/t/how-to-write-filter-part-in-my-conf-file-for-json-file-as-input/86075/3 "2017-06-14T11:25:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.