# JSON file processing - looking for direction
**URL:** https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811
**Category:** Logstash
**Created:** [December 14, 2017, 4:36pm UTC](https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811 "2017-12-14T16:36:43Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Blake\_2112](https://discuss.elastic.co/u/Blake_2112)
#### Post date: [December 14, 2017, 4:36pm UTC](https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811/1 "2017-12-14T16:36:43Z")
To start, I have been successful procssing single JSON files with the following LS config file. I use the 'exec' command to cat the JSON file. Works fine.
The issue I have is that I want to be able to process 100 different JSON files in a directory and this method will not work.
I have attempted to shift to the 'file' input, but haven't had successful results. Basically, I cannot get it to work.
I'm looking to the community for direction as to the best way to process 'x' number of JSON files.
Thanks in advance.
Logstash configuration file:
input {
```
#file {
#path => "C:\ELK_Stack\logstash-5.6.3\br_configs\sampleD_embedded.json"
#tags => ["FileNet", "nmon"]
#codec => "json"
#}
exec {
command => "cat C:\ELK_Stack\logstash-6.0.0\br_configs\sampleD_embedded.json"
codec => "json"
interval => 3600
tags => ["FileNet", "nmon"]
type => "nmon_output"
}
}
```
filter {
json {
source =\> "message"
}
```
mutate {
add_field => ["serverTZ", "US/Central"]
replace => ["@source_host", "%{host}"]
}
date {
match => ["timestamp", "HH:mm:ss'T'dd-MMM-yyyy"]
timezone => "America/Chicago"
target => "@timestamp"
}
```
}
output {
stdout {
codec =\> "rubydebug"
}
#elasticsearch {
#hosts =\> "localhost:9200"
#index =\> "nmon-es-index"
#}
}
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 14, 2017, 4:43pm UTC](https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811/2 "2017-12-14T16:43:58Z")
> The issue I have is that I want to be able to process 100 different JSON files in a directory and this method will not work.
So what happens?
Have you tried using the json\_lines codec instead of json for your exec input?
---
### Author:  [@Blake\_2112](https://discuss.elastic.co/u/Blake_2112)
#### Post date: [December 14, 2017, 7:45pm UTC](https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811/3 "2017-12-14T19:45:04Z")
So, to start, logstash does nothing (just sits there) as if you're trying to process a file yet the file isn't there.
I have bounced back and forth between json and json\_lines without success.
```
input {
file {
path => "C:\ELK_Stack\logstash-6.0.0\br_configs\sampleD_embedded.json"
tags => ["FileNet", "nmon"]
codec => "json_lines"
sincedb_path => "NUL"
start_position => "beginning"
}
#exec {
#command => "cat C:\ELK_Stack\logstash-6.0.0\br_configs\sampleD_embedded.json"
#codec => "json"
#interval => 3600
#tags => ["FileNet", "nmon"]
#type => "nmon_output"
#}
}
filter {
json {
source => "message"
}
```
But, from a run with 'debug' enabled it does look like the JSON file is read through, values notes, etc.
[2017-12-14T13:05:03,431][DEBUG][logstash.inputs.file] \_globbed\_files: C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json: glob is: []
[2017-12-14T13:05:03,432][DEBUG][logstash.inputs.file] \_globbed\_files: C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json: glob is: ["C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json"] because glob did not work
[2017-12-14T13:05:03,435][DEBUG][logstash.inputs.file] \_discover\_file: C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json: new: C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json (exclude is [])
[2017-12-14T13:05:03,449][DEBUG][logstash.inputs.file] \_open\_file: C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json: opening
[2017-12-14T13:05:03,455][DEBUG][logstash.inputs.file] C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json: initial create, no sincedb, seeking to beginning of file
[2017-12-14T13:05:03,456][DEBUG][logstash.inputs.file] Received line {:path=\>"C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json", :text=\>"["}
[2017-12-14T13:05:03,460][DEBUG][logstash.inputs.file] Received line {:path=\>"C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json", :text=\>"{"}
[2017-12-14T13:05:03,462][DEBUG][logstash.inputs.file] Received line {:path=\>"C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json", :text=\>""timestamp": "00:05:32T24-MAR-2016","}
[2017-12-14T13:05:03,464][DEBUG][logstash.inputs.file] Received line {:path=\>"C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json", :text=\>""filename": "sampleD.nmon","}
[2017-12-14T13:05:03,465][DEBUG][logstash.inputs.file] Received line {:path=\>"C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json", :text=\>""host": "sampleN","}
[2017-12-14T13:05:03,466][DEBUG][logstash.inputs.file] Received line {:path=\>"C:\ELK\_Stack\logstash-6.0.0\br\_configs\sampleD\_embedded.json", :text=\>""date": "24-MAR-2016","}
If that is the case, is it just that I don't have the remainder of the LS configuration correct?
Thus, am I using the incorrect filter, etc.?
Thanks.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 11, 2018, 7:45pm UTC](https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811/4 "2018-01-11T19:45:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.