# Logstash does nothing for valid json input

**URL:** https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595
**Category:** Logstash
**Created:** [March 7, 2017, 4:00am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595 "2017-03-07T04:00:53Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![tushartyagi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tushartyagi/32/16140_2.png) [@tushartyagi](https://discuss.elastic.co/u/tushartyagi)
#### Post date: [March 7, 2017, 4:00am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/1 "2017-03-07T04:00:53Z")

</div>

Hi,

Since yesterday I am trying to work with logstash but I am not able to make much progress.

I have some sentiment data in json which I am trying to read in. If there is a syntax error in the file, or if I use "json" codec for newline delimited file, then I get loads of errors in the stdout. This shows that logstash is at-least trying to read the file.

If I have a single-line json, and I use the codec "json", then I see absolutely nothing. Just some configuration info is printed and the app exits.

Here is my config file:

```auto
input {  
  file {
  	path => "e:\json_data.json"
  	codec => "json"
  	sincedb_path => "NUL"
    start_position => "beginning"
  }
}

output {  
    stdout {
        codec => "rubydebug"
    }
}

```

Here is the data I want to read-in. I remove all the spaces and turn the file into a non-newline delimited json before passing the data

```auto
[
{
  "status": "OK",  
  "docSentiment": {
    "score": "-0.831986", 
    "type": "negative"
  }, 
  "language": "english", 
  "docEmotions": {
    "anger": "0.5748", 
    "joy": "0.046363", 
    "fear": "0.041051", 
    "sadness": "0.036716", 
    "disgust": "0.399636"
  } 
}
]

```

For the interested, here's the data in one line:

```auto
[{"status":"OK","totalTransactions":"2","docSentiment":{"score":"-0.831986","type":"negative"},"language":"english","docEmotions":{"anger":"0.5748","joy":"0.046363","fear":"0.041051","sadness":"0.036716","disgust":"0.399636"}}]

```

What I am not able to understand is that using `stdin { codec => "json" }` and pasting the data on the console works fine. But redirecting the correct data does nothing. Again, passing the errorneous data throws a lots of errors.

Since I am not using `elasticsearch` as the `output`, I at least want to see how the data is being parsed by the application.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 7, 2017, 6:14am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/2 "2017-03-07T06:14:13Z")

</div>

Try the json\_lines codec instead. You're going to have to pass the whole JSON object on a single line.

---

<div class="post-metadata">

### Author: ![tushartyagi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tushartyagi/32/16140_2.png) [@tushartyagi](https://discuss.elastic.co/u/tushartyagi)
#### Post date: [March 7, 2017, 6:46am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/3 "2017-03-07T06:46:59Z")

</div>

The documentation says that `json_codec` should be used for newline delimited lines, and I am sending json as a single line w/o newlines.  
Anyhow, tried with json\_codec and still I see no output.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 7, 2017, 6:53am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/4 "2017-03-07T06:53:03Z")

</div>

> and I am sending json as a single line w/o newlines.

Exactly, but if you have multiple JSON objects they will be separated by newline characters and therefore the json\_lines codec is a good choice.

Your problem is most likely that you're not setting `start_position => beginning` for your file input, so Logstash is tailing your file.

---

<div class="post-metadata">

### Author: ![tushartyagi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tushartyagi/32/16140_2.png) [@tushartyagi](https://discuss.elastic.co/u/tushartyagi)
#### Post date: [March 7, 2017, 7:04am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/5 "2017-03-07T07:04:07Z")

</div>

I am using `start_position => "beginning"` (with quotes) but have not listed it here ☹  
Let me change the config above to reflect that.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 7, 2017, 7:05am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/6 "2017-03-07T07:05:53Z")

</div>

Increase the log level with `--verbose` or even `--debug`to get more about what's happening.

---

<div class="post-metadata">

### Author: ![tushartyagi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tushartyagi/32/16140_2.png) [@tushartyagi](https://discuss.elastic.co/u/tushartyagi)
#### Post date: [March 7, 2017, 7:10am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/7 "2017-03-07T07:10:14Z")

</div>

It's sort of an infinite loop of:

```auto
12:38:04.351 [Ruby-0-Thread-10: D:/programs/logstash/logstash-core/lib/logstash/pipeline.rb:538] DEBUG logstash.pipeline - Pushing flush onto pipeline
12:38:04.743 [[main]<file] DEBUG logstash.inputs.file - each: file grew: d:\elk\input2.json: old size 531, new size 532
12:38:05.748 [[main]<file] DEBUG logstash.inputs.file - each: file grew: d:\elk\input2.json: old size 531, new size 532
12:38:06.762 [[main]<file] DEBUG logstash.inputs.file - each: file grew: d:\elk\input2.json: old size 531, new size 532
12:38:07.776 [[main]<file] DEBUG logstash.inputs.file - each: file grew: d:\elk\input2.json: old size 531, new size 532
12:38:08.777 [[main]<file] DEBUG logstash.inputs.file - each: file grew: d:\elk\input2.json: old size 531, new size 532
12:38:08.777 [[main]<file] DEBUG logstash.inputs.file - _globbed_files: d:\elk\input2.json: glob is: []
12:38:08.777 [[main]<file] DEBUG logstash.inputs.file - _globbed_files: d:\elk\input2.json: glob is: ["d:\\elk\\input2.json"] because glob did not work

```

These lines keep printing again & again.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 7, 2017, 7:23am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/8 "2017-03-07T07:23:03Z")

</div>

The last two lines are interesting. Have you double-checked that the file listed there actually exists and is accessible to Logstash? Have you tried using forward slashes instead of backslashes in the path?

---

<div class="post-metadata">

### Author: ![tushartyagi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tushartyagi/32/16140_2.png) [@tushartyagi](https://discuss.elastic.co/u/tushartyagi)
#### Post date: [March 7, 2017, 7:28am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/9 "2017-03-07T07:28:03Z")

</div>

Putting forward slashes changes the globbed\_files output to

```auto
12:55:36.390 [[main]<file] DEBUG logstash.inputs.file - _globbed_files: d:/elk/input2.json: glob is: ["d:/elk/input2.json"]

```

Also, this glob info comes in 1 every 5 or 6 loops. The "file grew" log comes everytime (5 times in a loop).

Is this glob related to filename-wildcard-glob pattern?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 4, 2017, 7:28am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595/10 "2017-04-04T07:28:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
