# Elasticsearch / Logstash isnt indexing all lines of json file
**URL:** https://discuss.elastic.co/t/elasticsearch-logstash-isnt-indexing-all-lines-of-json-file/218411
**Category:** Logstash
**Created:** [February 7, 2020, 8:55pm UTC](https://discuss.elastic.co/t/elasticsearch-logstash-isnt-indexing-all-lines-of-json-file/218411 "2020-02-07T20:55:11Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@richk516](https://discuss.elastic.co/u/richk516)
#### Post date: [February 7, 2020, 8:55pm UTC](https://discuss.elastic.co/t/elasticsearch-logstash-isnt-indexing-all-lines-of-json-file/218411/1 "2020-02-07T20:55:11Z")
I am running a very simple config, which should index a json file. It works fine when the json is small (4 or 5 lines) but then
when it gets any longer, I get an error and it only indexes the last few lines. To test. To try different approaches (codecs, etc),
I run DELETE index in Kibana, change the file slightly by adding a new line, and try again. Its very consistent--if the json is
only a few lines its fine, but when it gets longer, no good. I will paste my config, my json, and my error below. I am using
Logstash 7.3.2:
**CONFIG**
input {
file {
type =\> "json"
#codec =\> "plain"
path =\> ["/home/yaz/app/logstash/logstash-7.3.2/logs\_to\_process/testlog.log"]
start\_position =\> beginning
#codec =\> multiline{
# pattern =\> "^["
# negate =\> true
# what =\> "previous"
#}
}
}
filter{
json{
source =\> "message"
}
}
output {
elasticsearch {
hosts =\> ["[http://myserver.com:9200](http://myserver.com:9200)"]
index =\> "tst\_idx\_data"
}
stdout {
codec =\> rubydebug
}
}
**JSON**
\*(Note that there are no commas. From what I can see, proper json should have after each closing bracket, but I get a parsing error
every time. I have also tried around the entire json, with and without commas. cpu\_seconds are sequential to make it easier to
track.)\*
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":1}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":2}
{"bus":"bbb","ds":"ccc","freq":"mon","busdsfreq":"stcccmon","cpu\_seconds":3}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":4}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":5}
{"bus":"bbb","ds":"ccc","freq":"mon","busdsfreq":"stcccmon","cpu\_seconds":6}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":7}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":8}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":9}
{"bus":"bbb","ds":"ccc","freq":"mon","busdsfreq":"stcccmon","cpu\_seconds":10}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":11}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":12}
{"bus":"bbb","ds":"ccc","freq":"mon","busdsfreq":"stcccmon","cpu\_seconds":13}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":14}
{"bus":"bbb","ds":"ccc","freq":"mon","busdsfreq":"stcccmon","cpu\_seconds":15}
{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":16}
**ERROR**
\*(notice only the line with cpu\_seconds=16 is indexed. All others are left out due to a parsing error. If I were to delete all
lines except cpu\_seconds \<4 or so, all would be fine)\*
[2020-02-07T15:25:35,566][WARN][logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>"","busdsfreq":
"stcccmon","cpu\_seconds":15}", :exception=\>#\}
/home/yaz/app/logstash/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/awesome\_print-
1.7.0/lib/awesome\_print/formatters/base\_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
"message" =\> "","busdsfreq":"stcccmon","cpu\_seconds":15}",
"path" =\> "/home/yaz/app/logstash/logstash-7.3.2/logs\_to\_process/testlog.log",
"host" =\> "[myserver.com](http://myserver.com)",
"@version" =\> "1",
"@timestamp" =\> 2020-02-07T20:25:32.944Z,
"tags" =\> [
[0] "\_jsonparsefailure"
],
"type" =\> "json"
}
{
"freq" =\> "mon",
"busdsfreq" =\> "aaacccmon",
"message" =\> "{"bus":"aaa","ds":"ccc","freq":"mon","busdsfreq":"aaacccmon","cpu\_seconds":16}",
"path" =\> "/home/yaz/app/logstash/logstash-7.3.2/logs\_to\_process/testlog.log",
"host" =\> "[myserver.com](http://myserver.com)",
"type" =\> "json",
"bus" =\> "aaa",
"cpu\_seconds" =\> 16,
"ds" =\> "ccc",
"@version" =\> "1",
"@timestamp" =\> 2020-02-07T20:25:32.987Z
}
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 6, 2020, 8:55pm UTC](https://discuss.elastic.co/t/elasticsearch-logstash-isnt-indexing-all-lines-of-json-file/218411/2 "2020-03-06T20:55:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.