Logstash only reading first JSON line

Elastic Stack Logstash
1

Hi,

Suddenly my S3 input stopped processing complete files. This morning the same files in the same bucket were processed just fine, but after deleting some files, the remaining files suddenly only have the first JSON line processed per file and the reset is ignored. I did not make any changes anywhere.

I'm testing with only 3 files now each containing only a dozen or so JSON entries so performance shouldn't be the issue.

JSON example:

{"@timestamp":"2018-01-18T13:44:44.000Z","state":"LOGGED IN"}{"@timestamp":"2018-01-18T13:44:44.000Z","Eb":"01.6"}{"@timestamp":"2018-01-18T13:45:20.000Z","hrs":"100"}

config

input {
  s3 {
    access_key_id => "secret"
    secret_access_key => "secret"
    region => "eu-west-2"
    bucket => "testbucket"
    sincedb_path => "/dev/null"
    codec => "json"
  }
}


filter {
  mutate {
     remove_field => ["@version"]
  }
  ruby {
     code => 'event.set("@ingest_time", Time.now.strftime("%Y-%m-%dT%H:%M"))'
  }
}


output {
  stdout {
     codec => rubydebug
  }
}

What is going on? If I remove the json codec I do see the whole message appearing.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.