Logstash updating not working


(Gobinda Nandi) #1

this is logstash.conf file

input {
  file {
    path => "/var/www/html/review/EsData/review_data.json"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => "json"        
  }
}

filter { 
}

output {
  if [reviewId] != "" {
    stdout { codec => rubydebug }
    elasticsearch {
      hosts => ["http://localhost:9200/"] 
      index => "review_location_gobinda_26"
      document_id => "%{reviewId}"
    }
  }
}

review_data.json file

[{"reviewId":"first data"}]

i am running the service using this command

sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/review.conf

and keeping this service on if i change that json file (review_data.json file ) like this

[{"reviewId":"second data"}]

I am getting this kind of error msg

[ERROR] 2019-01-11 13:31:12.687 [[main]<file] json - JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for root starting at [Source: (String)"}]"; line: 1, column: 0])
 at [Source: (String)"}]"; line: 1, column: 2]>, :data=>"}]"}
{
       "message" => "}]",
          "tags" => [
        [0] "_jsonparsefailure"
    ],
    "@timestamp" => 2019-01-11T08:01:12.689Z,
      "@version" => "1",
          "host" => "skills34-Lenovo-ideapad-330-15IKB",
          "path" => "/var/www/html/review/EsData/review_data.json"
}

when I run the service with data there is no issue but while updating why this error is coming. not able to understand. Need help.

thanks in advance

@inandi


#2

The file input is like tail. If you have a file that contains

[{"reviewId":"first data"}]

(27 characters) and you change it to contain

[{"reviewId":"second data"}]

(28 characters) then it will only consume the last character. It looks like it is actually consuming the last two characters of the file, which may indicate you have an extra space in the file.

If, however, you append that second line to the file then it should work OK.


(Gobinda Nandi) #3

what should be the logstash conf setting? is it correct?