Elastic search logstash

Elastic Stack Logstash
21

You need to update the file by

  • writing the new contents of the file into a temporary file and
  • rename that file into place so it overwrites the old file.

You can't just overwrite the existing file.

22

Thank you sir, i wil try that.

Thanks for all your help, it's great talking to you.

23

Hello sir,

I added that feature and it is getting updated.

But during updating the existing document it is adding one more document .

_index: "collegestudents",
_type: "college",
_id: "%{userId}",
_score: 1,
_source: {
tags: [
"_csvparsefailure"
],
message: "na, India","4444444444", ",
host: "DESKTOP-TT7RK4V",
path: "E:\backend\logstash-input.csv",
@timestamp: "2018-02-22T11:10:45.443Z",
@version: "1"
}

can we please help how to handle this case.

Please Help!

24

The CSV parsing failed. Not sure why but I'm sure the Logstash log contains details.

Why are you converting from JSON to CSV in your Node code instead of just serializing the data to JSON and have Logstash read that directly?

25

Hello sir,

Because i am new to logstash and do not know how to configure the input type and filter for json file , i tried but when i am parsing the array of json , it is getting parsed but not in the right format.

Can you please help me in writing the input and filter config.

I have an array object of this type:

   [{
"user": {
    "_id": "5a7bec992e5521389cd4c8d5",
    "email": "ajit009kumar@gmail.com",
    "name": "Ajeet kumar"
},
"location": {
    "id": "ef5aa289432a57b6d718e98ddd5107b7d46b068e",
    "description": "Hyderabad, Telangana, India",
    "_id": "5a7c2ad259ad013024a33056"
},
"contact": "9973903785"

}]

Please help!

26

It'll be much easier if you serialize each object on a line of its own, like this:

{"user": "rishu", "email": "...", ...}
{"user": "abhishek", "email": "...", ...}

Then you can just use a json or json_lines codec in your file input and you won't need any filters.

27

Hello sir,

codec => json worked for me after configuring the json array into the format provided by you.

i am using the same configuration for the arrayofObject, but at that time it is not working,
Can you please tell me why it is not working for the data format provided by me.

Thanks for your help!

28

Logstash's file input reads files line by line and turns them into events. If you want a multi-line file containing an array of objects to be read into a single event (that you later split into multiple events) you need to use a multiline codec to join the physical lines into one logical event.

29

Ok .

Thank You!

30

Hello sir,

Logstash is not creating index for json file iput , when i am not setting
sincedb_path => "/dev/null"

But when i am using the sincedb_path => "/dev/null" sometimes, it is throwing some error

Error: No such file or directory - E:/dev/null
Exception: Errno::ENOENT
Stack: org/jruby/RubyIO.java:3622:in `write'

will i need to create a folder and set the sincedb_path=> folder name.

Please Help!

31

But when i am using the sincedb_path => "/dev/null" sometimes, it is throwing some error

Error: No such file or directory - E:/dev/null

If you're running on Windows and want to disable the sincedb functionality set sincedb_path to "nul" instead.

32

yes i am running on windows.

ok i am going to set

sincedb_path => "null"

Thanks for help!

33

"nul", not "null".

34

ok thanks

I will set

sincedb_path => "nul"

35

Hello sir,

it is working properly after setting
sincedb_path => "nul"

Thanks for help!

36

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.