# Elastic search logstash

**URL:** <https://discuss.elastic.co/t/elastic-search-logstash/120568>\
**Category:** Logstash\
**Created:** [February 20, 2018, 7:04am UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568 "2018-02-20T07:04:59Z")\
**Posts on this page:** 16\
**Page:** 2

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 21, 2018, 9:20am UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/21 "2018-02-21T09:20:25Z")

</div>

You need to update the file by

- writing the new contents of the file into a temporary file and
- rename that file into place so it overwrites the old file.

You can't just overwrite the existing file.

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 21, 2018, 9:47am UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/22 "2018-02-21T09:47:15Z")

</div>

Thank you sir, i wil try that.

Thanks for all your help, it's great talking to you.

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 22, 2018, 11:13am UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/23 "2018-02-22T11:13:39Z")

</div>

Hello sir,

I added that feature and it is getting updated.

But during updating the existing document it is adding one more document .

\_index: "collegestudents",  
\_type: "college",  
\_id: "%{userId}",  
\_score: 1,  
\_source: {  
tags: [  
"\_csvparsefailure"  
],  
message: "na, India","4444444444", ",  
host: "DESKTOP-TT7RK4V",  
path: "E:\backend\logstash-input.csv",  
@timestamp: "2018-02-22T11:10:45.443Z",  
@version: "1"  
}

can we please help how to handle this case.

Please Help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 22, 2018, 12:18pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/24 "2018-02-22T12:18:05Z")

</div>

The CSV parsing failed. Not sure why but I'm sure the Logstash log contains details.

Why are you converting from JSON to CSV in your Node code instead of just serializing the data to JSON and have Logstash read that directly?

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 22, 2018, 12:37pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/25 "2018-02-22T12:37:25Z")

</div>

Hello sir,

Because i am new to logstash and do not know how to configure the input type and filter for json file , i tried but when i am parsing the array of json , it is getting parsed but not in the right format.

Can you please help me in writing the input and filter config.

I have an array object of this type:

```
   [{
"user": {
    "_id": "5a7bec992e5521389cd4c8d5",
    "email": "ajit009kumar@gmail.com",
    "name": "Ajeet kumar"
},
"location": {
    "id": "ef5aa289432a57b6d718e98ddd5107b7d46b068e",
    "description": "Hyderabad, Telangana, India",
    "_id": "5a7c2ad259ad013024a33056"
},
"contact": "9973903785"

```

}]

Please help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 22, 2018, 1:42pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/26 "2018-02-22T13:42:40Z")

</div>

It'll be much easier if you serialize each object on a line of its own, like this:

```
{"user": "rishu", "email": "...", ...}
{"user": "abhishek", "email": "...", ...}

```

Then you can just use a json or json\_lines codec in your file input and you won't need any filters.

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 22, 2018, 7:07pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/27 "2018-02-22T19:07:30Z")

</div>

Hello sir,

codec =\> json worked for me after configuring the json array into the format provided by you.

i am using the same configuration for the arrayofObject, but at that time it is not working,  
Can you please tell me why it is not working for the data format provided by me.

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 22, 2018, 9:52pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/28 "2018-02-22T21:52:29Z")

</div>

Logstash's file input reads files line by line and turns them into events. If you want a multi-line file containing an array of objects to be read into a single event (that you later split into multiple events) you need to use a multiline codec to join the physical lines into one logical event.

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 23, 2018, 8:37am UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/29 "2018-02-23T08:37:16Z")

</div>

Ok .

Thank You!

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 24, 2018, 12:31pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/30 "2018-02-24T12:31:45Z")

</div>

Hello sir,

Logstash is not creating index for json file iput , when i am not setting  
sincedb\_path =\> "/dev/null"

But when i am using the sincedb\_path =\> "/dev/null" sometimes, it is throwing some error

Error: No such file or directory - E:/dev/null  
Exception: Errno::ENOENT  
Stack: org/jruby/RubyIO.java:3622:in `write'

will i need to create a folder and set the sincedb\_path=\> folder name.

Please Help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 24, 2018, 2:06pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/31 "2018-02-24T14:06:24Z")

</div>

> But when i am using the sincedb\_path =\> "/dev/null" sometimes, it is throwing some error
> 
> Error: No such file or directory - E:/dev/null

If you're running on Windows and want to disable the sincedb functionality set `sincedb_path` to "nul" instead.

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 24, 2018, 2:12pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/32 "2018-02-24T14:12:25Z")

</div>

yes i am running on windows.

ok i am going to set

sincedb\_path =\> "null"

Thanks for help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 24, 2018, 2:13pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/33 "2018-02-24T14:13:09Z")

</div>

"nul", not "null".

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 24, 2018, 2:20pm UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/34 "2018-02-24T14:20:42Z")

</div>

ok thanks

I will set

sincedb\_path =\> "nul"

---

<div class="post-metadata">

**Author:** ![Ajit\_Kumar1](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\
**Post date:** [February 27, 2018, 10:27am UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/35 "2018-02-27T10:27:36Z")

</div>

Hello sir,

it is working properly after setting  
sincedb\_path =\> "nul"

Thanks for help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2018, 10:27am UTC](https://discuss.elastic.co/t/elastic-search-logstash/120568/36 "2018-03-27T10:27:37Z")

</div>

Closed on March 27, 2018, 10:27am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/elastic-search-logstash/120568.md?page=1)
