# Logstash: Input from SQS does not get inserted to InfluxDB

**URL:** https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996
**Category:** Logstash
**Created:** [April 2, 2019, 1:04pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996 "2019-04-02T13:04:22Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 2, 2019, 1:04pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/1 "2019-04-02T13:04:22Z")

</div>

I am trying to insert data to InfluxDB using Logstash with input method SQS: I send message to Sqs and Logstash reads the messages from Sqs and tries to put data to InfluxDB.

I have set logstash like below, however, the attempt to insert data to influxDB keeps making an error saying `[WARN][logstash.outputs.influxdb] Non recoverable exception while writing to InfluxDB {:exception=>#<Errno::EPIPE: Broken pipe - No message available>}` . As you can see, the output of `stdout` seems fine. What am I doing wrong?

Any advice or suggestion would be very appreciated.

Thank you in advance.

**stdout value**

```auto
{
      "userId" => 1,
         "lng" => 127,
    "@version" => "1",
  "@timestamp" => 2019-04-02T12:46:33.006Z,
         "lat" => 36
}

```

**logstash.conf**

```auto
input {
    sqs {
        codec => "json"
        queue => "logstash-test"        
    }
}

output {
    stdout {}
    influxdb {
        measurement => "location"
        send_as_tags => ["userId"]
        data_points => {
            "userId" => "%{[userId]}"
            "lat" => "%{[lat]]}"
            "lng" => "%{[lng]]}"
        }
        host => "localhost"
        port => "xxxx"
        db => "db_name"
        codec => json
    }
}

```

Used logstash plugins:  
logstash-input-sqs  
logstash-output-influxdb

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 2, 2019, 3:52pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/2 "2019-04-02T15:52:47Z")

</div>

In v 5.0.0 we switched to using the influx db ruby client instead of a general http client.

There is a report of the same error you are seeing in the pre 5.0.0 version but nothing since.

That said, the current release of the logstash-output-influxdb is constrained to use version 0.3.X of the influxdb client. Since then the infuxdb folks have released 0.4.X -\> 0.7.X of their client.

What versions do you have?

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 12:32am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/3 "2019-04-03T00:32:21Z")

</div>

I found that im using `influxdb-0.7.0.gem`. So I have removed 0.7 and installed `influxdb-0.3.17.gem` and still does not work ☹ (still the same error)

Logstash:.6.7.0  
InfluxDB: v1.7.5  
Influx Output Plugin: v5.0.5

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 8:45am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/4 "2019-04-03T08:45:38Z")

</div>

Do you get any data into influxdb?

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 11:01am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/5 "2019-04-03T11:01:33Z")

</div>

no I do not get any data.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 11:34am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/6 "2019-04-03T11:34:35Z")

</div>

I have to presume that other users can use this output with v1.7.5 of influxdb.

The last "problem" issue was logged in the plugin repo in October last year indicated an issue with interpolation of the `db` setting and not a general communications issue, so presumably it mostly worked for this user.

I will do a local test myself.

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 11:52am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/7 "2019-04-03T11:52:40Z")

</div>

I appreciate your help. If the plugin does not work, I will have to use http output plugin and influxdb's http api.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 11:54am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/8 "2019-04-03T11:54:52Z")

</div>

Happy to help. I need to become familiar with this plugin anyway.

Hold off on trying http as this plugin obviously has benefits.

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 11:56am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/9 "2019-04-03T11:56:42Z")

</div>

Yes, definitely. I also want to utilize this plugin if possible. I wil try to find the way to work it out, too.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 11:58am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/10 "2019-04-03T11:58:46Z")

</div>

Any instructions on the InfluxDB setup would help me recreate what you have? I want to compare apples to apples not oranges 🙂

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 12:01pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/11 "2019-04-03T12:01:10Z")

</div>

I have followed the instructions on `https://portal.influxdata.com/downloads/`(install via Homebrew) which is like below

```auto
brew update
brew install influxdb

```

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 12:02pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/12 "2019-04-03T12:02:16Z")

</div>

any config changes?

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 12:03pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/13 "2019-04-03T12:03:32Z")

</div>

I haven't touched any configuration after installation.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 12:05pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/14 "2019-04-03T12:05:09Z")

</div>

Did you create the user and password etc

```auto
With InfluxDB running, visit localhost:9999.
Click Get Started

Enter a Username for your initial user.
Enter a Password and Confirm Password for your user.
Enter your initial Organization Name.
Enter your initial Bucket Name.

```

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 12:06pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/15 "2019-04-03T12:06:22Z")

</div>

I did not create any user and set up any password for the user (since I didnt create any user). No organization nor bucket.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 12:12pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/16 "2019-04-03T12:12:53Z")

</div>

Seems that `bucket` = `database`

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 12:13pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/17 "2019-04-03T12:13:36Z")

</div>

I have created database using CLI command.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [April 3, 2019, 12:18pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/18 "2019-04-03T12:18:54Z")

</div>

Perhaps you need to create a organization, user, password and database (bucket) and then update the influxdb output to use those values (except organization, there is no setting for this).

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 3, 2019, 12:39pm UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/19 "2019-04-03T12:39:55Z")

</div>

I will try to create user and password.

---

<div class="post-metadata">

### Author: ![smchae](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@smchae](https://discuss.elastic.co/u/smchae)
#### Post date: [April 4, 2019, 1:20am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996/20 "2019-04-04T01:20:17Z")

</div>

After adding user and password, I was able to make it work. I think the doc should be updated accordingly.

[Next page](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996.md?page=2)
