# Logstash - output csv plugin - all fields are strings

**URL:** <https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513>\
**Category:** Logstash\
**Created:** [March 26, 2021, 3:32pm UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513 "2021-03-26T15:32:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![YaronB](https://avatars.discourse-cdn.com/v4/letter/y/b4bc9f/32.png) [@YaronB](https://discuss.elastic.co/u/YaronB)\
**Post date:** [March 26, 2021, 3:32pm UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/1 "2021-03-26T15:32:10Z")

</div>

i'm trying to move some data to mongodb from elastic using elastic input plugin and csv ouput plugin and then mongoimport.

how can i define the fields types , including nested fields?  
everything is showing up as strings in the csv output -\> mongo  
here is an example of \_source data:

```auto
{
    "id": "uf02fh-ch29ufh-hd289guf-hf92ui",
    "platform": {
      "description": "Chrome 90.0.4430.19 on OS X 10.14.6 64-bit",
      "layout": "Blink",
      "manufacturer": null,
      "os": {
        "architecture": 64,
        "family": "OS X",
        "version": "10.14.6"
      }
    },
    "videos": [
      {
        "filename": "1.mpeg"
      },
      {
        "filename": "2.mpeg"
      }
    ],
    "completed": false,
    "createdAt": "2021-03-15T14:39:56.059Z",
    "brightnessLevel": [
      106.56687102472326
    ],
    "gdprTimestamp": "2021-03-15T14:39:57.468Z",
    "quality": "hd",
    "updatedAt": "2021-03-15T14:40:53.147Z"
}

```

here is the conf file:

```auto
input {
 elasticsearch {
    hosts => "xxx:port"
    user => "elastic"
    password => "password"
    index => "some_index"
    query => '{
    "query": { "match_all": {} },
    "_source": ["id","platform","videos","completed","createdAt","brightnessLevel","gdprTimestamp","quality","updatedAt"]

    }'
    }
  }

output {
  csv {
    # elastic field name
    fields => ["id","platform","videos","completed","createdAt","brightnessLevel","gdprTimestamp","quality","updatedAt"]
    # This is path where we store output.   
    path => "/tmp/csv-export.csv"
  }

}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 26, 2021, 5:13pm UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/2 "2021-03-26T17:13:50Z")

</div>

The csv output calls Array::to\_csv, which is equivalent to [CSV::generate\_line](https://ruby-doc.org/stdlib-3.0.0/libdoc/csv/rdoc/CSV.html#method-c-generate_line), which is defined as returning a string. Anything in the csv file will be a string.

---

<div class="post-metadata">

**Author:** ![YaronB](https://avatars.discourse-cdn.com/v4/letter/y/b4bc9f/32.png) [@YaronB](https://discuss.elastic.co/u/YaronB)\
**Post date:** [March 26, 2021, 8:02pm UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/3 "2021-03-26T20:02:58Z")

</div>

Is there a way to avoid that?  
What would the best practice to move some index data from elastic to mongo using logstash?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 26, 2021, 8:25pm UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/4 "2021-03-26T20:25:42Z")

</div>

You could try using the [mongodb output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-mongodb.html).

---

<div class="post-metadata">

**Author:** ![YaronB](https://avatars.discourse-cdn.com/v4/letter/y/b4bc9f/32.png) [@YaronB](https://discuss.elastic.co/u/YaronB)\
**Post date:** [March 27, 2021, 12:30pm UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/5 "2021-03-27T12:30:03Z")

</div>

so i tried with the following conf:

```auto
input {
 elasticsearch {
    hosts => "ES:port"
    user => "elastic"
    password => "password"
    index => "some_index"
    query => '{
    "query": { "match_all": {} },
    "_source": ["id","platform","videos","completed","createdAt","brightnessLevel","gdprTimestamp","quality","updatedAt"]
    }'
    }
  }

output {
mongodb {
    id => "my_id"
    collection => "some_collection"
    database => "dbname"
    uri => "mongodb+srv://user:pass@mongo_hostname/dbname?retryWrites=true&w=majority"
    codec => "json"
  }

}

```

but got this error:  
[2021-03-27T12:33:37,286][WARN][logstash.outputs.mongodb][main] MONGODB | Failed to handshake with [mongodb shard address]: ArgumentError: wrong number of arguments (given 2, expected 1)

any idea what's wrong in the config ? logstash-output-mongodb (3.1.6)

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 27, 2021, 3:56pm UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/6 "2021-03-27T15:56:00Z")

</div>

A similar error came up recently [here](https://discuss.elastic.co/t/logstash-mongodb-output-plugin-command-insert-requires-authentication/268265). I do not have a good answer for it. If it threw an exception you could capture the init of that exception in a debugger, if it provided a stack trace you could review the code where it happens. However, we have neither.

---

<div class="post-metadata">

**Author:** ![YaronB](https://avatars.discourse-cdn.com/v4/letter/y/b4bc9f/32.png) [@YaronB](https://discuss.elastic.co/u/YaronB)\
**Post date:** [March 30, 2021, 10:11am UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/7 "2021-03-30T10:11:55Z")

</div>

Thanks @Badger  
i was thinking logstash and its plugins are much more stabled and working out of the box.  
especially for such basic use cases...  
it's a bummer.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 27, 2021, 10:12am UTC](https://discuss.elastic.co/t/logstash-output-csv-plugin-all-fields-are-strings/268513/8 "2021-04-27T10:12:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
