# Configure a list of integers with Logstash

**URL:** https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098
**Category:** Logstash
**Created:** [August 7, 2017, 12:03pm UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098 "2017-08-07T12:03:18Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![marwa](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@marwa](https://discuss.elastic.co/u/marwa)
#### Post date: [August 7, 2017, 12:03pm UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/1 "2017-08-07T12:03:18Z")

</div>

Hi,  
i want to load datasets (csv file) to elasticseach viaa logstash,this is a snippet of my data sets

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/7/07724262dbddc71de4a6fdd4b7613e25faae9633.png)transaction\_date,customer\_name,age,job,fidelity\_card\_id,product\_id\_sold,

2/24/2017;14:15:43,Roberto Michel,25,Analyst Programmer,4.17500359418653E+015,"431, 1032, 197, 684, 201, 206, 1022, 922, 556, 677","4, 9, 8, 5, 2, 3, 10, 67, 1"  
5/15/2016;10:51:44,Reiko Branchet,22,Engineer III,374283539519835,"680, 495, 584, 161, 972, 1045, 615, 15, 788, 579","67, 1, 4, 2, 8, 9, 5, 3, 10"

the two last fields are lists of integers ,i want to configure them with logstash as a list but i don't know how .i tried to use

mutate {

convert=\>{"[quantity]"=\>"[integer]"}

}

this is the error  
Error registering plugin {:plugin=\>"#\<LogStash::FilterDelegator:0x1e22751a @id="1840e7e66654467e4855f4a7e3f7cab6e8ecc2e8-5", @klass=LogStash::Filters::Mutate, @metric\_events=#\<LogStash::Instrument::NamespacedMetric:0x78f7d852 @metric=#\<LogStash::Instrument::Metric:0x6978def3 @collector=#\<LogStash::Instrument::Collector:0x8f987f9 @agent=nil, @metric\_store=#\<LogStash::Instrument::MetricStore:0x57eb959 @store=#\<Concurrent:🗺0x00000000061fc4 entries=3 default\_proc=nil\>, @structured\_lookup\_mutex=#Mutex:0x430dd4ec, @fast\_lookup=#\<Concurrent:🗺0x00000000061fc8 entries=63 default\_proc=nil\>\>\>\>, @namespace\_name=[:stats, :pipelines, :main, :plugins, :filters, :"1840e7e66654467e4855f4a7e3f7cab6e8ecc2e8-5", :events]\>, @logger=#\<LogStash::Logging::Logger:0x28af810b @logger=#Java::OrgApacheLoggingLog4jCore::Logger:0xdcd1c90\>, @filter=\<LogStash::Filters::Mutate convert=\>{"age"=\>"integer", "[quantity]"=\>"[integer]"}, id=\>"1840e7e66654467e4855f4a7e3f7cab6e8ecc2e8-5", enable\_metric=\>true, periodic\_flush=\>false\>\>", :error=\>"translation missing: en.logstash.agent.configuration.invalid\_plugin\_register"}  
[2017-08-07T12:49:10,469][ERROR][logstash.agent] Pipeline aborted due to error {:exception=\>#\<LogStash::ConfigurationError: translation missing: en.logstash.agent.configuration.invalid\_plugin\_register\>, :backtrace=\>["/opt/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.5/lib/logstash/filters/mutate.rb:189:in `register'", "org/jruby/RubyHash.java:1342:in`each'", "/opt/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.5/lib/logstash/filters/mutate.rb:183:in `register'", "/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:281:in`register\_plugin'", "/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:292:in `register_plugins'", "org/jruby/RubyArray.java:1613:in`each'", "/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:292:in `register_plugins'", "/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:302:in`start\_workers'", "/opt/logstash-5.5.0/logstash-core/

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 8, 2017, 5:28am UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/2 "2017-08-08T05:28:51Z")

</div>

This log entry is the interesting one:

> [2017-08-07T12:49:10,469][ERROR][logstash.agent] Pipeline aborted due to error {:exception=\>#\<LogStash::ConfigurationError: translation missing: en.logstash.agent.configuration.invalid\_plugin\_register\>, :backtrace=\>["/opt/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.5/lib/logstash/filters/mutate.rb:189:in register'", "org/jruby/RubyHash.java:1342:ineach’", “/opt/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.5/lib/logstash/filters/mutate.rb:183:in register'", "/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:281:inregister\_plugin’”, “/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:292:in register\_plugins'", "org/jruby/RubyArray.java:1613:ineach’”, “/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:292:in register\_plugins'", "/opt/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:302:instart\_workers’”, "/opt/logstash-5.5.0/logstash-core/

Since you didn't format the posted log as preformatted text it has been mangled, but look in the original log and you'll find this:

> Invalid conversion type '[integer]', expected one of 'string, integer, float, boolean'

To turn the string "67, 1, 4, 2, 8, 9, 5, 3, 10" into an array of those integers, use a mutate filter and its split option.

---

<div class="post-metadata">

### Author: ![marwa](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@marwa](https://discuss.elastic.co/u/marwa)
#### Post date: [August 8, 2017, 8:14am UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/3 "2017-08-08T08:14:42Z")

</div>

thank you for your reply  
i used mutate in the config file as bellow

filter {  
csv {  
separator =\> ","  
#transaction\_date,customer\_name,age,job,fidelity\_card\_id,product\_id\_sold,quantity  
columns =\> ["transaction\_date","customer\_name","age","job","fidelity\_card\_id","product\_id\_sold","quantity"]  
}

date {  
match =\> ["transaction\_date", "M/d/yyyy;HH:mm:ss"]  
target =\> "transaction\_date"

```
    }

```

mutate {  
convert =\>{"age" =\> "integer" }  
split =\> { "quantity" =\> "," }  
split =\> { "product\_id\_sold" =\> ","}  
}  
}

the fields are still strings have a look at the mapping in kibana

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/9/a9b6db7d024d75b96741a5542ecd224602331e50.png)

so they didn't turn into array of integers i also had this warn when loading data with logstash

[WARN][logstash.filters.csv] Error parsing csv {:field=\>"message", :source=\>"", :exception=\>#\<NoMethodError: undefined method `each\_index' for nil:NilClass\>}  
2017-08-08T08:10:19.855Z vds002.insightscale.tn transaction\_date,customer\_name,age,job,fidelity\_card\_id,product\_id\_sold,quantity

maybe the problem maintains in the format of the csv file !!!

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/e/9efc4639e7e0af9b29c5f829a03be28d9b9f9bdb.png)

this is the JSON document  
"product\_id\_sold": [  
"723",  
" 334",  
" 297",  
" 666",  
" 182",  
" 530",  
" 978",  
" 133",  
" 229",  
" 281"  
],

---

<div class="post-metadata">

### Author: ![marwa](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@marwa](https://discuss.elastic.co/u/marwa)
#### Post date: [August 8, 2017, 10:18am UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/4 "2017-08-08T10:18:17Z")

</div>

so now i only need to covert the string value into integers ,i tryed it with convert but it did not work

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 8, 2017, 11:06am UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/5 "2017-08-08T11:06:33Z")

</div>

Please show your current configuration where you try to convert the elements of `product_id_sold` to integers.

---

<div class="post-metadata">

### Author: ![marwa](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@marwa](https://discuss.elastic.co/u/marwa)
#### Post date: [August 8, 2017, 11:41am UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/6 "2017-08-08T11:41:17Z")

</div>

filter {  
csv {  
separator =\> ","  
#transaction\_date,customer\_name,age,job,fidelity\_card\_id,product\_id\_sold,quantity  
columns =\> ["transaction\_date","customer\_name","age","job","fidelity\_card\_id","product\_id\_sold","quantity"]  
}

date {  
match =\> ["transaction\_date", "M/d/yyyy;HH:mm:ss"]  
target =\> "transaction\_date"

```
    }

```

mutate {  
convert =\>{"age" =\> "integer" }  
split =\> { "quantity" =\> "," }  
split =\> { "product\_id\_sold" =\> ","}  
convert =\> {"product\_id\_sold" =\> "integer"}  
}  
}

---

<div class="post-metadata">

### Author: ![marwa](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@marwa](https://discuss.elastic.co/u/marwa)
#### Post date: [August 8, 2017, 1:43pm UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/7 "2017-08-08T13:43:25Z")

</div>

how should i fix this ?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 9, 2017, 5:36am UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/8 "2017-08-09T05:36:48Z")

</div>

Mutate actions aren't applied in the order listed in the configuration file. Their execution order is fixed, and it so happens that `convert` is applied before `split`. So, split your mutate filter into at least two to make sure the split takes place before you attempt the conversion.

> <https://github.com/logstash-plugins/logstash-filter-mutate/issues/28>

---

<div class="post-metadata">

### Author: ![marwa](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@marwa](https://discuss.elastic.co/u/marwa)
#### Post date: [August 9, 2017, 12:04pm UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/9 "2017-08-09T12:04:39Z")

</div>

thank u a lot it worked for me

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 6, 2017, 12:04pm UTC](https://discuss.elastic.co/t/configure-a-list-of-integers-with-logstash/96098/10 "2017-09-06T12:04:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
