# LogStash Imports Numbers as Strings into Elastic **URL:** \ **Category:** Logstash\ **Created:** [April 12, 2016, 1:58pm UTC](https://discuss.elastic.co/t/logstash-imports-numbers-as-strings-into-elastic/47126 "2016-04-12T13:58:11Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![wstrater](https://avatars.discourse-cdn.com/v4/letter/w/b9e5f3/32.png) [@wstrater](https://discuss.elastic.co/u/wstrater)\ **Post date:** [April 12, 2016, 1:58pm UTC](https://discuss.elastic.co/t/logstash-imports-numbers-as-strings-into-elastic/47126/1 "2016-04-12T13:58:11Z")
I am struggling with importing fields into Elastic with the correct format. I am getting numbers stored as string and strings stored as IP addresses and wonder if it is my template. I eventually gave up in getting IP address stored as strings and started dropping the field if it does not match the IPv4 format since Elastic was rejecting the events. I delete the indicies from Elastic and Kafka settings page. ``` $ curl -XDELETE 'http://localhost:9200/access_log-*' {"acknowledged":true} ``` I import the template. I have switched clientIP back to ip but tried to import it as string. ``` $ curl -XPUT 'http://localhost:9200/_template/access_log?pretty' -d@/etc/elasticsearch/scripts/access_log.template.json { "acknowledged" : true } $ curl -s 'localhost:9200/_template/access_log?pretty' | egrep -A2 "bytes|clientIP|status" "clientIP" : { "index" : "not_analyzed", "type" : "ip" -- "bytes" : { "index" : "not_analyzed", "type" : "long" -- "status" : { "index" : "not_analyzed", "type" : "long" ``` I edit and check the config. ``` $ vi /etc/logstash/conf.d/kafka/config.json $ /opt/logstash/bin/logstash -t -f /etc/logstash/conf.d/kafka/config.json Configuration OK ``` Here is the grok for the message. I have switched from using IPORHOST to DATA since our security scans are putting garbage in the first to fields. ``` "message" => "%{DATA:serverName} %{DATA:clientIP} %{QS:xForwardedFor} (?:%{USER:ident}|-) (?:%{USER:auth}|-) \[%{HTTPDATE:timeStamp}\] \"(?:%{WORD:method} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpVersion})?|%{DATA:rawRequest})\" %{NUMBER:status} (?:%{NUMBER:bytes}|-) (?:%{NUMBER:duration}|-) (?:%{DATA:browserTrackingId}|-) %{QS:referrer} %{QS:agent}" ``` I convert the numbers to integer. ``` mutate { convert => { "bytes" => "integer" "duration" => "integer" "status" => "integer" } } ``` I output the event to Elastic. ``` elasticsearch { hosts => [...] manage_template => false index => "access_log-%{+YYYY.MM.dd}" document_type => "access_log" } ``` When I add start LogStash and the index back into Kafka I see the bytes and status as string on the Kafka settings page and have a t next to them on the discover page. Here is the template with most of the fields removed for brevity. I import multiple templates and the section from _default_ to properties is the same for all of them. Not sure if that is an issue. ``` { "template": "access_log-*", "order" : 0, "mappings": { "_default_": { "_all": { "enabled": false }, "dynamic_templates": [ { "template1": { "mapping": { "doc_values": true, "ignore_above": 1024, "index": "not_analyzed", "type": "{dynamic_type}" }, "match": "*" } } ], "properties": { "@timestamp": { "type": "date", "format": "strict_date_optional_time||epoch_millis" }, "@version": { "type": "string" }, ... "bytes": { "type": "long", "index": "not_analyzed" }, "clientIP": { "type": "ip", "index": "not_analyzed" }, "count": { "type": "long" }, "duration": { "type": "long", "index": "not_analyzed" }, ... "status": { "type": "long", "index": "not_analyzed" }, ... } } } } ``` I would appreciate any help, Wes. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 13, 2016, 1:53am UTC](https://discuss.elastic.co/t/logstash-imports-numbers-as-strings-into-elastic/47126/2 "2016-04-13T01:53:19Z")
What's the mapping (not template) look like after you ingest the data? ---
**Author:** ![wstrater](https://avatars.discourse-cdn.com/v4/letter/w/b9e5f3/32.png) [@wstrater](https://discuss.elastic.co/u/wstrater)\ **Post date:** [April 21, 2016, 9:14pm UTC](https://discuss.elastic.co/t/logstash-imports-numbers-as-strings-into-elastic/47126/3 "2016-04-21T21:14:20Z")
Sorry for not getting back to you earlier. I am not too sure what you mean by mapping but when I look at Indicies under Settings I see the type for status is String. When I look in Discover I see status of 200 with a 't' next to it. I exported the template again and it is still long. ``` # curl -s 'localhost:9200/_template/access_log?pretty' | egrep -A2 "bytes|clientIP|status" "clientIP" : { "index" : "not_analyzed", "type" : "ip" -- "bytes" : { "index" : "not_analyzed", "type" : "long" -- "status" : { "index" : "not_analyzed", "type" : "long" ``` I started using the Ruby filter more when I ran into issues with field names while trying to use the kv filter on the query string and parameter names had illegal characters. I am now using Ruby to convert strings to numbers. ``` @number_arr = ['bytes', 'duration', 'status']; @number_re = Regexp.new('^[+-]?[0-9]+$'); @number_arr.each { |field| event[field] = event[field].to_i if @number_re.match(event[field]) }; ``` Wes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:01am UTC](https://discuss.elastic.co/t/logstash-imports-numbers-as-strings-into-elastic/47126/4 "2017-07-06T05:01:13Z")