How to handle fields containg ipv4 OR ipv6 (actually only ::1 - ie. localhost)?


(Klavs Klavsen) #1

elasticsearch seems to go bananas when it gets data set to be ip - but
containing "::1".. :frowning:

I can see that the IP type does not support ipv6 yet.. but I would like to
be able to do range stuff on ip's..

Is there any way, I can make elasticsearch handle this better? (clientip
field is mapped to be of type ip).

it keeps logging:
[2014-02-17 10:20:25,580][DEBUG][action.bulk ] [Creed, Victor]
[logstash-2014.02.16][0] failed to execute bulk item (index) index
{[logstash-2014.02.16][apache][+sn/qjkJWZxrfuuYZdS5yyXAsFk=],
source[{"message":"::1 - - [16/Feb/2014:15:49:08 +0100] "OPTIONS *
HTTP/1.0" 200 - 0
51","@version":"1","@timestamp":"2014-02-16T15:49:08.000+01:00","type":"apache","tags":["mysite.example.org","apache"],"host":"p-asu-web02","path":"/var/log/httpd/http.mysite.example.org-access.log","message_id":"+sn/qjkJWZxrfuuYZdS5yyXAsFk=","clientip":"::1","ident":"-","auth":"-","timestamp":"16/Feb/2014:15:49:08
+0100","verb":"OPTIONS","request":"*","httpversion":"1.0","response":"200","time_used_secs":"0","time_used_usecs":"51"}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[clientip]
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:401)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:613)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:466)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:516)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:460)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:374)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:397)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:156)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticSearchIllegalArgumentException: failed
to parse ip [::1], not full ip address (4 dots)
at
org.elasticsearch.index.mapper.ip.IpFieldMapper.ipToLong(IpFieldMapper.java:82)
at
org.elasticsearch.index.mapper.ip.IpFieldMapper.innerParseCreateField(IpFieldMapper.java:291)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:194)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:390)
... 12 more

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4a72d2e2-6de8-49ee-845f-971f78f15685%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Binh Ly) #2

Yes, unfortunately it has to be a valid IPv4 address. For now, you can
probably do a replace from the LS side to translate ::1 to something like
127.0.0.1. For example:

filter {
mutate {
gsub => [ "clientip", "::1", "127.0.0.1" ]
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0011e47c-a9aa-43fc-92a9-75fecab63df3%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #3