IP getting mapped as string

Elastic Stack Elasticsearch
1

Hi,

I am new with ELK. I am trying to load a log file via logstash; my log file is in json format. I am able to load it in Elastic, however on closely looking i found out that the ip field is loaded as string instead of IP datatype. I have tried multiple things like using mutate function, template updation ; but nothing seems to be working. Would really appreciate if somebody can help me with this

My config file:

input {
file {
codec => json
path => "C:\Users\asing178\Downloads\Log\DBAudit.log-2017-04-14-001.json"
start_position => "beginning"
codec => json
sincedb_path => "/dev/null"
}
}

output {
elasticsearch {
hosts => "http://localhost:9200"
#user=>"elastic"
#password=>"123456"
index => "bi_hadoop_log_abhi"
document_type => "log"

}
stdout { }
}

Json data sample:

{"timestamp":{"$date":"2017-04-12T06:37:44.572Z"},"operation":"DB_SCAN","user":"sramakr2","uid":43826,"ipAddress":"10.205.82.141","VolumeName":"uhclake_rx1_snapshot","volumeId":196457823,"columnFamily":"ci","columnQualifier":"snapshot_crt_ts","tablePath":"/datalake/uhclake/tst/t_hdfs/optum/Enriched/standard_access/current_snapshot/rx1/data/CLMPRDEXT2_RCEX1P_snaphbase","tableFid":"199722.16724.1003498","status":0}

my updated template file (I have overwritten the default template file)

{
"template" : "logstash-",
"version" : 50001,
"settings" : {
"index.refresh_interval" : "5s"
},
"mappings" : {
"default" : {
"_all" : {"enabled" : true, "norms" : false},
"dynamic_templates" : [ {
"message_field" : {
"path_match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text",
"norms" : false
}
}
}, {
"string_fields" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text", "norms" : false,
"fields" : {
"keyword" : { "type": "keyword" }
}
}
}
} ],
"properties" : {
"@timestamp": { "type": "date", "include_in_all": false },
"@version": { "type": "keyword", "include_in_all": false },
"ipAddress": { "type": "ip" },
"geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "half_float" },
"longitude" : { "type" : "half_float" }
}
}
}
}
}
}

Template file location: C:\Users\asing178\Downloads\logstash-5.2.1\vendor\bundle\jruby\1.9\gems\logstash-output-elasticsearch-6.2.6-java\lib\logstash\outputs\elasticsearch\elasticsearch-template-es5x.json

Kindly let me know if any other detail is required

2

You aren't using the geoip filter on the ipAddress field, so it will never create that info.

However what does it end up being mapped as?

3

Thanks Mark...my ip field is getting mapped as string in my indexes. So you are suggesting that i need to use geoip filter in my config

4

The issue is resolved now; I have used the explicit mapping for the index where I have specified the field type as ip and then ran the logstash

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.