Hello everyone and welcome to another rooky question 
In a very simple logstash pipeline, I want to enter via stdin an ipv4 address and ship it to elasticsearch...than get the geoip location drawn on a map.
However...I get no results (and no error).
Yes I read a lot..but I had no luck to solve it on my own.
What I did:
In Elasticsearch, I created an index:
PUT testme
{
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"source": {
"type": "ip"
},
"geoip": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}
}
}
Once created, I also checked the mapping:
{
"testme": {
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"client": {
"properties": {
"geo": {
"properties": {
"city_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"continent_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"properties": {
"lat": {
"type": "float"
},
"lon": {
"type": "float"
}
}
},
"postal_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timezone": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"ip": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"mmdb": {
"properties": {
"dma_code": {
"type": "long"
}
}
}
}
},
"event": {
"properties": {
"original": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"geoip": {
"properties": {
"location": {
"type": "geo_point"
}
}
},
"host": {
"properties": {
"hostname": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"source": {
"type": "ip"
}
}
}
}
}
I also created a data-view in kibana.
Here is my logstash config:
input {
stdin { }
}
filter {
grok {
match => { "message" => "%{IP:source}" }
}
geoip {
source => "source"
target => "client"
}
}
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "testme"
}
stdout {
codec => rubydebug { }
}
}
Starting logstash and entering an IP gives me this:
11.11.11.11
{
"client" => {
"ip" => "11.11.11.11",
"geo" => {
"city_name" => "Bullard",
"country_name" => "United States",
"country_iso_code" => "US",
"postal_code" => "75757",
"location" => {
"lon" => -95.3381,
"lat" => 32.1118
},
"region_iso_code" => "US-TX",
"continent_code" => "NA",
"region_name" => "Texas",
"timezone" => "America/Chicago"
},
"mmdb" => {
"dma_code" => 709
}
},
"@timestamp" => 2022-09-11T16:23:49.714266Z,
"@version" => "1",
"host" => {
"hostname" => "rocky-8-1"
},
"source" => "11.11.11.11",
"message" => "11.11.11.11",
"event" => {
"original" => "11.11.11.11"
}
}
Looks good to me....
Elasticsearch shows this content:
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "testme",
"_id": "oNNdLYMBPd5sI551QSM_",
"_score": 1,
"_source": {
"client": {
"ip": "11.11.11.11",
"geo": {
"city_name": "Bullard",
"country_name": "United States",
"country_iso_code": "US",
"postal_code": "75757",
"location": {
"lon": -95.3381,
"lat": 32.1118
},
"region_iso_code": "US-TX",
"continent_code": "NA",
"region_name": "Texas",
"timezone": "America/Chicago"
},
"mmdb": {
"dma_code": 709
}
},
"@timestamp": "2022-09-11T16:23:49.714266Z",
"@version": "1",
"host": {
"hostname": "rocky-8-1"
},
"source": "11.11.11.11",
"message": "11.11.11.11",
"event": {
"original": "11.11.11.11"
}
}
},
{
"_index": "testme",
"_id": "ntNaLYMBPd5sI551KiPj",
"_score": 1,
"_source": {
"client": {
"geo": {
"timezone": "Europe/Berlin",
"location": {
"lon": 9.491,
"lat": 51.2993
},
"continent_code": "EU",
"country_name": "Germany",
"country_iso_code": "DE"
},
"ip": "5.4.3.2"
},
"@timestamp": "2022-09-11T16:20:27.205240Z",
"@version": "1",
"host": {
"hostname": "rocky-8-1"
},
"source": "5.4.3.2",
"message": "5.4.3.2",
"event": {
"original": "5.4.3.2"
}
}
},
{
"_index": "testme",
"_id": "n9NaLYMBPd5sI551ayN8",
"_score": 1,
"_source": {
"client": {
"geo": {
"timezone": "America/Chicago",
"location": {
"lon": -97.822,
"lat": 37.751
},
"continent_code": "NA",
"country_name": "United States",
"country_iso_code": "US"
},
"ip": "4.3.2.1"
},
"@timestamp": "2022-09-11T16:20:43.906904Z",
"@version": "1",
"host": {
"hostname": "rocky-8-1"
},
"source": "4.3.2.1",
"message": "4.3.2.1",
"event": {
"original": "4.3.2.1"
}
}
}
]
}
}
In Kibana, I see the geo.location field...MAPS shows me the countries based on the 2 field code if I want...but only the geo.location delivers nothing ...(time range is set correctly).
So after 1.5 days of testing and reading...I need help. 1000x Kudos to the chosen one who helps me out.
I feel like a teen right now (I'm old !), thank you so much Stephen...you took your time to explain something to a stranger on a sunday. I really really appreciate that.
not only exact matches... I will bring that up again!