Unable to find source.geo.location in index pattern logstash-*

Elastic Stack Elasticsearch
1

Got the above error when trying to view map under Security>Network in Elastic.

Index mapping

  "mappings" : {

	"geoip" : {
          "dynamic" : "true",
          "properties" : {
            "city_name" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "continent_code" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "country_code2" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "country_code3" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "country_name" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "dma_code" : {
              "type" : "long"
            },
            "ip" : {
              "type" : "ip"
            },
            "latitude" : {
              "type" : "half_float"
            },
            "location" : {
              "type" : "geo_point"
            },
            "longitude" : {
              "type" : "half_float"
            },
            "postal_code" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "region_code" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "region_name" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            },
            "timezone" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              },
              "norms" : false
            }
          }
        }
		
	  }
2

Hi @Ronnie_Raraihuru

Question are you ingesting from one of the beats modules through logstash or is this just a custom source? I ask because if it is a module I have one suggestion, if it is a custom source then I will have different device.

I can tell you in short you are ingesting data not in the correct fields if you want them to show up in in the security app the need to be in the right fields

See Here

Your mapping is missing the network fields and location fields

Your mapping is not correction the geoip is at the wrong level these should be at the top level.

  • @timestamp
  • destination.geo.location (required for displaying map data)
  • destination.ip
  • source.geo.location (required for displaying map data)
  • source.ip
3

Thanks Stephen ,
I am ingesting through logstash. ( Filebeat >Logstash>Elastic)
How to i add the destination.geo.location and source.geo.location to my current mapping ?

Hear from you.

4

Are you using a predefined module? like ngnix or PANOS etc? You did not answer... it may be much simpler.

Otherwise, You need to create a mapping ahead of time something like the following and you should use an index template so ever index automatically get the mapping.

Something like this... this should get you started, follow the pattern and the docs

PUT _index_template/my-data
{
  "index_patterns": [
    "my-data-*"
  ],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "lifecycle": {
        "name": "my-data",
        "rollover_alias": "my-data"
      }
    },
    "aliases": {},
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "@version": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "event": {
          "properties": {
            "dataset": {
              "type": "keyword"
            },
            "category": {
              "type": "keyword"
            }
          }
        },
        "host": {
          "properties": {
            "name": {
              "type": "keyword"
            },
            "ip": {
              "type": "ip"
            },
            "os": {
              "properties": {
                "name": {
                  "type": "keyword"
                },
                "version": {
                  "type": "keyword"
                }
              }
            }
          }
        },
        "message": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "source": {
          "properties": {
            "ip": {
              "type": "ip"
            },
            "geo": {
              "properties": {
                "location": {
                  "type": "geo_point"
                }
              }
            }
          }
        },
        "destination": {
          "properties": {
            "ip": {
              "type": "ip"
            },
            "geo": {
              "properties": {
                "location": {
                  "type": "geo_point"
                }
              }
            }
          }
        }
      }
    }
  }
}
5

Thanks Stephen ,
Have not used a predefined module.
Will get started with the patterns you have provide like wise for the index template

Thanks

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.