# Unable to find source.geo.location filebeat aws module

**URL:** <https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 24, 2021, 6:26pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254 "2021-03-24T18:26:05Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![EvanGertis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evangertis/32/85191_2.png) [@EvanGertis](https://discuss.elastic.co/u/EvanGertis)\
**Post date:** [March 24, 2021, 6:26pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/1 "2021-03-24T18:26:05Z")

</div>

I would like to convert the following to a geo\_point type

```auto
           "location": {
              "properties": {
                "lat": {
                  "type": "float"
                },
                "lon": {
                  "type": "float"
                }
              }
            }

```

The issue that I am running into is that the field source.geo.location is not of type geo\_point. I would like to modify the location mapping to look like this

```auto
             "location": {
              "type": "geo_point"
              "properties": {
                "lat": {
                  "type": "float"
                },
                "lon": {
                  "type": "float"
                }
              }

```

I have reviewed this issue here: [Unable to find source.geo.location in index pattern logstash-\*](https://discuss.elastic.co/t/unable-to-find-source-geo-location-in-index-pattern-logstash/264496), but it doesn't answer my question. I am using the preconfigured elastic common schema from the aws module. This particular issue is related to the vpc flow logs visualization. I have attached a screen shot for extra clarity.

 ![Screen Shot 2021-03-24 at 2.07.34 PM](https://us1.discourse-cdn.com/elastic/original/3X/5/2/5221496556067d78d02bb14d30b7930f22a9b0cb.jpeg)  
My attempt at fixing the problem

```auto
PUT /vpc-7.10.2-2021.03.05/_mapping
{
  "mappings" : {
    "properties": {
      "source.geo.location": {
        "type": "geo_point"
      }
    }
  }
}

```

this just produced

```auto
{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Root mapping definition has unsupported parameters: [mappings : {properties={source.geo.location={type=geo_point}}}]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Root mapping definition has unsupported parameters: [mappings : {properties={source.geo.location={type=geo_point}}}]"
  },
  "status" : 400
}

```

This doesn't work either

```auto
PUT /vpc-7.10.2-2021.03.05/_mapping
{
  "mappings" : {
    "properties": {
      "location": {
        "type": "geo_point"
      }
    }
  }
}

```

This does not help either

```auto
PUT /vpc-7.10.2-2021.03.05/_mapping
{
    "properties": {
      "location": {
        "type": "geo_point"
      }
    }
}

```

This one doesn't do it either

```auto
PUT /vpc-*/_mapping
{
  "source": {
    "geo":{
      "location":{
        "properties":{
          "type": "geo_point"
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 24, 2021, 7:26pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/2 "2021-03-24T19:26:58Z")

</div>

You were close... 🙂

first when you use the` _mapping` endpoint... you must `PUT` (create) the index first then apply the mapping.

This syntax does both creates the index and applies the mapping, of course you will end up putting this in a `_template` at some point

```
PUT /vpc-7.10.2-2021.03.05/
{
  "mappings": {
    "properties": {
      "source": {
        "properties": {
          "geo": {
            "properties": {
              "location": {
                "type": "geo_point"
              }
            }
          }
        }
      }
    }
  }
}

```

your shorthand will work... but the syntax above is "more correct / descriptive" it will be what show when you do a `GET` on the index

```auto
PUT /vpc-7.10.2-2021.03.05/
{
  "mappings" : {
    "properties": {
      "source.geo.location": {
        "type": "geo_point"
      }
    }
  }
}

```

This will not automatically fix your issue you will need to map... I am not sure of the issue...

First is the data in the actual source

2nd if the flow logs are only internal IPs then there will be no sou`rce.geo.location` data.

---

<div class="post-metadata">

**Author:** ![EvanGertis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evangertis/32/85191_2.png) [@EvanGertis](https://discuss.elastic.co/u/EvanGertis)\
**Post date:** [March 24, 2021, 7:32pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/3 "2021-03-24T19:32:57Z")

</div>

> [@stephenb](#):
>
> ```auto
> PUT /vpc-7.10.2-2021.03.05/
> {
> "mappings" : {
> "properties": {
> "source.geo.location": {
> "type": "geo_point"
> }
> }
> }
> }
> 
> ```

Thank you for the response. I am still facing an issue updating the index. Will this request create a new mapping? I only ask because the index already exists. This is the error that I receive

```auto
{
  "error" : {
    "root_cause" : [
      {
        "type" : "resource_already_exists_exception",
        "reason" : "index [vpc-7.10.2-2021.03.05/638qM2S5ToqEuEX0PGCCLA] already exists",
        "index_uuid" : "638qM2S5ToqEuEX0PGCCLA",
        "index" : "vpc-7.10.2-2021.03.05"
      }
    ],
    "type" : "resource_already_exists_exception",
    "reason" : "index [vpc-7.10.2-2021.03.05/638qM2S5ToqEuEX0PGCCLA] already exists",
    "index_uuid" : "638qM2S5ToqEuEX0PGCCLA",
    "index" : "vpc-7.10.2-2021.03.05"
  },
  "status" : 400
}

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 24, 2021, 7:36pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/4 "2021-03-24T19:36:23Z")

</div>

You can not update / change the mapping of existing fields in an existing index.

You can only add new fields or create the mapping in a new index and reindex that data into that.

OR fix the template and you ingest so your data into the correct index and mapping in the first place.

There is no way to just fix / change that mapping and fields in an existing index / documents.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 24, 2021, 7:53pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/5 "2021-03-24T19:53:44Z")

</div>

@EvanGertis

Just a little detail / subtlety on that error

Because you tried the all in one syntax

```auto
PUT /vpc-7.10.2-2021.03.05/
{
  "mappings" : {

```

You are actually trying to create the index with the same name... that is the error above.

```auto
"type" : "resource_already_exists_exception",

```

If you used the explicit mapping syntax

```
PUT /vpc-7.10.2-2021.03.05/_mapping
{
    "properties": {
      "source": {
        "properties": {
...

```

You would get an mapping exception like

```
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "mapper [source.geo.location] cannot be changed from type [float] to [geo_point]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "mapper [source.geo.location] cannot be changed from type [float] to [geo_point]"
  },
  "status" : 400
}
```

---

<div class="post-metadata">

**Author:** ![EvanGertis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evangertis/32/85191_2.png) [@EvanGertis](https://discuss.elastic.co/u/EvanGertis)\
**Post date:** [March 29, 2021, 4:06pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/6 "2021-03-29T16:06:40Z")

</div>

@stephenb is there an issue with using multiple indicies for a filebeat configuration like

```auto
indices:
  - index: "cloudtrail-%{[agent.version]}-%{+yyyy.MM.dd}"
    when.contains:
      event.dataset: "aws.cloudtrail"
  - index: "elb-%{[agent.version]}-%{+yyyy.MM.dd}"
    when.contains:
      event.dataset: "aws.elb"
  - index: "vpc-%{[agent.version]}-%{+yyyy.MM.dd}"
    when.contains:
      event.dataset: "aws.vpc"

```

Even after I add the field I'm still running into issues with missing field for elb-\* and vpc-\*. I thought that the fields would be generated through ECS by default?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 29, 2021, 5:35pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/7 "2021-03-29T17:35:55Z")

</div>

Hi @EvanGertis

Apologies... I think I have lost track of the overall goal.

Are you using the AWS Module? Do you want to use the module?

It seems that you want to use the module but customize all the indices etc... which may results in a number of unintended consequences.

There are a number of moving parts of a modules (e.g. AWS module) which may include : inputs, outputs, templates(mappins), index names, ingest pipelines, dashboards, visualizations, Index Lifecycle Management etc... all need to be aligned to get the desired results.

This is why they are modules 🙂

Once you start changing things the proper template may (will) not be use (probably your initial issues, an ongoing issue) and then pipeline (may or may not be properly applied

That is not to say you can not... it just you need to understand all the parts and relationships

Generally I suggest users use all the defaults to start... get that all working and then start to customize unless you already understand all the components and pieces and their relationships.

i.e.  
(Cleanup if need be)  
Just configure the modules endpoint the aws endpoints / creds perhaps for a single type  
Configure the output default.

`filebeat setup -e`  
then run  
`filebeat -e`  
and see if it all works.

Then if you want to change all the indices name etc, you will need to modify the template so it gets applied to the new indices name, then make sure the pipelines are getting called

> [@EvanGertis](#):
>
> Even after I add the field I'm still running into issues with missing field for elb-\* and vpc-\*. I thought that the fields would be generated through ECS by default?

All that said, I am unclear what is missing what is the problem? Is it not sorting to the new index names? and Yes if you are sending to new indices without setting up the correct templates and pipelines the data will not be parsed and created with the correct fields / the way you want.

In a meta sense you would need to add your index patterns to the filebeat template and then make sure `manage_template : false` from then on or it will get over written... this is just an example but that is an approach.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 26, 2021, 7:36pm UTC](https://discuss.elastic.co/t/unable-to-find-source-geo-location-filebeat-aws-module/268254/8 "2021-04-26T19:36:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.