# GeoIp based on custom field source.ip

**URL:** <https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [July 11, 2023, 10:08am UTC](https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088 "2023-07-11T10:08:43Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vasile](https://avatars.discourse-cdn.com/v4/letter/v/53a042/32.png) [@vasile](https://discuss.elastic.co/u/vasile)\
**Post date:** [July 11, 2023, 10:08am UTC](https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088/1 "2023-07-11T10:08:43Z")

</div>

Hi all,

I am trying to parse a log message. The original log looks like this:

```auto
Jul 10 08:51:10 prometheus sshd[19074]: Accepted password for my_user from 1.1.1.1 port 1111 ssh2

```

My filebeat conf is bellow:

```auto
---
filebeat.inputs:
  - type: filestream
    id: default-filestream
    paths:
      - ingest_data/*.log
      - /var/log/auth.log
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
processors:
  - add_docker_metadata: null
  - drop_event:
      when:
        not.contains:
          message: Accepted
  - dissect:
      tokenizer: "%{} %{} %{} prometheus sshd[%{pid|integer}]: Accepted password for %{service.user} from %{source.ip} port %{source.port} ssh2"
      field: "message"
      target_prefix: ""
      overwrite_keys: true

setup.kibana:
  host: ${KIBANA_HOSTS}
  username: ${ELASTIC_USER}
  password: ${ELASTIC_PASSWORD}
output.elasticsearch:
  hosts: ${ELASTIC_HOSTS}
  username: ${ELASTIC_USER}
  password: ${ELASTIC_PASSWORD}
  ssl.enabled: true
  ssl.certificate_authorities: certs/ca/ca.crt
  pipeline: geoip-info

```

I would like to have all the fields for the geoip enrichment.  
The geoip is taken from the default processor ==\> [geoip](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-geoip.html)

Basicity I plan to do the geolocation based on this custom field `source.ip` field from the log/message.

What I am doing wrong?

---

<div class="post-metadata">

**Author:** ![vasile](https://avatars.discourse-cdn.com/v4/letter/v/53a042/32.png) [@vasile](https://discuss.elastic.co/u/vasile)\
**Post date:** [July 11, 2023, 12:39pm UTC](https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088/2 "2023-07-11T12:39:28Z")

</div>

The error seems:

```auto
{\\\\\\\"type\\\\\\\":\\\\\\\"illegal_argument_exception\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"pipeline with id [geoip-info] does not exist\

```

---

<div class="post-metadata">

**Author:** ![btsinfo](https://avatars.discourse-cdn.com/v4/letter/b/ecccb3/32.png) [@btsinfo](https://discuss.elastic.co/u/btsinfo)\
**Post date:** [July 11, 2023, 12:49pm UTC](https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088/3 "2023-07-11T12:49:51Z")

</div>

> [@vasile](#):
>
> `geoip-info`

You need to create a "geoip-info" pipeline:

PUT \_ingest/pipeline/geoip-info  
{  
"description": "Add geoip info",  
"processors": [  
{  
"geoip": {  
"field": "client.ip",  
"target\_field": "client.geo",  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"database\_file": "GeoLite2-ASN.mmdb",  
"field": "client.ip",  
"target\_field": "client.as",  
"properties": [  
"asn",  
"organization\_name"  
],  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"field": "source.ip",  
"target\_field": "source.geo",  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"database\_file": "GeoLite2-ASN.mmdb",  
"field": "source.ip",  
"target\_field": "source.as",  
"properties": [  
"asn",  
"organization\_name"  
],  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"field": "destination.ip",  
"target\_field": "destination.geo",  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"database\_file": "GeoLite2-ASN.mmdb",  
"field": "destination.ip",  
"target\_field": "destination.as",  
"properties": [  
"asn",  
"organization\_name"  
],  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"field": "server.ip",  
"target\_field": "server.geo",  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"database\_file": "GeoLite2-ASN.mmdb",  
"field": "server.ip",  
"target\_field": "server.as",  
"properties": [  
"asn",  
"organization\_name"  
],  
"ignore\_missing": true  
}  
},  
{  
"geoip": {  
"field": "host.ip",  
"target\_field": "host.geo",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "server.as.asn",  
"target\_field": "server.as.number",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "server.as.organization\_name",  
"target\_field": "server.as.organization.name",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "client.as.asn",  
"target\_field": "client.as.number",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "client.as.organization\_name",  
"target\_field": "client.as.organization.name",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "source.as.asn",  
"target\_field": "source.as.number",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "source.as.organization\_name",  
"target\_field": "source.as.organization.name",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "destination.as.asn",  
"target\_field": "destination.as.number",  
"ignore\_missing": true  
}  
},  
{  
"rename": {  
"field": "destination.as.organization\_name",  
"target\_field": "destination.as.organization.name",  
"ignore\_missing": true  
}  
}  
]  
}

---

<div class="post-metadata">

**Author:** ![vasile](https://avatars.discourse-cdn.com/v4/letter/v/53a042/32.png) [@vasile](https://discuss.elastic.co/u/vasile)\
**Post date:** [July 13, 2023, 7:09am UTC](https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088/4 "2023-07-13T07:09:24Z")

</div>

Thank you! I did! I added the pipeline, but despite it exists, the logs from filebeat says it doesn't exist as stated in the comment. So the pipeline is in the kibana/es and is not seen by filebeat.

---

<div class="post-metadata">

**Author:** ![vasile](https://avatars.discourse-cdn.com/v4/letter/v/53a042/32.png) [@vasile](https://discuss.elastic.co/u/vasile)\
**Post date:** [July 13, 2023, 7:27am UTC](https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088/5 "2023-07-13T07:27:41Z")

</div>

I am retrieving the source.ip from the log file. It might be changed the order?

In my case I am:

1. taking the log
2. retrieve source.ip
3. send for geolocation

It might be a different order for the pipeline?

1. take the log
2. do geolocation
3. retrieve source.ip

Because the field doesn't exist while taking the log the geolocation is not done?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 10, 2023, 9:28am UTC](https://discuss.elastic.co/t/geoip-based-on-custom-field-source-ip/338088/6 "2023-08-10T09:28:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.