Help with filter to create Latitude and Longitude fields

Elastic Stack Logstash
1

Hello,

Can I please have some help with a filter for this sample data?

{
    "took": 319,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1784,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "index_for_some_data",
                "_id": "dkgirndy58gmdu69fys5",
                "_score": 1.0,
                "_source": {
                    "event": {
                        "original": "Return-Path: <some_one@some_where.com>\r\nX-Original-To: mailbox@needed.local\r\nReceived: from internal.host.local (internal.host.local  ..  [SNIP]  ..  \r\nSubject: Something of interset to us\r\n  ..  [SNIP]  ..  "
                    },
                    "message": "Return-Path: <some_one@some_where.com>\r\nX-Original-To: mailbox@needed.local\r\nReceived: from internal.host.local (internal.host.local  ..  [SNIP]  ..  \r\nSubject: Something of interset to us\r\n  ..  [SNIP]  ..  \r\n\r\nUnit Location: Lat = -31.80355 Long = 115.84676\r\n  ..  [SNIP]  ..  ",
                    "@version": "1",
                    "tags": [
                        "RabbitMQ",
                        "SMTP"
                    ],
                    "@timestamp": "2024-08-08T03:30:31.158337781Z"
                }
            },
..  [SNIP]  ..

Filter currently being used thanks to some help with my previous post looks like this.

filter {
  mutate { remove_field => [ "event", "host", "log" ] }
  grok {
    break_on_match => false
    match => {
      "message" => [
        "^X-Original-To: %{EMAILADDRESS:email_to}",
        "^Subject: %{DATA:email_subject}
" ] }
  }
}

In addition to email_to and email_subject, we would also like fields for unit_location_latitude and unit_location_longitude as well.

Cheers,
Eddie.

2

You can trivially add patterns to match the latitude and longitude.

            "message" => [
                "^X-Original-To: %{EMAILADDRESS:email_to}",
                "^Subject: %{DATA:email_subject}
",
                "Lat = %{NUMBER:latitude:float}",
                "Long = %{NUMBER:longitude:float}"
            ]

If you want to combine those to create a geo_point in elasticsearch then you will need a template.