# No data for query "exists": { "field": "event.original" }

**URL:** https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355
**Category:** Elasticsearch
**Created:** [February 5, 2022, 5:42am UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355 "2022-02-05T05:42:53Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Slavik\_Fursov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slavik_fursov/32/48975_2.png) [@Slavik\_Fursov](https://discuss.elastic.co/u/Slavik_Fursov)
#### Post date: [February 5, 2022, 5:42am UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/1 "2022-02-05T05:42:53Z")

</div>

I'm sending web access logs from Nginx server to Elastic. Everything works.  
I can see all data in Kibana correctly:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/9/f915de52502190ca13ae94bf0eb70cd84751152e.jpeg)

However, when I try to add  
"exists": { "field": "event.original" }  
I'm getting no data:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/e/5e16826ce292cb598779528b546484109bcf416d.jpeg)

What am I missing?  
Looks like Elasticsearch bug.

However, same query for other fields is working:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/8/289540b0eb952ec0c6f78190aa211b155350cd4e.jpeg)

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [February 6, 2022, 10:31pm UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/2 "2022-02-06T22:31:03Z")

</div>

> [@Slavik\_Fursov](#):
>
> "exists": { "field": "event.original" }

Can you try `"exists": { "field": "event.original.keyword" }` and see if that works?

---

<div class="post-metadata">

### Author: ![BenB196](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benb196/32/83401_2.png) [@BenB196](https://discuss.elastic.co/u/BenB196)
#### Post date: [February 6, 2022, 11:48pm UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/3 "2022-02-06T23:48:54Z")

</div>

If you are ingesting with Filebeat, the default mapping for `event.original` is:

```auto
"original": {
  "ignore_above": 1024,
  "index": false,
  "type": "keyword",
  "doc_values": false
}

```

The `"index": false` means that the field can't be used for a filter, and only can retrieved from `_source`.

You might want to look at the `message` field, generally it contains the same information, and is indexed by default.

---

<div class="post-metadata">

### Author: ![Slavik\_Fursov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slavik_fursov/32/48975_2.png) [@Slavik\_Fursov](https://discuss.elastic.co/u/Slavik_Fursov)
#### Post date: [February 7, 2022, 2:32am UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/4 "2022-02-07T02:32:18Z")

</div>

Tried.  
No, doesn't work. Getting "No results match you search criteria"...

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [February 7, 2022, 3:11am UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/5 "2022-02-07T03:11:35Z")

</div>

Let's back up a bit..what are you trying to accomplish?

The `event.original` is parsed up into all the separate fields... Those can be searched on.. the `event.original` can be returned if needed from source as @BenB196 stated.

If you are trying to find docs that don't have `event.original` you won't have any of the other fields.. so pick one of those other fields to test for do not exist..

BTW pretty sure that ngnix logs do not have a `message` field

List of fields...

> **[Nginx fields | Filebeat Reference \[8.0\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/8.0/exported-fields-nginx.html)**

So back to my first question...what are you trying to accomplish / figure out?

---

<div class="post-metadata">

### Author: ![Slavik\_Fursov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slavik_fursov/32/48975_2.png) [@Slavik\_Fursov](https://discuss.elastic.co/u/Slavik_Fursov)
#### Post date: [February 7, 2022, 4:10am UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/6 "2022-02-07T04:10:03Z")

</div>

There is no `message` field. I see, that nginx's module creates pipeline, which actually renames `message` to `event.original`:  
[pipeline.yml](https://github.com/elastic/beats/blob/main/filebeat/module/nginx/access/ingest/pipeline.yml)

```auto
- rename:
    field: message
    target_field: event.original

```

Generally, what I'm trying to do is to build Web Analytics dashboard, using, non-Elastic application.  
I already using Filebeat to send all web access logs to Elastic.  
I did basic web analytics dashboard, but ES has big limitations for web analytics:

- I can't get session counts, session length, returning users. etc...
- There is no good set of filter to filter out all bots in the logs. I can do basic one, but it's moving target.

So, I decided, that I'll create daily cron job to pull all web access data from ES and feed it to Matomo. I only pull `event.original` field.  
It generally works, but the shell scripts I wrote to pull that data sometimes fails, because some of the events doesn't have "event.source". So, I tried to write the query to make sure, that events I pull have `event.original` but such query fails.

Here is the daily cron script:

```auto
NAME="meshumad.com-$DATE"
curl -XGET "https:// ***.westus2.azure.elastic-cloud.com:9243/filebeat-*/_search?size=10000" -u elastic:*** -H 'Content-Type: application/json' -d'
{
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "gte": "'$DATE'T00:00:00",
              "lte": "'$DATE'T23:59:59"
            }
          }
        },
        {
          "match_phrase": {
            "container.name": "meshumad.com"
          }
        },
        {
          "match_phrase": {
            "http.request.method": "GET"
          }
        },
        {
          "exists": {
            "field": "source.ip"
          }
        },
       {
         "exists": {
           "field": "event.original"
         }
      }
      ],
      "must_not": [
        {
          "match_phrase": {
            "url.original": "/csrftoken"
          }
        }
      ]
    }
  },
  "fields": [
    "event.original"
  ],
  "sort": [
    {
      "@timestamp": "asc"
    }
  ],
  "_source": false
}' > $NAME.raw
cat $NAME.raw | jq -r '.hits.hits[] | if has("fields") then .fields."event.original"[] else .ignored_field_values."event.original"[] end' > $NAME.log
sudo docker exec matomo-app python3 /var/www/html/misc/log-analytics/import_logs.py --url=http://localhost --login=slavik --password=*** --add-sites-new-hosts --recorders=1 /import/*.log

```

The error I was getting:

> jq: error (at :0): Cannot iterate over null (null)

And when I debugged I found, that `jq` fails because some of entries doesn't have `event.original` nor `ignored_field_values."event.original"`.

For example I found this entry in the the result:

> {"\_index":"filebeat-7.9.0-2022.01.08-000017","\_type":"\_doc","\_id":"lYzMsn4BnBx-UCJhOtim","\_score":null,"sort":[1643677228000]}

---

<div class="post-metadata">

### Author: ![Slavik\_Fursov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slavik_fursov/32/48975_2.png) [@Slavik\_Fursov](https://discuss.elastic.co/u/Slavik_Fursov)
#### Post date: [February 7, 2022, 4:13am UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/7 "2022-02-07T04:13:46Z")

</div>

After some more troubleshooting, I found, that one of the web server had older version of filebeat, which had customized pipeline, which didn't have `event.original`.

When I excluded that host - everything works, without issues.

Issue resolved for my case.

Even though I still puzzled, why`"exists": { "field": "event.original" }` doesn't work. BenB196 explanation make sense, so I guess that's why.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 7, 2022, 4:14am UTC](https://discuss.elastic.co/t/no-data-for-query-exists-field-event-original/296355/8 "2022-03-07T04:14:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
