# Filter data using HttpRequest

**URL:** <https://discuss.elastic.co/t/filter-data-using-httprequest/281508>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 16, 2021, 6:58am UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508 "2021-08-16T06:58:56Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![fed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fed/32/93195_2.png) [@fed](https://discuss.elastic.co/u/fed)\
**Post date:** [August 16, 2021, 6:58am UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/1 "2021-08-16T06:58:56Z")

</div>

Hi!  
I'm new to the ELK stack and I'd like to know if there's a way to filter the logs in filebeat using a processor that checks some values from an external REST API, and based on the response, it drops or sends the events to elasticsearch. I tried using the script processor and an XMLHttpRequest inside it, but it doesn't seem to work, probably due to the implementation of ECMAScript that is used.  
Does anyone have a suggestion?  
Thanks!

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [August 16, 2021, 1:53pm UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/2 "2021-08-16T13:53:54Z")

</div>

U can use the httpjson input, [HTTP JSON input | Filebeat Reference [7.14] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html)

---

<div class="post-metadata">

**Author:** ![fed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fed/32/93195_2.png) [@fed](https://discuss.elastic.co/u/fed)\
**Post date:** [August 16, 2021, 7:41pm UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/3 "2021-08-16T19:41:31Z")

</div>

Thanks for your answer, but I wasn't clear explaining my situation. I use filebeat to collect the logs outputted from a machine's syslog. I need to be able to filter those messages based on the state of the REST APIs that are generated from a server that is not linked in any way to the first machine.  
Hope this explains the issue better.

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [August 16, 2021, 8:12pm UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/4 "2021-08-16T20:12:24Z")

</div>

Gotcha. Can u provide a sample of what you're trying to do?

---

<div class="post-metadata">

**Author:** ![fed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fed/32/93195_2.png) [@fed](https://discuss.elastic.co/u/fed)\
**Post date:** [August 17, 2021, 7:02am UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/5 "2021-08-17T07:02:24Z")

</div>

Here's the idea. Drop or pass each new syslog based on the values of the JSON response fetched from the server.

```auto
---
filebeat.config:
  modules:
    path: "${path.config}/modules.d/*.yml"
    reload.enabled: false

output.elasticsearch:
  hosts:
    - http://elasticsearch:9200
  index: "logs-rpi-%{+yyyy.MM.dd}"

setup.template:
  name: "log"
  pattern: "log-*"
  enabled: false

setup.ilm.enabled: false

filebeat.inputs:
  - type: "log"
    enabled: true
    scan_frequency: 1s
    paths:
      - "/var/log/testLog/machine.log" #this is the file where the machine's syslog are sent to

processors:
  - script:
      lang: javascript
      id: my_filter
      source: >
        
        var xmlhttp = new XMLHttpRequest();
        var url = "Server URL";

        xmlhttp.onreadystatechange = function() {
            if (this.readyState == 4 && this.status == 200) {
                var answer = JSON.parse(this.responseText);
                if (answer.state == "offline")
                  event.Cancel();
            }
        }
        xmlhttp.open("GET", url, true);
        xmlhttp.send();

  - add_fields:
      target: tag
      fields:
        client_id: client_1
        color: red
        key_1: value_1

```

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [August 17, 2021, 11:26am UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/6 "2021-08-17T11:26:18Z")

</div>

Ya, as far as I'm tracking the XMLHttpRequest() function is not available. The only JavaScript is the basic string/object manipulation functions. Unfortunately I don't think u r a going to be able to do what u want with Filebeat. Maybe look at logstash to see if it has a processor that could do what u want.

---

<div class="post-metadata">

**Author:** ![fed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fed/32/93195_2.png) [@fed](https://discuss.elastic.co/u/fed)\
**Post date:** [August 17, 2021, 12:02pm UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/7 "2021-08-17T12:02:12Z")

</div>

Okay. Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 14, 2021, 2:02pm UTC](https://discuss.elastic.co/t/filter-data-using-httprequest/281508/8 "2021-09-14T14:02:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.