# HTTP JSON input: one array one document

**URL:** <https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 29, 2021, 9:05am UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327 "2021-06-29T09:05:26Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![thopic](https://avatars.discourse-cdn.com/v4/letter/t/b487fb/32.png) [@thopic](https://discuss.elastic.co/u/thopic)\
**Post date:** [June 29, 2021, 9:05am UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/1 "2021-06-29T09:05:26Z")

</div>

Hello,

I am using the [HTTP JSON input](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html) of Filebeat to query some APIs and get some metrics. The problem is for some of them to send a JSON array instead of a JSON object. If I want to make a count of a specific metric, I can't sum the values for each object in the array. I went through all the above documentation, but failed to find a solution to my problem.

Has anyone found a workaround?

Thanks!

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [June 30, 2021, 12:54am UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/2 "2021-06-30T00:54:39Z")

</div>

Can u clarify what you're trying to do with the array of data?

---

<div class="post-metadata">

**Author:** ![thopic](https://avatars.discourse-cdn.com/v4/letter/t/b487fb/32.png) [@thopic](https://discuss.elastic.co/u/thopic)\
**Post date:** [June 30, 2021, 7:15am UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/3 "2021-06-30T07:15:32Z")

</div>

Sure!

For example, if the API request returns this type of data :

```json
[{"username": "foo",
  "messages": "x"},
 {"username": "bar",
  "messages": "y"}]

```

I would like to sum the `messages` values to get the total number of messages (x+y).

From what I've experimented, this kind of response is processed in two times :

1. First with `foo`
2. Then with `bar`

I can use [`response.transforms`](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-transforms) but only in the scope of one JSON object. So I'm stuck.

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [June 30, 2021, 10:55am UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/4 "2021-06-30T10:55:18Z")

</div>

Can you post the config for your httpjson input?

---

<div class="post-metadata">

**Author:** ![thopic](https://avatars.discourse-cdn.com/v4/letter/t/b487fb/32.png) [@thopic](https://discuss.elastic.co/u/thopic)\
**Post date:** [June 30, 2021, 2:27pm UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/5 "2021-06-30T14:27:28Z")

</div>

Hm no I can't because I didn't find any that could work. But I can try to give a more meaningful example.

For example for collecting emails count on a daily basis for Mailcow, I could use this [endpoint](https://mailcow.docs.apiary.io/#reference/mailboxes/get-mailboxes/get-mailboxes). But the output will be an array with all the mailboxes details (below is only one mailbox as the API example) :

```json
[
  {
    "max_new_quota": 10737418240,
    "username": "info@doman3.tld",
    "rl": false,
    "is_relayed": 0,
    "name": "Full name",
    "active": "1",
    "domain": "doman3.tld",
    "local_part": "info",
    "quota": 3221225472,
    "attributes": {
      "force_pw_update": "0",
      "tls_enforce_in": "0",
      "tls_enforce_out": "0",
      "sogo_access": "1",
      "mailbox_format": "maildir:",
      "quarantine_notification": "never"
    },
    "quota_used": 0,
    "percent_in_use": 0,
    "messages": 0, <-- I want to sum this field for all mailboxes
    "spam_aliases": 0,
    "percent_class": "success"
  }
]

```

Actually I realize there are 2 questions in my post:

1. How to have access to all the mailboxes details within one response and be able to process this response with processors;
2. With making use of processors, how could I end up with the sum of all messages count?

Maybe my case is too specific, I can keep searching, but the first step is to know if it is possible to process all json documents within one response?

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [July 1, 2021, 12:52am UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/6 "2021-07-01T00:52:04Z")

</div>

to split the response use [HTTP JSON input | Filebeat Reference [7.13] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-split). The MISP module does, [beats/config.yml at master · elastic/beats · GitHub](https://github.com/elastic/beats/blob/master/x-pack/filebeat/module/threatintel/misp/config/config.yml#L33), to split the API response.

```auto
response.split:
  target: body.response
  split:
    target: body.Event.Attribute
    keep_parent: true

```

You'd probably just do

```auto
response.split:
  target: body.response
  split:
    target: body
    keep_parent: true

```

---

<div class="post-metadata">

**Author:** ![thopic](https://avatars.discourse-cdn.com/v4/letter/t/b487fb/32.png) [@thopic](https://discuss.elastic.co/u/thopic)\
**Post date:** [July 2, 2021, 1:19pm UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/7 "2021-07-02T13:19:18Z")

</div>

I thought I've tried to split the response (in a lot of different ways) but this operation only applied to the JSON objects and not to the whole array. Anyway, I will give it another try with what you provided, but in a few days (maybe weeks...) unfortunately. I will keep you posted. Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 30, 2021, 3:20pm UTC](https://discuss.elastic.co/t/http-json-input-one-array-one-document/277327/8 "2021-07-30T15:20:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.