# Batch input data and send parallel http calls

**URL:** https://discuss.elastic.co/t/batch-input-data-and-send-parallel-http-calls/225315
**Category:** Logstash
**Created:** [March 27, 2020, 12:21am UTC](https://discuss.elastic.co/t/batch-input-data-and-send-parallel-http-calls/225315 "2020-03-27T00:21:49Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![mikes](https://avatars.discourse-cdn.com/v4/letter/m/919ad9/32.png) [@mikes](https://discuss.elastic.co/u/mikes)
#### Post date: [March 27, 2020, 12:21am UTC](https://discuss.elastic.co/t/batch-input-data-and-send-parallel-http-calls/225315/1 "2020-03-27T00:21:49Z")

</div>

Hello,  
I am trying to do the following:  
Take input on tcp and send it to a http endpoint. I need to send batches of events to the http endpoint. I am trying to use the Aggregate filter to batch events and then sending to http endpoint as a json array. Here is what the config looks like:

```auto
input {
	tcp {
		port => 8081
	}
}

filter {

  ruby {
    code => 'event.set("randomNumber", rand(10))'
  }
  aggregate {
    task_id => "%{randomNumber}"
    timeout => 5
    push_map_as_event_on_timeout => true
    code => "
        map['messages'] ||= []
        map['messages'] << event.get('message')
        event.cancel()
    "
  }

  json_encode {
    source => "messages"
    target => "messages"
  }

}

output {

http {
        http_method => "post"
        headers => ["Content-Type", "application/json", "Content-Encoding", "gzip"]
        url => "http://localhost:8086/logs/json"
        format => "message"
        message => "%{messages}"
        content_type => "application/json"
        http_compression => "true"
    }
}

```

pipeline workers is set to 20

My expectation is:  
The aggregate filter will put the incoming data in 10 random buckets and send the bucket after 5s timeout to the output. I am expecting to see upto 10 parallel http calls.  
However, I only see one call and if that call is slow then no other http call happens.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 24, 2020, 12:21am UTC](https://discuss.elastic.co/t/batch-input-data-and-send-parallel-http-calls/225315/2 "2020-04-24T00:21:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
