# BulkProcessor API not indexing data to ES **URL:** \ **Category:** Elasticsearch\ **Created:** [June 8, 2017, 7:57am UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679 "2017-06-08T07:57:54Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![jshraddha](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@jshraddha](https://discuss.elastic.co/u/jshraddha)\ **Post date:** [June 8, 2017, 7:57am UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/1 "2017-06-08T07:57:54Z")
Hi, I am using BulkProcessor to ingest data to elasticsearch. My index is getting created but I could not see any data ingested to that index. I added logs into beforeBulk and afterBulk listeners logs are getting printed but data is not getting ingested. This is the code I am using ``` bulkProcessor.add(new IndexRequest(INDEX_NAME, type, id).source( sourceJson, XContentType.JSON)); bulkProcessor.awaitClose(5, TimeUnit.SECONDS); ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 8, 2017, 8:11am UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/2 "2017-06-08T08:11:08Z")
Can you share more code? ---
**Author:** ![jshraddha](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@jshraddha](https://discuss.elastic.co/u/jshraddha)\ **Post date:** [June 8, 2017, 8:16am UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/3 "2017-06-08T08:16:42Z")
This is the way I am creating bulkprocessor BulkProcessor .builder(ESConnectionFactory.newConnection(), new BulkProcessor.Listener() { ``` public void beforeBulk(long executionId, BulkRequest request) { logger.info("Before bulk"); } public void afterBulk(long executionId, BulkRequest request, Throwable failure) { } public void afterBulk(long executionId, BulkRequest request, BulkResponse response) { logger.info("After bulk"); } }) .setBulkActions(1) .setFlushInterval(TimeValue.timeValueSeconds(5)) .setConcurrentRequests(1) .setBackoffPolicy( BackoffPolicy.exponentialBackoff( TimeValue.timeValueMillis(100), 3)).build(); ``` ---
**Author:** ![jshraddha](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@jshraddha](https://discuss.elastic.co/u/jshraddha)\ **Post date:** [June 8, 2017, 9:14am UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/4 "2017-06-08T09:14:02Z")
My mistake JSON was having some problems so data was not getting indexed. One more thing I want to ask related to this is Previously I used RestCLient and there I was appending following line `String actionMetaData = String.format("{ \"index\" : { \"_index\" : \"%s\", \"_type\" : \"%s\" } }%n", index, type);` This way I was creating my json for almost 500+ docs and then creating entity object and ingesting data to elasticsearch through RestClient.performRequest. This operation was very quick. But now when I use transport client and try bulkprocessing it is taking so much time. Is there any way to do it similar to RestClient where I can give whole list of document in a single request as it is without iterating it. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 8, 2017, 2:49pm UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/5 "2017-06-08T14:49:19Z")
Not sure I understand the whole thing but doing a Bulk request over REST or a Bulk request over TRANSPORT should behave in a very similar way. If you see noticeable difference and that TRANSPORT is slower than REST, then I suspect you are doing something wrong or that you are not testing exactly with the same elasticsearch version. ---
**Author:** ![jshraddha](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@jshraddha](https://discuss.elastic.co/u/jshraddha)\ **Post date:** [June 9, 2017, 3:55am UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/6 "2017-06-09T03:55:08Z")
@dadoonet Let me explain you about the data. I am having a json file which contains an array of elements. Array size is around 300/400. What I am doing is I am iterating over this array and taking single element as a one document adding that document to bulkProcessor. So what happens is it iterates over 300 elements adds to bulkProcessor one by one. This takes time. This is a single file containing 300 elements but I will be having so many files like this. So inserting let say 20 file's data takes almost 5mins or something. May be I will be doing it is some wrong way. Previously when I used RestClient I used to append the line which i have mentioned above to each document and just inserting all the element's array so that was a one time request for 300 docs and it was doing it very quickly so inserting data for 20 files was not taking more than a minute. Is there anything similar for transport client or bulkProcessor where I can give whole array of 300 elements at one go? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 30, 2017, 2:20pm UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/7 "2017-06-30T14:20:12Z")
Why did you set `setBulkActions(1)`. It's going to make it pretty much slow as you will basically send one document by one instead of sending a batch of 1000 or 10000 docs at once. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 28, 2017, 2:20pm UTC](https://discuss.elastic.co/t/bulkprocessor-api-not-indexing-data-to-es/88679/8 "2017-07-28T14:20:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.