# 400 malformed issue while inserting into Elastic using JAVA Apache Beam

**URL:** <https://discuss.elastic.co/t/400-malformed-issue-while-inserting-into-elastic-using-java-apache-beam/363101>\
**Category:** Elastic Search\
**Created:** [July 14, 2024, 2:26pm UTC](https://discuss.elastic.co/t/400-malformed-issue-while-inserting-into-elastic-using-java-apache-beam/363101 "2024-07-14T14:26:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![WBENJAM6\_ford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wbenjam6_ford/32/136023_2.png) [@WBENJAM6\_ford](https://discuss.elastic.co/u/WBENJAM6_ford)\
**Post date:** [July 14, 2024, 2:26pm UTC](https://discuss.elastic.co/t/400-malformed-issue-while-inserting-into-elastic-using-java-apache-beam/363101/1 "2024-07-14T14:26:48Z")

</div>

Hi,

We are trying to inject data using our Apache Beam JAVA code to Elastic search Index. But whenever we try to inject the data we get a malformed error 400. This is the code I am trying to insert data into Elasticsearch. Can someone help?

PCollection jsonRecords = pipeline.apply(READ\_PARQUET\_FILES,  
ParquetIO.read(helpJoinedRecordSchema)  
.from(helpDailyFolder))  
.setCoder(AvroCoder.of(GenericRecord.class, helpJoinedRecordSchema))  
.apply(CONVERT\_TO\_JSON,  
ParDo.of(new HelpDailyGenericRecordToJsonFn(indexName)));

```
    PCollection<String> json = jsonRecords
            .apply("Create Bulk Requests", ParDo.of(new DoFn<String, String>() {
                @ProcessElement
                public void processElement(@Element String json, OutputReceiver<String> out) {
                    String documentId = UUID.randomUUID().toString();
                    try {
                        String indexActionJson = "{ \"index\" : { \"_index\" : \"" + indexName + "\", \"_id\" : \"" + documentId + "\" } }";
                        String bulkRequestJson = indexActionJson + "\n" + json + "\n";
                        //log.info("bulk " + bulkRequestJson);
                        out.output(bulkRequestJson);
                    } catch (Exception e) {
                        log.error("Error creating Bulk API request JSON: ", e);
                    }
                }
            }));
    json.apply("Write to Elasticsearch", ElasticsearchIO.write()
            .withConnectionConfiguration(connectElasticSearchIndex(indexName))
            .withMaxBatchSize(BATCH_SIZE_BYTES)
            .withMaxBatchSizeBytes(5000));

```

Caused by: org.elasticsearch.client.ResponseException: method [POST], host [xxxxxx:9243], URI [/aws\_live/\_doc/\_bulk], status line [HTTP/1.1 400 Bad Request]  
{"error":{"root\_cause":[{"type":"illegal\_argument\_exception","reason":"Malformed content, found extra data after parsing: START\_OBJECT"}],"type":"illegal\_argument\_exception","reason":"Malformed content, found extra data after parsing: START\_OBJECT"},"status":400}

---

<div class="post-metadata">

**Author:** ![Tiharqa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tiharqa/32/95336_2.png) [@Tiharqa](https://discuss.elastic.co/u/Tiharqa)\
**Post date:** [July 14, 2024, 10:09pm UTC](https://discuss.elastic.co/t/400-malformed-issue-while-inserting-into-elastic-using-java-apache-beam/363101/2 "2024-07-14T22:09:11Z")

</div>

i suspect some thing with data format. because the error Malformed content found extra data after parsing: START\_OBJECT typically indicates an issue with the format of the JSON being sent to Elasticsearch’s bulk API ….The bulk API expects newline-delimited JSON (NDJSON) format, where each line contains a separate JSON object and actions (such as “index”) are specified on a separate line from the document.

---

<div class="post-metadata">

**Author:** ![WBENJAM6\_ford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wbenjam6_ford/32/136023_2.png) [@WBENJAM6\_ford](https://discuss.elastic.co/u/WBENJAM6_ford)\
**Post date:** [July 15, 2024, 6:00am UTC](https://discuss.elastic.co/t/400-malformed-issue-while-inserting-into-elastic-using-java-apache-beam/363101/3 "2024-07-15T06:00:19Z")

</div>

Yes I thought that could be the issue. So I tried to print my data before doing the Elastic write operation. I could notice the JSON formatted with the index name and my actual data in another line with new line space as shown below. This is how i could see the values formed when i am trying to print the json logs. Also i tried using ElasticsearchIO.docToBulk() instead of ElasticsearchIO.write(). When i use docToBulk() method i dont get this error but i dont see the data getting written to my Elasticsearch index. So i am not sure what i am missing here?

{ "index" : { "\_index" : "xxxx", "\_id" : "9ee466d5-a72d-4fa1-bde6-96dad3e4d503" } }  
{"id":null,"myfields":null}

{ "index" : { "\_index" : "xxxx", "\_id" : "9ee466d5-a72d-4fa1-bde6-96dad3e4d504" } }  
{"id":null,"myfields":null}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 12, 2024, 6:00am UTC](https://discuss.elastic.co/t/400-malformed-issue-while-inserting-into-elastic-using-java-apache-beam/363101/4 "2024-08-12T06:00:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.