# Bulk NodeJS fetch

**URL:** <https://discuss.elastic.co/t/bulk-nodejs-fetch/366577>\
**Category:** Elastic Search\
**Created:** [September 14, 2024, 4:36pm UTC](https://discuss.elastic.co/t/bulk-nodejs-fetch/366577 "2024-09-14T16:36:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![haborda](https://avatars.discourse-cdn.com/v4/letter/h/8e8cbc/32.png) [@haborda](https://discuss.elastic.co/u/haborda)\
**Post date:** [September 14, 2024, 4:36pm UTC](https://discuss.elastic.co/t/bulk-nodejs-fetch/366577/1 "2024-09-14T16:36:01Z")

</div>

Hi all:

I do have a file (my-file.json) that meets ES Bulk REST API. I can call it like this:

```auto
curl --data-binary "@./<my-file>json" -X POST <ES-cloud-URL>:443/_bulk?pretty \
  -H "Authorization: <my-APi-key>" \
  -H "Content-Type: application/json" 

```

and the data gets loaded into my index. So I know the file is correct.

I would like not to use ES' API, and try to call the same REST end-point from NodeJS

```auto
const fs = require('fs/promises');

async function sendItToES()
    {
    const filePath = "<my-file>.json";
    const stats = await fs.stat(filePath); //added this to be sure I was reading the right file
    const fileSizeInBytes = stats.size;
    const formData = new FormData();
    formData.append('file', filePath);
 
    const theURL = '<ES-cloud-URL>:443/_bulk?pretty -H';
   
    const theLoad = await fetch (theURL, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',

            // 'Content-Type': 'application/x-ndjson', //this one did not work either
            'Authorization': '<my-APi-key>'
        },
        body: formData
    })
    .then(response => {
        if (response.ok) {
            console.log('the response:', response);
        } else {
            console.log('upload the error');
        }
        return response.json();
    })
    .then (data => {
        console.log('server reponse', data);
    })
    .catch(err => {
        console.error('error uploading file:', err);
    });
}

I trie many different permutations and I always get "server reponse {error: {…}, status: 400}"

Thanks in advance.
```

---

<div class="post-metadata">

**Author:** ![ashishtiwari1993](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashishtiwari1993/32/135241_2.png) [@ashishtiwari1993](https://discuss.elastic.co/u/ashishtiwari1993)\
**Post date:** [September 15, 2024, 4:20pm UTC](https://discuss.elastic.co/t/bulk-nodejs-fetch/366577/2 "2024-09-15T16:20:13Z")

</div>

I think you need to send file data in binary. Also could you share exact error you getting?

---

<div class="post-metadata">

**Author:** ![haborda](https://avatars.discourse-cdn.com/v4/letter/h/8e8cbc/32.png) [@haborda](https://discuss.elastic.co/u/haborda)\
**Post date:** [September 15, 2024, 11:00pm UTC](https://discuss.elastic.co/t/bulk-nodejs-fetch/366577/3 "2024-09-15T23:00:51Z")

</div>

Thanks Ashish, it finally got it to work. bellow if the code just incase somebody else run into the same issue

```auto
const fs = require('fs/promises'); // for file manipulation, notice we are not using fs but fs/promises

async function sendItToES()
{
    const theURL = 'https://<for ES cluster>:443/_bulk?pretty';
   
    const theLoad = await fetch (theURL, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/x-ndjson',
            'Authorization': 'ApiKey <your API key>'
        },
        body: await fs.readFile('<your file and path>')
    })
    .then(response => {
        if (response.ok) {
            console.log('succeded sending payload to ES');
        } else {
            console.log('upload to ES failed: ', response);
        }
        return response.json();
    })
    .then (data => {
        console.log('data sent to ES: ', data);
    })
    .catch(err => {
        console.error('error uploading file: ', err);
    });
}

```

---

<div class="post-metadata">

**Author:** ![joemcelroy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joemcelroy/32/103757_2.png) [@joemcelroy](https://discuss.elastic.co/u/joemcelroy)\
**Post date:** [September 17, 2024, 8:09am UTC](https://discuss.elastic.co/t/bulk-nodejs-fetch/366577/4 "2024-09-17T08:09:09Z")

</div>

Curious - Why not use the Node Elasticsearch Client here?

Node Elasticsearch client has some nice helpers which allow you to import docs very easily

```javascriptconst
const split = require('split2')
const { Client } = require('@elastic/elasticsearch')

const client = new Client({
  node: 'https://<for ES cluster>:443/',
  auth: { apiKey: 'base64EncodedKey' }
})
const result = await client.helpers.bulk({
  datasource: createReadStream('<your ndjson file and path>').pipe(split()),
  onDocument (doc) {
    return {
      index: { _index: 'my-index' }
    }
  }
})

console.log(result)

```

For more information [Client helpers | Elasticsearch JavaScript Client [8.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/client-helpers.html#_usage_3)

```auto

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 15, 2024, 8:09am UTC](https://discuss.elastic.co/t/bulk-nodejs-fetch/366577/5 "2024-10-15T08:09:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.