Import specific fields only - Elasticsearch bulk import

(Abhilash Sathe) #1

Is there a way where we can ignore certain JSON keys inside a JSON dump while doing a bulk import in elasticsearch.

I'm currently working with Node JS. As mentioned in the below code, data.json contains certain JSON keys that I would want elasticsearch to ignore while importing. These keys can be of any type, example an Object, a string, a boolean, etc.

var bulkIndex = function bulkIndex(index, type, data) {
    let bulkBody = [];

    data.forEach(item => {
            index: {
                _index: index,
                _type: type,


            body: bulkBody
        .then(response => {
            let errorCount = 0;
            response.items.forEach(item => {
                if (item.index && item.index.error) {
                    console.log(++errorCount, item.index.error);
            console.log(`Successfully indexed ${data.length - errorCount} out of ${data.length} items`);
const test = function test() {
    const rawJSON = fs.readFileSync('data.json');
    const processedJSON = JSON.parse(rawJSON);
    console.log(`${processedJSON.length} items parsed from data file`);
    bulkIndex('myIndex', 'myType', processedJSON);

(David Pilato) #2

Can't you do that in your JS code?

(Abhilash Sathe) #3

Yes, that's what I'm going to do now. The JSON is basically Shopify's storefront product data. I've decided to use GraphQL and get specific fields only from the corresponding API.

(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.