ES ignores mapping when using Python DSL bulk

Elastic Stack Elasticsearch
1

I'm trying to upload a CSV file to an elasticsearch index. Let's say the file is something like this (no headers, just data):

bird,10,dog
cat,20,giraffe

This is the code I have:

from elasticsearch_dsl import DocType, Integer, Keyword
from elasticsearch_dsl.connections import connections
from elasticsearch.helpers import bulk
import csv

connections.create_connection(hosts=["localhost"])

class Mapping(DocType):    
    animal1 = Keyword()
    number = Integer()
    animal2 = Keyword()

    class Meta:
        index = "index-name"
        doc_type = "doc-type" 

Mapping.init()
with open("/path/to/file", "r", encoding="latin-1") as f:
    reader = csv.DictReader(f)
    bulk(
        connections.get_connection(),
        (Mapping(**row).to_dict(True) for row in reader)
    )

The problem is that elasticsearch seems to be ignoring the mapping and using the first line of the file as headers (and creating a mapping based on that).

Edit: it really uses my mapping and the first line of the file. The mapping it generates is:

{
  "index-name": {
    "mappings": {
      "doc-type": {
        "properties": {
          "10": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "dog": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "animal1": {
            "type": "keyword"
          },
          "animal2": {
            "type": "keyword"
          },
          "bird": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "number": {
            "type": "integer"
          }
        }
      }
    }
  }
}

If I only create the index without uploading data, the mapping seems fine:

{
  "index-name": {
    "mappings": {
      "doc-type": {
        "properties": {
          "animal1": {
            "type": "keyword"
          },
          "animal2": {
            "type": "keyword"
          },
          "number": {
            "type": "integer"
          }
        }
      }
    }
  }
}

How can I make ES use the given mapping and just that?

2

The problem wasn't bulk. csv.DictReader always reads the first line from the file to get the headers for subsequent rows. So if you're going to use DictReader, the file needs a header.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.