Empty fields in Kibana if you ingest an index using python

Elastic Stack Elasticsearch
1

I am pushing a CSV file in Elasticsearch using python with a change in mappings. The error that I'm facing is that the mappings are getting updated by ingesting the CSV through python but while creating the data view manually through kibana it is showing empty fields. If I upload the same CSV file manually, it does not show any empty fields. So, I'm able to create a dashboard with the manual CSV upload but not with the file uploaded through python. What is the problem here and how can I solve it?

2

Sharing the full error will be helpful, as would sharing the code you are using.

3

I'm not getting any error to show. It's just that I'm creating an index in Elasticsearch using python by pushing a CSV file and it is getting created. But if I am creating a data view in Kibana to build a dashboard using the same index it is showing empty fields for all the types. But if I am pushing the CSV file manually in Elasticsearch and creating a dashboard it does not show empty fields. And because of that, I'm able to build a dashboard. So, I want to create an index using python which when creating a data view in Kibana manually does not show empty records.

from elasticsearch import Elasticsearch, helpers
import configparser
import csv

config = configparser.ConfigParser()
config.read('D:/Pushing data/push/example.ini')

es = Elasticsearch( cloud_id=config['ELASTIC']['cloud_id'], basic_auth=(config['ELASTIC']['user'], config['ELASTIC']['password']))
print(es.info())

settings = {
       "number_of_shards": 1,
      "number_of_replicas": 1
}
mappings = {
        "properties": {
    "cleanText": {
      "type": "text",
      "fielddata": True
    },
    "content": {
      "type": "keyword"
    },
    "description": {
      "type": "text",
      "fielddata": True
    },
    "followers": {
      "type": "long"
    },
    "following": {
      "type": "long"
    },
    "hashtags": {
      "type": "keyword"
    },
    "id": {
      "type": "long"
    },
    "lang": {
      "type": "keyword"
    },
    "likeCount": {
      "type": "long"
    },
    "mentions": {
      "type": "keyword"
    },
    "no": {
      "type": "long"
    },
    "retweetCount": {
      "type": "long"
    },
    "source": {
      "type": "keyword"
    },
    "status": {
      "type": "keyword"
    },
    "subjectivity": {
      "type": "double"
    },
    "time": {
      "type": "date",
      "format": "epoch_millis"
    },
    "topic": {
      "type": "keyword"
    },
    "totaltweets": {
      "type": "long"
    },
    "translatedText": {
      "type": "text",
      "fielddata": True
    },
    "url": {
      "type": "keyword"
    },
    "user_location": {
      "type": "keyword"
    },
    "user_username": {
      "type": "keyword"
    },
    "user_verified": {
      "type": "keyword"
    },
    "vaderSentiment": {
      "type": "double"
    }
  }
    }

       

with open('D:/Pushing data/push/Topic_Rohit Sharma.csv',encoding="UTF-8") as f:
    reader = csv.DictReader(f)
    es.indices.create(index="rohit", ignore=400, settings = settings, mappings = mappings)
    helpers.bulk(es,reader,index = "rohit")

The credentials of my cloud id are in a separate example.ini file. The index is getting created through this code but while creating a data view to build dashboards it does not show any records.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.