FIleBeat import csv file into elasticsearch,but each line was treated as a string, I want each line was spilit by comma


(Liu Lantern) #1
  1. I got a problem when I import csv file into elasticsearch
  2. here is my csv file looks like

name,age,sex
amy,18,girl
tom,19,boy

  1. and I use the default template in /etc/filebeat/filebeat.template.json which is below

{
"mappings": {
"default": {
"_all": {
"enabled": true,
"norms": {
"enabled": false
}
},
"dynamic_templates": [
{
"template1": {
"mapping": {
"doc_values": true,
"ignore_above": 1024,
"index": "not_analyzed",
"type": "{dynamic_type}"
},
"match": ""
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"message": {
"type": "string",
"index": "analyzed"
},
"offset": {
"type": "long",
"doc_values": "true"
}
}
}
},
"settings": {
"index.refresh_interval": "5s"
},
"template": "filebeat-
"
}

  1. the data I got in elasticseach was like
  1. message: "amy,18,girl"
  2. message:"tom,19,boy"
  1. I want to get the result like below
  1. {name: "amy", age: 18, sex: "girl"}
  2. {name: "tom", age: 19, sex: "boy"}
  1. So what should I config to get the result( just split the each line by comma when I import the csv file to elasticseach )

(Mark Walkom) #2

You need to use Logstash for that.


(Liu Lantern) #3

Thanks Very Much!
Finally we use Python to import the csv file into elasticsearch.


(system) #4

This topic was automatically closed after 21 days. New replies are no longer allowed.