Hi all,
I am new in Elastic and I want to visualize data in Kibana from csv file.
the Data (kibana_test1.csv) are in the following form:
user_id,item_id,event_type,user_location,user_favorite_brand,item_brand
278673,234,purchase,City,Nokia,Samsung
278674,235,purchase,City,Nokia,Samsung
278675,236,purchase,City,Nokia,Samsung
278676,237,purchase,City,Nokia,Samsung
278677,238,purchase,City,Nokia,Samsung
278678,239,purchase,City,Nokia,Samsung
278679,240,purchase,City,Nokia,Samsung
278680,241,purchase,City,Nokia,Samsung
278681,242,purchase,City,Nokia,Samsung
278682,243,purchase,City,Nokia,Samsung
278683,244,purchase,City,Nokia,Samsung
278684,245,purchase,City,Nokia,Samsung
My logstash configuration file ( logstash-simple.conf) is:
input {
file {
path => "/home/nasos/Documents/kibana_test1.csv"
start_position => "beginning"}
}
filter {
csv {
separator => ","
columns => ["user_id","item_id","event_type","user_location","user_favorite_brand","item_brand"]
}
}
output {
elasticsearch {
action => "index"
hosts => ["localhost:9200"]
index => "data"
user => "elastic"
password => "My_password"
}
stdout { codec => rubydebug }
}
I have put the password (of elastic) that is generated from the command:
bin/x-pack/setup-passwords auto
I have created an index in Kibana- Dev Tools:
PUT /kibana_test1
{
"mappings": {
"doc": {
"properties": {
"user_id": {"type": "integer"},
"item_id": {"type": "integer"},
"event_type": {"type": "keyword"},
"user_location": {"type": "keyword"},
"user_favorite_brand": {"type": "keyword"},
"item_brand": {"type": "keyword"}
}
}
}
}
and I used
sudo ./logstash -f logstash-simple.conf --path.settings=/etc/logstash
and I got the following message:
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
It seem that it works fine so far. Isn't it?
when I used the Elasticsearch bulk API to load the data sets;
curl -H 'Content-Type: application/x-ndjson' -XPOST -u elastic 'localhost:9200/kibana_test1/doc/_bulk?pretty' --data-binary @kibana_test1.csv
I got the following error:
{
"error" : {
"root_cause" : [
{
"type" : "json_parse_exception",
"reason" : "Unrecognized token 'user_id': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@4df98756; line: 1, column: 9]"
}
],
"type" : "json_parse_exception",
"reason" : "Unrecognized token 'user_id': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@4df98756; line: 1, column: 9]"
},
"status" : 500
}
Could you please help?
Thx