This is my csv field example input
50001979,60004305,DHL Reseller Nakorn In,THLZ3EL29,01-08-20 8:37,01-08-20 19:07,ศรีสุดา เตียเจริญ,34/53 ม.4 แขวง/ลาดพร้าว,Lat Phrao,Bangkok,10230,897957585,นภวรรณ สวัสดิ์มงคลกุล,504/1 ซ.สาธุประดิษฐ์ 49 บางโพงพาง ยานนาวา,Yannawa,Bangkok,10120,813393151,PDO,SML,0,0,NO,35,0,0,0,35,reseller.nakornin1@gmail.com
This is my logstash config
'
input
{
file{
path => ["C:/Users/nitikarnh/Desktop/DSL/Transaction_report/Aug.csv"]
start_position => "beginning"
sincedb_path => "NUL"
}
}
filter
{
csv {
separator => ","
columns => ["Service Point Code","Service Point PUID","Service Point Name","Shipment ID","Creation Date Time","Handover Date Time","Sender Name","Sender Address","Sender District","Sender Province","Sender Postal Code","Sender Mobile.","Recipient Name","Recipient Address","Recipient District","Recipient Province","Recipient Postal Code","Recipient Mobile","Shipping Service","Format","COD Amount","Insurance Amount","Obox","Shipping Fee","COD Fee","INS Fee","OBOX Fee","Total Fee","Booking Agent Email"]
skip_header => "true"
}
}
filter
{
date {
match => ["Creation Date Time","dd-MM-YY HH:mm"]
target => "Creation Date Time"
}
date {
match => ["Handover Date Time","dd-MM-YY HH:mm"]
target => "Handover Date Time"
}
}
filter
{
mutate
{
convert => {"COD Amount" => "integer" "Insurance Amount" => "integer" "Shipping Fee" => "integer" "COD Fee" => "integer" "INS Fee" => "integer" "OBOX Fee" => "integer" "Total Fee" => "integer"}
}
}
output
{
elasticsearch {
hosts => "https://localhost:9200"
user => "elastic"
password => "1234"
index=> "dsltest99"
}
stdout { codec => rubydebug }
}
'
This is my output
{
"path" => "C:/Users/nitikarnh/Desktop/DSL/Transaction_report/Aug.csv",
"@version" => "1",
"host" => "DESKTOP-NC07CL4",
"message" => "50001979,60004305,\"WARP SERVICE CO.,LTD. (DHL Reseller Nakorn In)\",TH2PL4ZFQ,19-08-20 11:36,19-08-20 16:28,Poom's plankids,35/3 \xCD.\xB9\xD3?\xB7\xC7\xD5 \xB5.\xBA?\xCA??,Bang kruai,Nonthaburi,11130,994178668,\xC1\xB9\xD1\xCA\xC7\xD5 \xE0\xA1\xC9\xC1\xCA?\xB5\xEC \xB3 \xCD\xC2?\xC2\xD2,4 \xC1.1 \xCB\xC1\xD9\xE8\xBA\xE9?\xC0?\xC0\xD4\xC3\xC1\xC2\xEC \xAB.\xCA\xD8\xC0?\xA7\xC9\xEC3 \xE1?10 \xE1\xA2?\xBA?\xBA\u0379,\xBB\xC3\xD0\xE0\xC7\xC8,\xA1\xC3?\u0DFE\xC1\xCB?\xA4\xC3,10250,877099912,PDO,JIF,0,0,NO,35,0,0,0,35,reseller.nakornin1@gmail.com\r",
"tags" => [
[0] "_csvparsefailure"
],
"@timestamp" => 2020-09-04T04:01:48.656Z
}