When parsing data I am getting this error

Elastic Stack Elasticsearch
1

I have a data without columns name and I am giving name of the columns in conf file. I don't know giving this way columns name is correct or not?
[2021-03-01T17:33:34,741][WARN ][logstash.filters.csv ][main][7c8e4b1853e62f45feefa30d1a08812903bc9db233255c5ec2cd00588b905f5f] Error parsing csv {:field=>"message", :source=>"2020-08-03 09:41:09,923361900066,UFone,"200.00 PKR have been Debited on 03-08-2020, 09:29:26. For assistance call 021-111-627-627." ,8627,1,2020-08-03 09:41:23,DELIVERED,SUCCESS\r", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field on line 1.>}

2

hi.. please attach your logstash configuration file

1 Like
3

input{

file{

    path => "C:/Users/Ubaid/Desktop/ElasticSearch/elasticsearch-7.11.1-windows-x86_64/NBP-Aug.csv"

    start_position => "beginning"

    sincedb_path => "Null"

}

}

filter{

csv{

    separator => ","

    columns => ["Date","Mobile","Telco","Msg","ShortCode","SMSLength","DeliveryDate","DeliveryStatus","Criteria"]

}

grok{

    match => {

        "message" => [

            "bill payment of amount %{NUMBER:billAmount} for Company Name %{NOTSPACE:Name}",

            "Rs.%{DATA:ATMamount} %{NOTSPACE:PaymentType} from ATM",

            "you have successfully %{WORD:login} at",

            "Ledger Bal: %{NUMBER:ledgerAmount}, Available Bal:%{NUMBER:AvailableBal}, Status:%{NOTSPACE:status}",

            "Rs.%{DATA:CreditedAmount} have been %{NOTSPACE:PaymentType} into your account",

            "You have successfully %{NOTSPACE:transffered} PKR. %{NUMBER:transAmount} from Account",

            "Your Complaint number 7221679 is in %{NOTSPACE:ComplaintProcess}"

        ]

    }

}

 mutate {

    convert => {

        "billAmount" => "integer"

        "ATMamount" => "integer"

        "CreditedAmount" => "integer"

        "ledgerAmount" => "integer"

        "AvailableBal" => "integer"

        "transAmount" => "integer"

    }

}

date{

    match => ["Date", "M/dd/yyyy HH:mm:ss"]

    target => "Date"

}

}

output{

elasticsearch{

    hosts => "http://localhost:9200"

    index => "august-nbp-data"

}

stdout{

    codec => rubydebug

}

}

4

Please help me to resolve above error. I have attached configuration file as well.
exception=>#<CSV:: MalformedCSVError: Unclosed quoted field on line 1.> }

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.