Logstash throwing error for filter portion when trying to convert csv to elastic search

Elastic Stack Logstash
1

Fixed so deleting

3

sorry, This is my config:

input {
file {
path => "/opt/TenantAvailabilityCheckConsolidatedCustomReport.csv"
start_position => "beginning"
ignore_older => 0
sincedb_path => "/persistent/log"
sincedb_write_interval => 10
}
}
filter {
csv {
seperator => ","
columns => ["Customer Name", "Tenant ID", "Cluster", "Application ID", "Business Type", "Component Name", "Error Message", "Reported On or Before Date-Time", "Closed On or Before Date-Time", "Current Status" ]
mutate {
convert => { "Reported On or Before Date-Time" => "integer"
"Closed On or Before Date-Time" => "integer"
}
}

    }

}

output {
elasticsearch {
hosts => "localhost"
index => "logs"
document_type => "error_logs"
}

stdout{}
}

4

You seem to be missing a curly brace to close the csv block before the mutate block starts.

1 Like
5

@Christian_Dahlqvist
Thanks, I done that... Now the error changed to

Unknown setting 'seperator' for csv {:level=>:error}
Error: Something is wrong with your configuration. {:level=>:error}

6

I think the error message is quite clear. It is separator, not seperator.

7

Thanks Christian. @Christian_Dahlqvist Now I am able to process the csv. But still I have some issue. say for example
This is my csv file

Header:
Customer Name,Tenant ID,Cluster,Application ID,Business Type,Component Name,Error Message,Reported On or Before Date-Time,Closed On or Before Date-Time,Current Status

Values:
customerxvz,e0431,GBT132,e0431tmn,Test,CXF-endpoint-IFLMAP-hcisbt,"[CXF][CXF][EXCEPTION]:java.net.SocketTimeoutException: Read timed out (local port 59858 to address 10.78.28.107 (vsa3873176.od.sap.biz), remote port 443 to address 155.56.210.165)",29-4-2018 00:03:54 UTC,29-4-2018 02:03:58 UTC,Completed

But with the above configuration file result I got is:

message:customerxyz,e0431,GBT132,e0431iflmap,Test,Messaging Service (JMS),State Message is returning Null,30-4-2018 07:45:30 UTC,08-5-2018 06:54:28 UTC,Completed Customer Name:customerxyz Tenant ID:e0431 @version:1 @timestamp:May 14th 2018, 18:07:06.805 path:/opt/TenantAvailabilityCheckConsolidatedCustomReport.csv host:devops Cluster:GBT132 Application ID:e0431iflmap Business Type:Test Component Name:Messaging Service (JMS) Error Message:State Message is returning Null Reported On or Before Date-Time:30 Closed On or Before Date-Time:8 Current Status:Completed _id:AWNep4WMau29r0E95zBe _type:error_logs _index:hurray _score:

We can see that these are not the fields expected. Fields mismatch each other. Some fits and some are misplaced. Is there any additional match I have to done?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.