How to index multiple files of different extensions using the same .config file

Elastic Stack Logstash
1

I have two files with .xlsx and .csv extensions. I want to index both the files into elasticsearch using the same .config file. The problem is in the filter section as the separator for both the files varies. Is there any way to index both the files into elasticsearch using one .config file?

2

Okay, so the .xlsx file is actually a CSV file?

But yeah, just assign different types or different tags to the messages read by each input and use conditionals in the filter and/or output sections to have different filtering behavior depending on the kind of message it is.

1 Like
3

No, the .xlsx file is not a CSV file. I'm not able to find the separator for .xlsx file as in case of CSV file.

Here's my .config file for my CSV file -

                          input {  
                                    file {
                                           path => ["/home/pavan/Downloads/test//home/pavan/Downloads/test/National-Christian.csv"]				      
                                           type => "Sample"
                                           start_position => "beginning"
    				       sincedb_path => "/home/pavan/Downloads/dbflile"
                                         }
                                   }
                            filter {  
                                     csv {
                                          
                                           separator => ","
                                         }
                                   }
    
                            output { 
                                     elasticsearch { host => localhost protocol => http index => test   } 
                                     stdout { codec => rubydebug } 
                                   }

Can you please post me the configurations for .xlsx file (City of Indian Wells Ca Bus lic info EIN.xlsx) which exist in same directory.

4

Out of the box Logstash doesn't read .xlsx files.

1 Like
5

Magnusbaeck, do you have any suggestions to overcome my problem?

6

Well, you'll have to find something that reads your Excel files and outputs them in a format that Logstash accepts.