Logstash index without timestemp

Elastic Stack Logstash
1

hello all,

i'm ELK newbie and i having a few problems with log reading.

i'm trying to read a csv file that have no times-temp field in it.
for that i built configuration file using this guide help (https://kevinkirsche.com/2014/08/25/using-logstash-to-import-csv-files-into-elasticsearch/).
but still the in kibana i cant create index pattern for the specific file. i cant help to find the right Input type to use.

appreciate the help.

guy

2

but still the in kibana i cant create index pattern for the specific file.

What does your elasticsearch output in your Logstash config look like?

i cant help to find the right Input type to use.

What do you mean? If you're talking about the type field set via the input option of your input plugin, that's just an arbitrary string.

3

hi,

(sorry abut the typo at the provisos massage)

attached the config file:

input {
file {
path => "/tmp/*.csv
type => "file"
start_position => "beginning"
}
}
filter {
csv {
columns => ["Sess IDL4 ProtoIn", "ternal IP PortI", "External IP PortE", "TTL DPI Stat","App"]
separator => " "
}
}
output {
elasticsearch {
action => "index"
host => "localhost"
index => "logstash-%{+YYYY.MM.dd}"
workers => 1
}
}

i know i have to change the index but to what?

thanks,
guy

4

Why would you have to change the index name?

5

because in the kibana i can't create index pattern and no data is being read from the csv file.

6

Are there any indexes in ES? If not you can't create an index pattern. Changing the index name won't make a difference.

If Logstash isn't reading your CSV files it's probably because it's tailing the files and no more data is being added. start_position => "beginning" only matters for unseen files. This matter has been covered here many many times. Setting sincedb_path to "/dev/null" effectively disables the sincedb feature which tracks the current position in each file (and most likely points to the end of the file right now).

7

Thanks for your answare and for your patienc. but as i said ELK is new to me so i'll try to better expline myself.

I have a static csv file with 500k lines i have to slice and dice but i can't understand how to connect to the file and display all the data in it.
Using the sincedb_path will resolve my problem?

Thanks

8

Yes, probably. I suggest you try it out rather than ask me about it. I'm just guessing.