Configure an Index pattern. Unable to fetch mapping

Elastic Stack Kibana
1

Hi all,

I'm using Elasticsearch, Logstash and Kibana to visualize network packet dump file (csv). The data in csv is like this:
source address, destination address, frame lenth
7a07:2fe4:9f07:d674:ba09:34e4:ba09:35ac,7a07:2fe4:9f09:d674:ba09:34e4:ba09:36e6,74

My logstach configure file is:

input {
        file {
          path => "/Users/pengzhou/test1.csv"
          start_position => "beginning"
          type => "data"
    }

}

filter {
  csv {
      separator => ","
      columns => ["src","dst","framelenth"]
      }
}

output {
    elasticsearch {
        action => "index"
        hosts => ["localhost:9200"]
        workers => 1
    }
    stdout {}
}

In Kibana web API, I don't know what to input for the Index.
Thanks a lot.

2

The default index name Logstash uses is: logstash-%{+YYYY.MM.dd}

See https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index

You can control the index name if you want by configuring it in the output section. This example has it dynamic using the %{foo} syntax.

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "%{index}-%{+YYYY.MM.dd}"
  }
}
3

Thanks Tim,

Does it mean that ELK is only suitable for date related analysis?
I want to use it for ip address analysis, how should the index be?
Thanks

4

You don't need to have the -%{+YYYY.MM.dd} suffix at the end of your index pattern. If you aren't indexing data over several days or months, then it really isn't necessary. It is useful to have a date pattern in the index name if you have months worth of data. There are performance benefits in Kibana to set up a time-based index that way because it can use the _fieldStats API and query only from the indices that have data in a range you are interested in.

But if you are not continuously collecting data, and your packet data was collected over a few hours, then it is not as important to use a date pattern in the index name. Most examples in Logstash will have that, and the default has it, because it makes life easier for people that are constantly collecting data over long periods of time. It won't make it harder or easier to do your IP Address analysis though.

5

Thanks, but I still don't understand what to put as index to let Kibana show correctly.
I guess I don't need timestamp as suffix since my current dump file is only for one hour packet collection.
However, I don't know what to input in the index name or pattern on kibana web.
I tried using the names of columns in dump file but showed "unable to fetch mapping..."
Also some related guess such as source or destination didn't work.
Is there a format for this?