Trying to import a stock price file into Elastic


I am a newbie and still learning.
Am trying to import a stock price file into Elastic.

Spent some time to setup a Logstash config file, but didn't succeed.

This is the format of the stock price file:

Date format: mm/dd/yyyy
Time format: HH:mm
The other fields are: opening price, highest price of the day, lowest price of the day, closing price, volume, open interest

Please help me out. Should I use a Logstash config file or can I use some plugin to make it easier for me?

Thanks very much for your help.

Kind regards,

Hi Sharon,

I'd approach the grok plugin in the filter. A quick look into the pattern, you could start with something like this:


There are many awesome examples here:

You create a config file as you already did, and pass your data through the filter using the great GROK plugin to parse it before sending them to ES.

Lastly you can use this nice web app to test out your dataset against a grok pattern:

The CSV filter will be as good, see

If the format of the file is exactly the same for every line and you want to be a bit adventurous you can try the dissect filter,
Here is a starter example.

input {
  generator {
    lines => [
    count => 3

filter {
  dissect {
    mapping => {
      message => "%{date},%{time},%{opening_price},%{highest_price_of_the_day},%{lowest_price_of_the_day},%{closing_price},%{volume},%{open_interest}"
    add_field => { datetime => "%{[date]} %{[time]}" }
    convert_datatype => {
      opening_price => "float"
      highest_price_of_the_day => "float"
      lowest_price_of_the_day => "float"
      closing_price => "float"
      volume => "int"
      open_interest => "int"

output {
  stdout {
    codec => rubydebug

After you are happy with the shape of the events that you see in the terminal, you can replace the generator input with the file input and the stdout output (+ codec) with the elasticsearch output.

Thanks for your help ericohtake and guyboertje!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.