Corona Virus data

So i'd like to try and generate a better dashboard within kibana to display the data that has been made available by john hopkins below are the link to that data.

Time series and Github repo

Header looks like this
Province/State,Country/Region,Lat,Long,1/22/20,1/23/20,1/24/20,1/25/20,1/26/20,1/27/20,1/28/20,1/29/20,1/30/20,1/31/20,2/1/20,2/2/20,2/3/20,2/4/20,2/5/2 0,2/6/20,2/7/20,2/8/20,2/9/20,2/10/20,2/11/20,2/12/20,2/13/20,2/14/20,2/15/20,2/16/20,2/17/20,2/18/20,2/19/20,2/20/20,2/21/20,2/22/20,2/23/20,2/24/20,2/ 25/20,2/26/20,2/27/20

The above csv files pose a challenge as the its the columns that have the date stamp. I'm not sure how i'd go about parsing this.

Things i've tried:
autodetect_column_names => true .. which does not render the desired results.
{ "8" => "0", "Mainland China" => "Iraq", "@version" => "1", "157" => "0", "159" => "0", "@timestamp" => 2020-02-29T06:19:13.954Z, "tags" => [ [0] "confirmed_ts" ], "40" => "0", "43" => "0", "117" => "0", "location" => { "lon" => "%{Long}", "lat" => "%{Lat}" }, "22" => "0", "4" => "0", "5" => "0", "19" => "0", "64" => "0", "46" => "0", "162" => "0", "106" => "0", "Hainan" => nil, "109.7453" => "44", "path" => "csse_covid_19_time_series/time_series_19-covid-Confirmed.csv", "19.1959" => "33", "163" => "0", "message" => ",Iraq,33,44,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,5,7\r", "80" => "0", "host" => "desktop", "99" => "0", "144" => "0", "168" => "7", "124" => "0", "52" => "0", "72" => "0", "138" => "0", "33" => "0", "131" => "0", "62" => "0" }

I can't define the columns manually as the next day a new column will be added. I feel like a ruby expression is required here but I'm not sure. Any suggestions are welcomed.

That looks like it has interpreted a data row as a header, that would indicate logstash has re-ordered rows. Make sure you have pipeline.workers set to 1 and disable java_execution.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.