How to make continuously insert -> Kibana Sample Data

Elastic Stack Kibana
1

Hi fellows!

I'm a newbie in Elastic Stack and I'm trying to learn day by day.
I've load/installed Kibana Sample Data (specifically Flights data).

The thing is, I notice that lots of data inserts with the current timestamp, even data with timestamp in the future (for example I can query next week and there is data inserted). I think 2 months are inserted.

I also noted that every few minutes there is some new doc (as far as I understand is not data inserted in "real time" but is the data that was inserted when install and as time goes by, data is recovered.

My question is: is there a way to "reinsert" data continuously into ELK Stack so it looks more like real time data?. And also "fill" the gaps between the data inserted?.

For example, reinsert the data (that is in "flights.json.gz") every 5 minutes continuously and indefinitely into the same kibana_sample_data_logs index?.

I've tryied to understand it and acheive such a thing but didn't have any luck.

I'm using: Elastic Stack 7.6.0

Thank you very much!
Kind regards!

time
time907×283 13.6 KB

every_few_minutes
every_few_minutes1032×448 47.8 KB

2

Update:

I manage to insert data into the index with:

curl -XPOST 'localhost:9200/kibana_sample_data_logs/_doc' -H "Content-Type: application/json" -d @log.json

{"_index":"kibana_sample_data_logs","_type":"_doc","_id":"q5VhwHEBInE1gNjuHYat","_version":1,"result":"created","_shards":{"total":1,"successful":1,"failed":0},"_seq_no":14191,"_primary_term":3}root@elk:/tmp#

cat log.json:

{"agent":"TEST","bytes":6219,"clientip":"223.87.60.27","extension":"deb","geo":{"srcdest":"IN:US","src":"IN","dest":"US","coordinates":{"lat":39.41042861,"lon":-88.8454325}},"host":"artifacts.elastic.co","index":"kibana_sample_data_logs","ip":"223.87.60.27","machine":{"ram":8589934592,"os":"win 8"},"memory":null,"message":"223.87.60.27 - - [2018-07-22T00:39:02.912Z] \"GET /elasticsearch/elasticsearch-6.3.2.deb_1 HTTP/1.1\" 200 6219 \"-\" \"Mozilla/5.0 (X11; Linux x86_64; rv:6.0a1) Gecko/20110421 Firefox/6.0a1\"","phpmemory":null,"referer":"http://twitter.com/success/wendy-lawrence","request":"/elasticsearch/elasticsearch-6.3.2.deb","response":200,"tags":["success","info"],"timestamp":"2020-04-28T12:37:10.912Z","url":"https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.2.deb_1","utc_time":"2018-07-22T00:39:02.912Z","event":{"dataset":"sample_web_logs"}}

  • I manually updated the timestamp to currently to test and docs_count reflects it but cannot see the doc in "Discover".

Thanks a lot!

3

Awesome. Another alternative for data is the makelogs node package which you can run to add data into elasticsearch.
Or, easiest way, run a metricbeat locally and that will get you data every 30s continuously.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.