prat
December 4, 2020, 6:12pm
1
Hi Team,
I am not able to see latest logs in discover tab in kibana. the last logs I can see is of 1 Dec 2020 but not after that.
I can see folders getting created till today in /opt/elasticsearch/nodes/0/indices/, then why new logs are not shown in kibana.
(/opt/elasticsearch is path.data)
I checked all the services are up. What I need to check to fix this issue?
Can you please help
Thanks,
Check date filter to ensure the correct range is correct?
If you look at the index under Management do you see the size or doc count increasing?
Is the heath yellow or green?
How are you ingesting the data?
prat
December 4, 2020, 6:35pm
3
Hi @aaron-nimocks ,
Thanks for your reply.
Check date filter to ensure the correct range is correct
How can I check this?
Data flow goes like,
Filebeat to kafka , kafka to logstatsh and then to
elasticsearch.
Check that filter and just use last 24 or 48 hours. Something after December 1st.
prat
December 4, 2020, 6:44pm
5
Hi @aaron-nimocks ,
When I login to kibana and click on discover, 15 mins is by default selected and I see no logs. I have to change that to 7 days ago and then I see last logs of 1 dec.
Do you have Stack Monitoring setup where you can see your Logstash ingestion stats? It would show if you have been receiving data.
Or I would check the logstash logs to see if anything is being processed.
prat
December 6, 2020, 7:00am
7
Hi @aaron-nimocks ,
Thanks for reply. yesterday I performed some housekeeping on /var,/opt to free up some space and also kafka service was down, I restarted it and was able to see logs in filebeat.
I saw some numbers in logstash in stack monitoring.
I will monitor if new logs are coming to kibana or not.
prat
December 6, 2020, 7:07am
8
I can see yesterday's (5th Dec) index got created after making some space and restarting kafka.
below is the screen shot of filebeat indices.
Why it is saying 7 indices have lifecycle errors? what it means?
for how many days, indices have retention period and do they get auto deleted after that period?
below screen shot is after clicking on Show errors
Thanks,
I am not sure for ILM policies. Might be best to make a new topic for this specific issue.
Most likely it was an error due to the disk being full?
prat
December 8, 2020, 5:23pm
11
Hi @aaron-nimocks
Thanks for your quick response.
I will create new topic but its always there even if there is free space.
1 Like
system
(system)
Closed
January 5, 2021, 5:23pm
12
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.