Where are the remote logs saves in my local ELK instance?

Elastic Observability
1

Dear Elastic community,
I am using filebeat and I am able to read the logs using the ELK GUI.
Where are the logs stored in my local ELK instance please? I would love to have them stored by Hostname, each hostname in a different folder. Is that possible?
Best regards,

2

The data is stored in Elasticsearch, but the structure of the files that it uses are not user configurable and they are not directly readable on the filesystem.

However, why do you want to do this?

3

I need to do this in order to browse through my logs via console, in case ELK GUI crashes or goes down for some reason.

4

Then you need to use Elasticsearch APIs, you cannot read the files on the filesystem like that.

5

Is there a way to store syslogs in the ELK instance in a folder containing hostname, using Filebeat?

6

You could use Configure the File output | Filebeat Reference [8.2] | Elastic, but it's not really something we see a lot of.

7

Is there some filebeat configuration template so I can save in a folder: Hostname > auth.log , syslog.log please? That would be really helpful

8

No, not possible, with the file output you can change the name of the output file based on the value of a field in your document, but not the path. Check the link in the previous answer.

Also, if you use the file output you will not have your logs in Elasticsearch and won't be able to see them in Kibana, if you want to see them in Kibana, then you need to use the elasticsearch output.

As already said, with your data in Elasticsearch you cannot access it directly by reading some file, you need to make a query to elasticsearch to see your logs, for that you can use Kibana or some of th Elasticsearch REST API.

9

I understand that, I have seen the previous link.
It is ok to have redundant logs: ELK format and Plain text kind, just as backup.
I don't want to replace the ELK format. I want to keep it, but also save extra logs in for exampe /srv/logs. like ntp01-server-syslog.log or ntp01-server-auth.log.

In other words: Use filebeat agent to send the /var/log/*.log to the ELK instance /srv/log/$hostname.syslog.log for example

Thank you very much in advance. It is really nice having such a community!!