Importing Apache logs with FIlebeats tutorial doesn't work

Elastic Stack Beats
1

Hello

I'm following this tutorial, where you ingest some Apache logs with FIlebeats into Elasticsearch to visualize them in Kibana.

I follow all the steps and none fails, but when I have a look at the index pattern there is no data, even changing dates.

Could someone help me? I'm doing this from Ubuntu 18.04.

Thanks in advance.

2

Hi!

Are you able to start filebeat without errors to collect apache logs?

C.

3

Yes, I think so.

filebeat_commands
filebeat_commands1017×297 41.4 KB

I've tried with both what is said on that tutorial (with the -E and -M options) and with what appears if I go to the Kibana home page > Add log data > Apache logs > DEB (for Ubuntu) (sudo filebeat setup and sudo service filebeat start)

But with any of those it doesn't seem to work, in Kibana there is no data.

kibana_indexpattern
kibana_indexpattern1919×442 46.4 KB

Thanks.

4

Hi!

Could you check the logs and see if there is anything problematic? (you can fined them with tail -f /var/log/filebeat/filebeat)

C.

5

Hi Chris

I've pasted the complete logs, not only the last ones, here: https://pastebin.com/gQyEAdUQ

I don't understand it very well, but to me it seems everything is ok.

Thank you.

6

Thanks!

It doesn't seem to keep running...Could you try starting Filebeat like filebeat -e -d "*"?

C.

7

That gave me an error, you meant:

filebeat -e -d "*"

right? That worked and the output is: https://pastebin.com/PKu9r11D

I have the logs file on Downloads but maybe I have to change the extension? I downloaded it and didn't touch it.

logs
logs1852×419 134 KB

Thanks.

8

Sorry for my typo!

It seems that the paths config is not an array.

Could you try with something like:

./filebeat -e \
     --modules=apache2 \
     --setup \
     -E cloud.id="Demo_Deployment:dXMtY2Vu..........jczODYke30="  \
     -E cloud.auth="sa_admin:s00p3rS3cr3t"  \
     -M "apache2.access.var.paths=[/Users/jamie/Downloads/apache_logs]"

C.

1 Like
9

Hello Chris

As I'm using Ubuntu I typed (I hope I didn't make a mistake "translating" it):

sudo filebeat -e --modules apache2 setup
    -E cloud.id="MPRO:whatever"
    -E cloud.auth="sa_admin:password"
    -M "apache2.access.var.paths=[/home/luis/Downloads/apache_logs]"

But it happens the same as previously, the logs are the same as the last pastebin.

Thanks.

10

Hi!

This one works for me locally:
./filebeat -e -d "*" --modules apache -M "apache.access.var.paths=[/tmp/apache_logs]"

Note that this only collects logs with apache module, you still need to setup things with setup command etc. Also in order to make it work with cloud you need the cloud flags too.

C.

1 Like
11

Hi Chris

You saying that the paths config is not an array made me search on Google the last line of the filebeat logs and found https://discuss.elastic.co/t/unable-to-ingest-apache2-logs-through-filebeat/146309/3. I had to change the apache.yml file from:

var.paths: /home/luis/Downloads/apache_logs

to

var.paths:
  - '/home/luis/Downloads/apache_logs'

And now it works with filebeat -e -d "*"!!

Thanks a lot, Chris.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.