Importing Apache logs with FIlebeats tutorial doesn't work

[lgo (Luis)]

Hello

I'm following this tutorial, where you ingest some Apache logs with FIlebeats into Elasticsearch to visualize them in Kibana.

Elastic Cloud Tutorial: Getting Started with a sample dataset

I follow all the steps and none fails, but when I have a look at the index pattern there is no data, even changing dates.

Could someone help me? I'm doing this from Ubuntu 18.04.

Thanks in advance.

[ChrsMark (Chris Mark)]

Hi!

Are you able to start filebeat without errors to collect apache logs?

C.

[lgo (Luis)]

Yes, I think so.

I've tried with both what is said on that tutorial (with the -E and -M options) and with what appears if I go to the Kibana home page > Add log data > Apache logs > DEB (for Ubuntu) (sudo filebeat setup and sudo service filebeat start)

But with any of those it doesn't seem to work, in Kibana there is no data.

Thanks.

[ChrsMark (Chris Mark)]

Hi!

Could you check the logs and see if there is anything problematic? (you can fined them with tail -f /var/log/filebeat/filebeat)

C.

[lgo (Luis)]

Hi Chris

I've pasted the complete logs, not only the last ones, here: https://pastebin.com/gQyEAdUQ

I don't understand it very well, but to me it seems everything is ok.

Thank you.

[ChrsMark (Chris Mark)]

Thanks!

It doesn't seem to keep running...Could you try starting Filebeat like filebeat -e -d "*"?

C.

[lgo (Luis)]

That gave me an error, you meant:

filebeat -e -d "*"

right? That worked and the output is: https://pastebin.com/PKu9r11D

I have the logs file on Downloads but maybe I have to change the extension? I downloaded it and didn't touch it.

Thanks.

[ChrsMark (Chris Mark)]

Sorry for my typo!

It seems that the paths config is not an array.

Could you try with something like:

./filebeat -e \
     --modules=apache2 \
     --setup \
     -E cloud.id="Demo_Deployment:dXMtY2Vu..........jczODYke30="  \
     -E cloud.auth="sa_admin:s00p3rS3cr3t"  \
     -M "apache2.access.var.paths=[/Users/jamie/Downloads/apache_logs]"

C.

[lgo (Luis)]

Hello Chris

As I'm using Ubuntu I typed (I hope I didn't make a mistake "translating" it):

sudo filebeat -e --modules apache2 setup
    -E cloud.id="MPRO:whatever"
    -E cloud.auth="sa_admin:password"
    -M "apache2.access.var.paths=[/home/luis/Downloads/apache_logs]"

But it happens the same as previously, the logs are the same as the last pastebin.

Thanks.

[ChrsMark (Chris Mark)]

Hi!

This one works for me locally:
./filebeat -e -d "*" --modules apache -M "apache.access.var.paths=[/tmp/apache_logs]"

Note that this only collects logs with apache module, you still need to setup things with setup command etc. Also in order to make it work with cloud you need the cloud flags too.

C.

[lgo (Luis)]

Hi Chris

You saying that the paths config is not an array made me search on Google the last line of the filebeat logs and found https://discuss.elastic.co/t/unable-to-ingest-apache2-logs-through-filebeat/146309/3. I had to change the apache.yml file from:

var.paths: /home/luis/Downloads/apache_logs

to

var.paths:
  - '/home/luis/Downloads/apache_logs'

And now it works with filebeat -e -d "*"!!

Thanks a lot, Chris.