Hello guys,
The Scinario:
I have 4 dockers, Elasticsearch, kibana, filebeat, dvwa.
So basically, I should attack dvwa and see the apache logs on kibana dashboard.
I use Ubuntu server 20.4 to set up the docker on it and it's a virtual machine on vbox
for the attacking, I use Kali Linux.
What happened before the problem:
after I set up the Ubuntu server on virtual box (vbox),
I used a similar installation script but has some more apt update and docker installation commands and everything it was working fine, I can access my Elasticsearch, I can see the Access and error logs ECS - Filebeat Apache module dashboard.
Then I used this removing script:
#!/usr/bin/env bash
set -e
echo -e "\e[34m {+} Stop and remove all containers \e[0m"
docker stop elasticsearch kibana dvwa filebeat
docker rm elasticsearch kibana dvwa filebeat
echo -e "\e[34m {+} Remove all images \e[0m"
docker image rm vulnerables/web-dvwa docker.elastic.co/beats/filebeat:7.15.2 docker.elastic.co/kibana/kibana:7.15.2 docker.elastic.co/elasticsearch/elasticsearch:7.15.2 -f
echo -e "\e[36m The End\e[0m"
After the removing I powered off the vm and I started again, and I used the next installation script.
The installation:
I used this script to install the images and start the containers and link everything and configure it.
NOTE: The configuration file (filebeat.yml) in the configure the filebeat step, it's already on the system so it's not missed.
echo -e "\e[34m {+} Wait for 3 minutes\e[0m"
sleep 180
echo -e "\e[34m {+} Start Filebeat and connect it with Kibana\e[0m"
docker run -it \
--link elasticsearch --link kibana \
docker.elastic.co/beats/filebeat:7.15.2 \
setup -E setup.kibana.host=kibana:5601 \
-E output.elasticsearch.hosts=["elasticsearch:9200"]
echo -e "\e[34m {+} Wait for 3 minutes\e[0m"
sleep 180
echo -e "\e[34m {+} Configure the Filebeat with the filebeat.yml file\e[0m"
docker run -d \
--link elasticsearch --link kibana \
--name=filebeat \
--user=root \
--volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
--volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
--volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
docker.elastic.co/beats/filebeat:7.15.2 filebeat -e -strict.perms=false \
-E output.elasticsearch.hosts=["elasticsearch:9200"]
echo -e "\e[34m {+} Wait for 3 minutes\e[0m"
sleep 180
echo -e "\e[34m {+} Customize the configuration and start DVWA docker\e[0m"
docker run \
--label co.elastic.logs/module=apache2 \
--label co.elastic.logs/fileset.stdout=access \
--label co.elastic.logs/fileset.stderr=error \
--label co.elastic.metrics/module=apache \
--label co.elastic.metrics/metricsets=status \
--label co.elastic.metrics/hosts='${data.host}:${data.port}' \
--detach=true \
--name dvwa \
-p 80:80 \
vulnerables/web-dvwa
echo -e "\e[34m {+} Check the running containers\e[0m"
docker ps
echo -e "\e[36m The End\e[0m"
After this, I can see the Overview under Security and I can see the events there,
but when I try to go to "Access and error logs ECS - Filebeat Apache module dashboard."
All I see is 'Error' all around and
'Unable to update UI Settings
Request failed with status code: 429'
What I tried to do?
I already know that code 429 should be something related to the disk usage exceeded flood-stage watermark, index has read-only-allow-delete block, but I didn't see this error.
Also, I tried to check the cluster and everything and is not really that obvious which one I should delete or what I should exactly.
I hope you can help me guys and I hope I made this clear as possible.
Thanks.