# Failing to capture oracle alert logs (kubernetes) **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 21, 2019, 9:35am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963 "2019-05-21T09:35:54Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Maurya\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maurya_m/32/46127_2.png) [@Maurya\_M](https://discuss.elastic.co/u/Maurya_M)\ **Post date:** [May 21, 2019, 9:35am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/1 "2019-05-21T09:35:54Z")
Hi, Having a setup wherein my oracle is deployed on kubernetes as statefulset, i need to monitor some logs other than the container logs, so from documentation and examples online was able create a filebeat config such as below: apiVersion: v1 kind: ConfigMap metadata: name: filebeat-config namespace: kube-system labels: k8s-app: filebeat [kubernetes.io/cluster-service:](http://kubernetes.io/cluster-service:) "true" data: filebeat.yml: |- filebeat.config: prospectors: # Mounted `filebeat-prospectors` configmap: path: {path.config}/prospectors.d/\*.yml # Reload prospectors configs as they change: reload.enabled: true modules: path: {path.config}/modules.d/\*.yml # Reload module configs as they change: reload.enabled: false filebeat.inputs: ``` - type: log paths: - /var/lib/docker/containers/*/*.log fields: type: docker qcdev: fields_under_root: true encoding: utf-8 ignore_older: 3h multiline: pattern: '^[[:space:]]|(at|\.{3})\b|^Caused by:' negate: true match: before - type: log paths: - /opt/oracle/oradata/diag/rdbms/*/ORCLCDB/trace/alert_ORCLCDB.log - /opt/oracle/oradata/diag/rdbms/*/ORCLCDB/trace/drcORCLCDB.log - /opt/oracle/oradata/diag/tnslsnr/*/listener/trace/listener.log document_type: oracle-tarce tags: ["oracle", "log"] fields: type: oracle qcdev: fields_under_root: true encoding: utf-8 ignore_older: 3h processors: - add_cloud_metadata: - add_kubernetes_metadata: in_cluster: true output.logstash: hosts: ['logstash-service:5044'] ``` but i am receiving only the container logs and i am not able to filter and data on the ELK stack i have deployed to visualize the data coming from a particular namespace. Any ideas / thoughts on this approach , appreciate the help & support here. Thanks, Maurya ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [May 21, 2019, 9:44am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/2 "2019-05-21T09:44:16Z")
Hi @Maurya_M and welcome 🙂 What version of Filebeat are you using? Take into account that [`prospectors` option was deprecated in 6.3](https://www.elastic.co/blog/brewing-in-beats-rename-filebeat-prospectors-to-inputs) and removed in 7.0. [`inputs`](https://www.elastic.co/guide/en/beats/filebeat/7.0/configuration-filebeat-options.html) should be used instead now. ---
**Author:** ![Maurya\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maurya_m/32/46127_2.png) [@Maurya\_M](https://discuss.elastic.co/u/Maurya_M)\ **Post date:** [May 21, 2019, 9:59am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/3 "2019-05-21T09:59:45Z")
hi @jsoriano, thanks fore replying back. i am using filebeat 6.3 - [docker.elastic.co/beats/filebeat:6.3.0](http://docker.elastic.co/beats/filebeat:6.3.0), as i am doing some quick logging from example , what would be the recommended image versions for the entire Elastic stack ? ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [May 21, 2019, 10:25am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/4 "2019-05-21T10:25:05Z")
I would recommend you to use the latest versions of the Elastic Stack, lots of fixes related to kubernetes support have been done till 6.3, and 7.0 was released recently. In any case I was asking for the version just to confirm that you are in a version where the `prospectors` options still work. I see you are configuring Filebeat to read the logs from `/opt/oracle`, is this directory mounted in the Filebeat container? It should be so Filebeat can read the files. ---
**Author:** ![Maurya\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maurya_m/32/46127_2.png) [@Maurya\_M](https://discuss.elastic.co/u/Maurya_M)\ **Post date:** [May 21, 2019, 11:22am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/5 "2019-05-21T11:22:30Z")
have added the volume mounts in the filebeat container ( see snippet below): Is this correct, Also i have the filebeat as deamonset , do i need to deploy as side-car container along with the oracle containers? volumeMounts: - name: config mountPath: /etc/filebeat.yml readOnly: true subPath: filebeat.yml - name: inputs mountPath: /usr/share/filebeat/inputs.d readOnly: true - name: data mountPath: /usr/share/filebeat/data - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true - **name: oracledata** \*\* mountPath: /opt/oracle/oradata\*\* \*\* readOnly: true\*\* volumes: - name: config configMap: defaultMode: 0600 name: filebeat-config - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers - **name: oracledata** \*\* hostPath:\*\* \*\* path: /opt/oracle/oradata\*\* - name: inputs configMap: defaultMode: 0600 name: filebeat-inputs # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart - name: data hostPath: path: /var/lib/filebeat-data type: DirectoryOrCreate ---
**Author:** ![Maurya\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maurya_m/32/46127_2.png) [@Maurya\_M](https://discuss.elastic.co/u/Maurya_M)\ **Post date:** [May 23, 2019, 10:38am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/6 "2019-05-23T10:38:30Z")
@jsoriano, i was finally able to deploy using sidecar-container to my oracle container, but i am not seeing any message as per my alert.log configured on Kibana!! Any ideas how to debug/ have checkpoints if my configuration / logs are reaching the elastic search & kibana. Having Filebeat --\> Logstash --\> ES --\> Kibaba setup. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [May 23, 2019, 11:55am UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/7 "2019-05-23T11:55:21Z")
@Maurya_M, some things you can check: - Double-check that files are accessible from the filebeat container - Check in filebeat logs that harvesters are being started for your log files (with messages like `Harvester started for file...`) - [Enable debug](https://www.elastic.co/guide/en/beats/filebeat/7.0/enable-filebeat-debugging.html) to see what events are being sent by filebeat - Check logstash logs for any error - Try to configure filebeat to send events directly to Elasticsearch, to discard problems in logstash By the way, is there any reason why you are using logstash? for many use cases it is enough with sending the data directly from filebeat to elasticsearch, and this way it is ieasier to take advantage of [filebeat modules](https://www.elastic.co/guide/en/beats/filebeat/7.0/filebeat-modules-overview.html). ---
**Author:** ![Maurya\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maurya_m/32/46127_2.png) [@Maurya\_M](https://discuss.elastic.co/u/Maurya_M)\ **Post date:** [May 23, 2019, 12:17pm UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/8 "2019-05-23T12:17:11Z")
thanks @jsoriano for the suggestions, - Harvester has started for the file mentioned in the "path" Harvester started for file: /opt/oracle/oradata/diag/tnslsnr/oradb-0/listener/trace/listener.log 2019-05-23T11:54:48.561Z INFO log/harvester.go:254 Harvester started for file: /opt/oracle/oradata/diag/rdbms/oracdb0/ORCLCDB/trace/alert\_ORCLCDB.log - Does this suffice the files are accesible from the fb container? - Do have debug option with in my side-car filebeat deloyment args: [ "-c", "/etc/filebeat.yml", "-e", ] should i change this to "filebeat -e -d "\*" " - where do i see these verbose message? On the logstash side, the idea was to had fields / remove unwanted / multiline handling, but i guess these can be done by filebeat filter too. But somehow got carried away to use the whole Elastic stack for now, may drop logstash later, but have added no filter in logstash just these configuration below: logstash.yml: | http.host: "0.0.0.0" path.config: /usr/share/logstash/pipeline logstash.conf: | # all input will come from filebeat, no local logs input { beats { port =\> 5044 } } output { elasticsearch { hosts =\> ["elasticsearch-logging:9200"] manage\_template =\> false index =\> "%{[kubernetes][namespace]}" } } Btw, i did run from dev-tools query on my namespace , but i dont see any of the alert log data which i verified got created, but not getting pushed to ES. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 20, 2019, 12:17pm UTC](https://discuss.elastic.co/t/failing-to-capture-oracle-alert-logs-kubernetes/181963/9 "2019-06-20T12:17:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.