Issue 1) i have configured file beat (filebeat-5.0.0-rc1-linux-x86_64) with elasticsearch (without logstash). Its observed after few scan cycles it is not able to ship the data from log files to elasticsearch engine.
here is the command executed to run filebeat as bacjground process
./filebeat -c filebeat.full.yml -e"*"&
Issue 2) Is there any way to safely shutdown filebeat without killing process?
Issue 3) also, on restart of filebeat, it is reading contents from scrach instaed of reading from last line which was added in log file. timebeing i have set tail_files: true not sure is this going to solve the problem.
Please help. Note i have raised 3 question, pl respond by each issue wise.
here is the filebeat.full.yml contents:
filebeat.prospectors: - input_type: log paths: - /opt/apache-tomcat-8.0.27/logs/brandster-request-response.log document_type: brandster json.message_key: timestamp json.keys_under_root: true json.overwrite_keys: true json.add_error_key: true tail_files: true filebeat.idle_timeout: 60s output.elasticsearch: hosts: ["10.122.65.43:9200","10.122.65.45:9200","10.122.65.46:9200","10.122.65.47:9200","10.122.65.48:9200"] index: "brandster-idx" template.enabled: true template.path: "/opt/filebeat-5.0.0-rc1-linux-x86_64/filebeat.template.json" template.overwrite: true output.file: enabled: true path: "/opt/filebeat-5.0.0-rc1-linux-x86_64/temp" output.console: enabled: true pretty: true logging.to_files: true logging.files: path: "/opt/filebeat-5.0.0-rc1-linux-x86_64/logs"