I have to read logs from different server so I am using filebeat for that.
in filebeat yml file I have mention
filebeat.inputs:
- type: log
enabled: true
paths:
- /home/3rdisearch-dev/project/mangeshs/logs/indexer-1/logs/*.log
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after
output.logstash:
hosts: ["XX.XX.X.XX:5044"]
and logstash conf file input as
input {
beats {
type => "test"
port => "5044"
}
}
sample log file
2021-09-28 11:04:56,060 INFO com.tdg.box.data.indexer.service.SolrIndexerService [Thread-165] Indexed docs:37272
2021-09-28 11:04:58,456 ERROR com.tdg.box.data.indexer.processor.BoxItemProcessor [Thread-166] Exception occured while getting file info, for fileid:473828671090 userId:177129950
com.box.sdk.BoxAPIResponseException: The API returned an error code [429 | xdg56cguwjrz1loo.0b2529a264e816335ba260ec03e13a4fa] rate_limit_exceeded - Request rate limit exceeded, please try again later
at com.box.sdk.BoxAPIResponse.<init>(BoxAPIResponse.java:92)
at com.box.sdk.BoxJSONResponse.<init>(BoxJSONResponse.java:32)
my questions are
- is filebeat is correct
- what exactly port 5044 is defined. is it logstash running port or anything
- changes in log file are not reflected in ELK