How to push a specific log to an existing index?

Hello Team,

I have a specific log that I want to push to elasticsearch's index.

Log name: example-2018-02-04.log
index name: filebeat-2019.02.04

Here is my filebeat.yml:

filebeat:
prospectors:
-
input_type: log
paths:
- /home/testuser/example-2019-02-04.log
fields:
type: console_log
environment: Production
layer: App
servername: prod-app
document_type: console_log
multiline.pattern: '^.|.|.|.| \d{2}:\d{2}:\d{2} (?:AM|PM)'
multiline.negate: true
multiline.match: after
-
input_type: log
paths:
- /home/ubuntu/example-2019-02-04.log
fields:
type: access_log
environment: Production
layer: App
servername: prod-app
document_type: access_log
multiline.pattern: '^.|.|.|.| \d{2}:\d{2}:\d{2} (?:AM|PM)'
multiline.negate: true
multiline.match: after

output:
logstash:
hosts: ["http://localhost:5044"]
bulk_max_size: 2048

shipper:
logging:
to_syslog: false
to_files: true
files:
path: /var/log/filebeat
name: filebeat.log
keepfiles: 7
rotateeverybytes: 10485760 # = 10MB
level: info

When I try to search for logs on 4th feb, 2019 I get the below error:

Discover: Bad Gateway
Less Info
OK
SearchError: Bad Gateway
at https://xxxxxx.com/bundles/commons.bundle.js:3:2878241
at processQueue (https://xxxxxx.com/bundles/vendors.bundle.js:277:199684)
at https://xxxxxx.com/bundles/vendors.bundle.js:277:200647
at Scope.$digest (https://xxxxxx.com/bundles/vendors.bundle.js:277:210409)
at Scope.$apply (https://xxxxxx.com/bundles/vendors.bundle.js:277:213216)
at done (https://lxxxxxx.com/bundles/vendors.bundle.js:277:132715)
at completeRequest (https://xxxxxx.com/bundles/vendors.bundle.js:277:136327)
at XMLHttpRequest.requestLoaded (https://xxxxxx.com/bundles/vendors.bundle.js:277:135223)

Every day a new index is created, but I want to push to an old one.

Regards,
Ashutosh.

Hi Ashutosh,

You are sending data to Logstash. Are you handling this accordingly in Logstash?

We discussed something similar to this in this thread:

https://discuss.elastic.co/t/logstash-conditional-indexes/170080/6

Cheers!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.