# Filebeat to send application logs to elasticsearch
**URL:** https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507
**Category:** Beats
**Tags:** filebeat
**Created:** [June 17, 2020, 4:13pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507 "2020-06-17T16:13:29Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@jacobzh](https://discuss.elastic.co/u/jacobzh)
#### Post date: [June 17, 2020, 4:13pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/1 "2020-06-17T16:13:29Z")
Hi,
I want to use filebeat to ship developers application logs to elasticsearch running in a developers servers. Developers application generates a log file in a particular folder. Whenever the log file reaches a configured size, then a new log file is started like app.log.$date1, app.log.$date2, app.log.$date3 etc. Can I make filebeat read new log files whenever the new log file is started.
Most important I have to see content of the application log files. Right now I can see only message.
Also, is there a possibility I can only send a subset of traces from these application log files to elasticsearch or does it require logstash to do this?
---
### Author:  [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)
#### Post date: [June 18, 2020, 7:04am UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/2 "2020-06-18T07:04:39Z")
Use the `log` input [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html)
If you need send some metadata, you can use `tags`
---
### Author:  [@jacobzh](https://discuss.elastic.co/u/jacobzh)
#### Post date: [June 18, 2020, 1:09pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/3 "2020-06-18T13:09:36Z")
Hi Mario,
Thank you for quick replay. Below my filebeat configuration:
=========================== Filebeat inputs =============================
filebeat.inputs:
# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.
**- type: log**
# Change to true to enable this input configuration.
**enabled: true**
# Paths that should be crawled and fetched. Glob based paths.
# paths:
**paths:**
**/var/log/localhost.log**
**/var/log/server.log\_y\***
# The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
**multiline.pattern: '^['**
# Defines if the pattern set under pattern should be negated or not. Default is false.
**multiline.negate: true**
# Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
# that was (not) matched before or after or as long as a pattern is not matched based on negate.
# Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
**multiline.match: after**
============================= Filebeat modules ===============================
**filebeat.config.modules:**
# Glob pattern for configuration loading
**path: ${path.config}/modules.d/\*.yml**
# Set to true to enable config reloading
**reload.enabled: false**
#==================== Elasticsearch template setting ==========================
**setup.template.settings:**
**index.number\_of\_shards: 3**
#index.codec: best\_compression
#\_source.enabled: false
#============================== Kibana =====================================
# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
**setup.kibana:**
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: [http://localhost:5601/path](http://localhost:5601/path)
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
**host: "x.x.53.201:5601"**
#-------------------------- Elasticsearch output ------------------------------
**output.elasticsearch:**
# Array of hosts to connect to.
**hosts: ["x.x.53.201:9200"]**
What I can see from Kibana:
---
### Author:  [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)
#### Post date: [June 18, 2020, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/4 "2020-06-18T14:00:07Z")
> [@jacobzh](#):
>
> **/var/log/localhost.log**
Use a wildcard `/var/log/*.log`
---
### Author:  [@jacobzh](https://discuss.elastic.co/u/jacobzh)
#### Post date: [June 18, 2020, 2:38pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/5 "2020-06-18T14:38:49Z")
After I used wild card /var/log/\*.log, I can’t see a message and files content.
---
### Author:  [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)
#### Post date: [June 18, 2020, 3:41pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/6 "2020-06-18T15:41:47Z")
Ok, that doesn't make sense 😄 It's a typical wildcard, I thought it was implicit, you can try `/var/log/*` to ingest all files in that folder (just to double check, I mean in the `path` object )
---
### Author:  [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [June 18, 2020, 3:58pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/7 "2020-06-18T15:58:19Z")
Better change the log files name such as: $date1.app.log $date2.app.log
So, the configuration of filebeat is simpler. just /var/log/\*.log
Another suggestion, if you want to put the message contents of the apps log to the meaningfull fields, use logstash to handle the specific log format, also, if there are not syslog format, better put into another folder, such as /var/log/specificapps/\*.log
Then use grok from logstash filter to dissect the message content into fields in the index.
Regards,
Fadjar Tandabawana
---
### Author:  [@jacobzh](https://discuss.elastic.co/u/jacobzh)
#### Post date: [June 18, 2020, 4:06pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/8 "2020-06-18T16:06:17Z")
I changed path to /var/log/\*. Same result. I can see only /var/log/syslog and /var/log/auth.log file. No messages from my files.
---
### Author:  [@jacobzh](https://discuss.elastic.co/u/jacobzh)
#### Post date: [June 19, 2020, 2:42pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/9 "2020-06-19T14:42:10Z")
Hi Fadjar,
I changed path to /var/log/awsapi/\*.log and used logstash to handle specific log format. Noting happened and it getting worth:
I can’t see even message with my files and loosed the server(awsapi02) for dashboard:
---
### Author:  [@fadjar340](https://discuss.elastic.co/u/fadjar340)
#### Post date: [June 19, 2020, 2:45pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/10 "2020-06-19T14:45:39Z")
Did you already add the path into the filebeat conf?
---
### Author:  [@jacobzh](https://discuss.elastic.co/u/jacobzh)
#### Post date: [June 19, 2020, 4:08pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/11 "2020-06-19T16:08:07Z")
Yes, I did
Thank you,
Jacob Zhitnitsky
St. System Administrator
Folio Financial, Inc
[zhitnitskyj@foliofinancial.com](mailto:zhitnitskyj@foliofinancial.com)
work: 703-245-4813
cell: 571-405-1043
> **[Home | Folio Financial](https://www.foliofinancial.com)**
>
> Folio Financial is a unique brokerage, custodian, and financial technology company delivering advanced fintech to investors and those who serve them.
Thank you,
Jacob Zhitnitsky
Sr. System Administrator
Folio Financial, Inc
[zhitnitskyj@foliofinancial.com](mailto:zhitnitskyj@foliofinancial.com)
work: 703-245-4813
cell: 571-405-1043
> **[Home | Folio Financial](https://www.foliofinancial.com)**
>
> Folio Financial is a unique brokerage, custodian, and financial technology company delivering advanced fintech to investors and those who serve them.
Please do not use email to transmit orders for securities or for other time-sensitive messages. Broker-dealer securities products and services are offered only through Folio Investments, Inc. Member FINRA/SIPC. Folio Investments, Inc. conducts business under various trade names, including: Folio Financial, Folio Institutional, Folio Investing, FolioFirst, VIAFolio, Folio Client, and DirectIndexer. Folio Investments, Inc. is a subsidiary of Folio Financial, Inc. and an affiliate of First Affirmative Financial Network, LLC and Folio Research, LLC. Investment advisory services are provided through First Affirmative Financial Network, LLC and Folio Research, LLC.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 17, 2020, 6:08pm UTC](https://discuss.elastic.co/t/filebeat-to-send-application-logs-to-elasticsearch/237507/12 "2020-07-17T18:08:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.