# Will the total data gets shipped every time by filebeats?

**URL:** https://discuss.elastic.co/t/will-the-total-data-gets-shipped-every-time-by-filebeats/55479
**Category:** Beats
**Created:** [July 14, 2016, 7:02am UTC](https://discuss.elastic.co/t/will-the-total-data-gets-shipped-every-time-by-filebeats/55479 "2016-07-14T07:02:21Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)
#### Post date: [July 14, 2016, 7:02am UTC](https://discuss.elastic.co/t/will-the-total-data-gets-shipped-every-time-by-filebeats/55479/1 "2016-07-14T07:02:21Z")

</div>

I want to ship the logs from a server to elk. I would like to know whether all the logs gets shipped everytime or only newly generated logs?

for example logs at point are A+B, they get shipped to elastic search. after 1 hour the logs are A+B+c.  
My question is does filebeat ship only new generated C logs?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 14, 2016, 7:04am UTC](https://discuss.elastic.co/t/will-the-total-data-gets-shipped-every-time-by-filebeats/55479/2 "2016-07-14T07:04:28Z")

</div>

Filebeat keeps track of the files it has processed and only ships new lines.

---

<div class="post-metadata">

### Author: ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)
#### Post date: [July 14, 2016, 11:49pm UTC](https://discuss.elastic.co/t/will-the-total-data-gets-shipped-every-time-by-filebeats/55479/3 "2016-07-14T23:49:22Z")

</div>

filebeat:

# List of prospectors to fetch data.

prospectors:  
# Each - is a prospector. Below are the prospector specific configurations  
-  
paths:  
#- /var/log/\*.log  
- C:\filebeattesting\log\*.log

registry\_file: "C:/ProgramData/filebeat/registry"  
output:

### Elasticsearch as output

#elasticsearch:  
# Array of hosts to connect to.  
# Scheme and port can be left out and will be set to the default (http and 9200)  
# In case you specify and additional path, the scheme is required: [http://localhost:9200/path](http://localhost:9200/path)  
# IPv6 addresses should always be defined as: https://[2001:db8::1]:9200  
#hosts: ["10.40.104.27:9200"]  
logstash:  
# The Logstash hosts  
hosts: ["10.40.40.128:5044"]

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 4, 2016, 7:02am UTC](https://discuss.elastic.co/t/will-the-total-data-gets-shipped-every-time-by-filebeats/55479/4 "2016-08-04T07:02:26Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
