# Logstash does not send all my data

**URL:** <https://discuss.elastic.co/t/logstash-does-not-send-all-my-data/128901>\
**Category:** Logstash\
**Created:** [April 20, 2018, 2:49pm UTC](https://discuss.elastic.co/t/logstash-does-not-send-all-my-data/128901 "2018-04-20T14:49:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jose\_Campos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jose_campos/32/11259_2.png) [@Jose\_Campos](https://discuss.elastic.co/u/Jose_Campos)\
**Post date:** [April 20, 2018, 2:49pm UTC](https://discuss.elastic.co/t/logstash-does-not-send-all-my-data/128901/1 "2018-04-20T14:49:11Z")

</div>

Hello

I have a problem with logstash 6.2.4, when I run my logstash manually, elasticsearch if it shows me my full information, for example a full hour. But if I let it run on systemctl in this case I use centos, it only loads me 30 minutes.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/d/6d5846e7918c0dd31d0c72c9343c7e0604a04ef7.png)

I'm working with multiple pipelines. I do not know if that has something to do with it.

- pipeline.id: nginx  
pipeline.workers: 2  
path.config: "/etc/logstash/conf.d/logstash-nginx.conf"  
queue.type: persisted

This is my config file

input {  
file {  
path =\> "/var/log/Nginx/syslogProxy"  
start\_position =\> "beginning"  
stat\_interval =\> 1  
discover\_interval =\> 15  
sincedb\_path =\> "/dev/null"  
sincedb\_write\_interval =\> 15  
}  
}

filter {

grok {  
patterns\_dir =\> "/etc/logstash/patterns/"  
match =\> { "message" =\> "%{FORTIDATE:logtimestamp}" }  
}

date { match =\> ["logtimestamp", "MMM dd HH:mm:ss", "MMM d HH:mm:ss"] }

grok {  
patterns\_dir =\> "/etc/logstash/patterns"  
match =\> { "message" =\> "%{NGINX\_ACCESS}" }  
remove\_tag =\> ["\_grokparsefailure"]  
add\_tag =\> ["nginx\_access"]  
}

grok {  
match =\> { "agent" =\> "[(]%{DATA:OS}; %{DATA:version}; %{DATA:device}[)]" }  
match =\> { "agent" =\> "[(]%{DATA:OS}[)]" }  
}

geoip {  
source =\> "clientip"  
}

mutate {  
remove\_field =\> ["type", "path", "host"]  
}  
}

output {

elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "logstash-nginx"  
}  
}

-Jose

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 18, 2018, 2:49pm UTC](https://discuss.elastic.co/t/logstash-does-not-send-all-my-data/128901/2 "2018-05-18T14:49:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.