# Logstash not sending data to ES **URL:** \ **Category:** Logstash\ **Created:** [June 2, 2017, 6:14am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-es/88020 "2017-06-02T06:14:07Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![keshav](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@keshav](https://discuss.elastic.co/u/keshav)\ **Post date:** [June 2, 2017, 6:14am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-es/88020/1 "2017-06-02T06:14:07Z")
I have a logstash config which reads nginx logs and sends data to elasticsearch. The log file contains two endpoints viz `/index` and `/info`. Logstash parses the `/index` endpoint and sends the data to ES but it doesn't do the same with `/info` endpoint. The issue is that when I run logstash in foreground using `/opt/logstash/bin/logstash` to debug the problem, it is able to parse both the endpoints and send data to ES, whereas as a daemon it does so only with the `/index` endpoint. I don't understand what I'm doing wrong here, can anybody please point me in the right direction? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 5, 2017, 3:55am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-es/88020/2 "2017-06-05T03:55:41Z")
Please show your Logstash configuration files and relevant contents from the Logstash log. ---
**Author:** ![keshav](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@keshav](https://discuss.elastic.co/u/keshav)\ **Post date:** [June 5, 2017, 4:54am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-es/88020/3 "2017-06-05T04:54:55Z")
This is my logstash config file - ``` input { file { start_position => "beginning" path => '/var/log/nginx/tracker.log' codec => plain { charset => "ISO-8859-1" } type => 'tracker' } } filter { if [type] == 'tracker' { urldecode { } grok { match => [ "message", '%{IP:clientip} %{GREEDYDATA} \[%{HTTPDATE:timestamp}\] "%{GREEDYDATA:request} HTTP%{GREEDYDATA}" %{INT:status} %{INT:body_bytes_sent} %{GREEDYDATA} \* "%{GREEDYDATA:http_post_data}"', "message", '%{GREEDYDATA:invalid_message}' ] } grok { match => [ "request", '%{WORD:verb} /%{GREEDYDATA:api}\?%{GREEDYDATA:query_string}', "request", '%{WORD:verb} /%{GREEDYDATA:api}' ] } mutate { gsub => [ "query_string", "timestamp", "et" ] } kv { source => "query_string" field_split => "&?" } if [mcc1] { ruby { code => " mnnc = [] mcc1 = event['mcc1'].gsub(/(^[0]*)/, '') mnc1 = event['mnc1'].gsub(/(^[0]*)/, '') mcc2 = event['mcc2'].gsub(/(^[0]*)/, '') mnc2 = event['mnc2'].gsub(/(^[0]*)/, '') if mcc1 == '' and mnc1 == '' mnnc << '0' else mnnc << mcc1+mnc1 end if mcc2 == '' and mnc2 == '' mnnc << '0' else mnnc << mcc2+mnc2 end event['mnnc'] = mnnc " } } mutate { lowercase => ['uid', 'keyword', 'code', 'location'] } mutate { remove_field => [ 'http_post_data', 'blah', 'query_string', 'http_referer', 'http_user_agent', 'ref', 'body_bytes_sent', 'http_forward', 'path', 'clientip', 'host', 'status', 'message', 'mcc', 'mnc', 'verb', 'lt', 'url' ] } } } output { if [type] == 'tracker' { elasticsearch { user => "uname" password => "pass" hosts => ["127.0.0.1:9200"] index => "my_index" document_id => "%{uid}" action => "create" } stdout { codec => rubydebug } } } ``` And this is an example log file - ``` 202.87.33.164 - - [01/Jun/2017:15:14:20 +0000] "GET /info?uid=d1&keyword=x1&code=y1&location=z1&mcc1=0&mnc1=0&mcc2=4&mnc2=4×tamp=1496330139314 HTTP/1.1" 204 118 "-" "Java/1.7.0_05" * "-" 202.87.33.164 - - [02/Jun/2017:05:22:00 +0000] "GET /info?uid=d2&keyword=x2&code=y2&location=z2&mcc1=0&mnc1=0&mcc2=4&mnc2=4×tamp=1496381000548 HTTP/1.1" 204 118 "-" "Java/1.7.0_05" * "-" 1.23.165.186 - - [02/Jun/2017:05:23:45 +0000] "GET /index?uid=d3&keyword=x3&code=y3&location=z3&date=02-06-2017&time=10%3A55%3A02 HTTP/1.1" 200 249 "-" "Dalvik/2.1.0 (Linux; U; Android 7.0; K9 Kavach 4G Build/NRD90M)" * "-" ``` The `logstash.log`, `logstash.err` and `logstash.stdout` files remain empty when I run logstash as a daemon. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 7, 2017, 5:59am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-es/88020/4 "2017-06-07T05:59:17Z")
How do you know Logstash is even reading from the file? It's probably tailing it, waiting for more lines to be added to it. Also, does the Logstash user have permissions to read the input file (and all directories leading up to it)? ---
**Author:** ![keshav](https://avatars.discourse-cdn.com/v4/letter/k/b5e925/32.png) [@keshav](https://discuss.elastic.co/u/keshav)\ **Post date:** [June 7, 2017, 6:57am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-es/88020/5 "2017-06-07T06:57:59Z")
I've set the start\_position to beginning to ensure that logstash doesn't tail the file. The system is live, therefore new lines are added to the log file automatically, and the logstash parses `/index` endpoint and send data to Elasticsearch but not the `/info` endpoint. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 6:58am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-es/88020/6 "2017-07-05T06:58:06Z")
Closed on July 5, 2017, 6:58am UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.