# I could not start logstash properly via rpm installed **URL:** \ **Category:** Logstash\ **Created:** [November 6, 2016, 4:37am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120 "2016-11-06T04:37:29Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 6, 2016, 4:37am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/1 "2016-11-06T04:37:29Z")
Hi, all I could not start logstash properly via rpm installed. Any helps here? ## Description `sudo systemctl start logstash.service` the command above would generate logs at /var/log/logstash/logstash-plain.log ```auto [2016-11-06T11:35:57,640][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>["http://192.168.1.57:9200"]}} [2016-11-06T11:35:57,643][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil} [2016-11-06T11:35:57,963][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}} [2016-11-06T11:35:57,968][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["192.168.1.57:9200"]} [2016-11-06T11:35:58,018][INFO][logstash.filters.geoip] Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.0.3-java/vendor/GeoLite2-City.mmdb"} [2016-11-06T11:35:58,032][INFO][logstash.pipeline] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000} [2016-11-06T11:35:58,035][INFO][logstash.pipeline] Pipeline main started [2016-11-06T11:35:58,084][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9601} ``` It seems Logstash is working normally. But actually, the ES did not create any indices. By the way, it could work well when I start Logstash via tar package like this `sudo /bin/logstash -f /etc/logstash/conf.d/test-pipeline.conf` Is that something important I have ignored? ## My configuration file `test-piple.conf` ```auto input { file { path => "/home/zh/Documents/*" start_position => beginning ignore_older => 0 } } filter { grok { match => {"message" => "%{COMBINEDAPACHELOG}"} } geoip { source => "clientip" } } output { elasticsearch { hosts => ["192.168.1.57:9200"] } stdout { codec => rubydebug } } ``` ## My environment Centos 7 ELK version 5.0 Any helps would appreciated! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 6, 2016, 4:40am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/2 "2016-11-06T04:40:36Z")
> [@Generalibm\_Jang](#): > > [2016-11-06T11:35:58,084][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9601} That means it **has** started. Have a look at [File input plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_tracking_of_current_position_in_watched_files) ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 6, 2016, 4:45am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/3 "2016-11-06T04:45:17Z")
Could you be more concrete? Is that my configuration file wrong? ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 7, 2016, 6:52am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/4 "2016-11-07T06:52:00Z")
I have not fixed the problem yet. Could you give me some more clues? ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 7, 2016, 7:41am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/5 "2016-11-07T07:41:37Z")
I have installed Logstash(5.0) in my system(Centos 7), But when I start it by command `sudo systemctl start logstash.service` The ES did not create any indices. The Json is ```auto {"host":"arena","version":"5.0.0","http_address":"127.0.0.1:9600","events":{"in":null,"filtered":0,"out":0,"duration_in_millis":null},"jvm":{"timestamp":1478503253667,"uptime_in_millis":210914,"memory":{"heap_used_in_bytes":358553984,"heap_used_percent":17,"heap_committed_in_bytes":519045120,"heap_max_in_bytes":2075918336,"non_heap_used_in_bytes":182509520,"non_heap_committed_in_bytes":191586304,"pools":{"survivor":{"peak_used_in_bytes":8912896,"used_in_bytes":11639576,"peak_max_in_bytes":35782656,"max_in_bytes":71565312,"committed_in_bytes":17825792},"old":{"peak_used_in_bytes":142783408,"used_in_bytes":272392512,"peak_max_in_bytes":715849728,"max_in_bytes":1431699456,"committed_in_bytes":357957632},"young":{"peak_used_in_bytes":71630848,"used_in_bytes":74521896,"peak_max_in_bytes":286326784,"max_in_bytes":572653568,"committed_in_bytes":143261696}}}}} ``` When I type command `GET http://localhost:9600/_stats` By the contrary, When I started Logstash via command `sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash` The ES created index logstash-\* and the Json is ```auto {"host":"arena","version":"5.0.0","http_address":"127.0.0.1:9601","events":{"in":39,"filtered":39,"out":39,"duration_in_millis":34529},"jvm":{"timestamp":1478501173112,"uptime_in_millis":2308559,"memory":{"heap_used_in_bytes":406420224,"heap_used_percent":19,"heap_committed_in_bytes":519045120,"heap_max_in_bytes":2075918336,"non_heap_used_in_bytes":192557088,"non_heap_committed_in_bytes":203669504,"pools":{"survivor":{"peak_used_in_bytes":8912896,"used_in_bytes":10214760,"peak_max_in_bytes":35782656,"max_in_bytes":71565312,"committed_in_bytes":17825792},"old":{"peak_used_in_bytes":142143056,"used_in_bytes":273810880,"peak_max_in_bytes":715849728,"max_in_bytes":1431699456,"committed_in_bytes":357957632},"young":{"peak_used_in_bytes":71630848,"used_in_bytes":122394584,"peak_max_in_bytes":286326784,"max_in_bytes":572653568,"committed_in_bytes":143261696}}}}} ``` When I type command `GET http://localhost:9601/_stats` It is obvious that events (input {file{}}) is the culprit. But how could I fix it??? I can not find any mistakes in the start script file `start.options`, Any helps here??? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 7, 2016, 7:57am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/7 "2016-11-07T07:57:58Z")
Perhaps Logstash is tailing the input file and waiting for more input? `start_position => beginning` only matters for new files. If you want to process a previously processed file from the beginning you have to remove its sincedb file. See the file input's documentation. ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 7, 2016, 8:07am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/8 "2016-11-07T08:07:19Z")
Yes, I have fixed my attention on this. But what a strange is that the Logstash did not update auto when both a new file created and a previous file added several lines. ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 7, 2016, 8:32am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/9 "2016-11-07T08:32:31Z")
I have removed all the .sincedb\_\* file , and restart the Logstash, But the status is still ```auto {"host":"arena","version":"5.0.0","http_address":"127.0.0.1:9600","events":{"in":null,"filtered":0,"out":0,"duration_in_millis":null},"jvm":{"timestamp":1478507126290,"uptime_in_millis":208555,"memory": ``` 😭 ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 7, 2016, 8:57am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/10 "2016-11-07T08:57:51Z")
Never mind the status. Look in the logs for clues and focus on the stuff related to the file input. You may have to increase the log level. ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 7, 2016, 9:48am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/11 "2016-11-07T09:48:34Z")
Yes, I switched log level to `debug` and restart Logstash again, I got more log infos like ```auto [2016-11-07T17:34:28,984][DEBUG][logstash.inputs.file] _globbed_files: /home/zh/Documents/*: glob is: [] [2016-11-07T17:34:29,584][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:29 +0800} [2016-11-07T17:34:30,457][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2016-11-07T17:34:30,585][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:30 +0800} [2016-11-07T17:34:31,586][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:31 +0800} [2016-11-07T17:34:32,588][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:32 +0800} [2016-11-07T17:34:33,592][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:33 +0800} [2016-11-07T17:34:34,594][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:34 +0800} [2016-11-07T17:34:35,457][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2016-11-07T17:34:35,595][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:35 +0800} [2016-11-07T17:34:36,597][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:36 +0800} [2016-11-07T17:34:37,598][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:37 +0800} [2016-11-07T17:34:38,600][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:38 +0800} [2016-11-07T17:34:39,601][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:39 +0800} [2016-11-07T17:34:40,456][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2016-11-07T17:34:40,602][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:40 +0800} [2016-11-07T17:34:41,604][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:41 +0800} [2016-11-07T17:34:42,605][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:42 +0800} [2016-11-07T17:34:43,606][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-07 17:34:43 +0800} ``` But they can not give me more key clues. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 7, 2016, 11:10am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/12 "2016-11-07T11:10:25Z")
> ``` > [2016-11-07T17:34:28,984][DEBUG][logstash.inputs.file] _globbed_files: /home/zh/Documents/*: glob is: [] > > ``` There it is. The configured filename pattern doesn't expand to any files, possibly because the Logstash user doesn't have permissions to read one of the directories. ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 8, 2016, 1:38am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/13 "2016-11-08T01:38:21Z")
Lack of permissions??? When I type command `ls -l /home/zh/Documents` , the output is ```auto drwxr-xr-x. 2 zh zh 99 Nov 8 09:33 ./ drwx------. 32 zh zh 4096 Nov 7 17:58 ../ -rw-r--r--. 1 zh zh 12 Nov 7 16:02 ELK2.log -rw-r--r--. 1 zh zh 1026 Nov 7 16:28 ELK.log -rw-r--r--. 1 zh zh 405 Oct 27 16:26 ELK.txt -rw-rw-r--. 1 zh zh 54 Nov 7 18:01 test.log ``` It seems that the permission is not the culprit. ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 8, 2016, 1:47am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/14 "2016-11-08T01:47:49Z")
When I type command `cat /var/lib/logstash/plugins/inputs/file/.sincedb_ **** ` there is nothing, the .sincedb\_\*\*\* is empty. what a strange!!! May that be the culprit??? ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 8, 2016, 1:58am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/15 "2016-11-08T01:58:01Z")
After I restart Logstash, the file `sincedb_*` becomes `0 0 0 0` It seems that no input file in directory(`/home/zh/Documents`) has been detected(not because the permission) , and it may be the why that the status of Logstash is `"events":{"in":null,"filtered":0,"out":0,"duration_in_millis":null}` But why??? ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 8, 2016, 3:21am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/16 "2016-11-08T03:21:14Z")
May it be the permission cause the key. Because I modified my configuration file `test-pipeline.conf` with ```auto input { file { path => /var/log/logstash/*.log } } ``` and then, the status of Logstash switch to `"events":{"in":9871,"filtered":9746,"out":8871,"duration_in_millis":76491}` as a result , ES created the index `logstash-*` by correspondingly. By the way, I remodified the configuration file`test-pipeline.conf` with ```auto input { file { path => /var/log/elasticsearch/*.log } } ``` ES collect index well too. But, the permissions of the directory (/var/log/eleasticsearch) for the other users is the same ```auto -rw-r--r--. ``` by comparison with directory `/home/zh/Docments`. The result above makes me confused!!! ---
**Author:** ![Generalibm\_Jang](https://avatars.discourse-cdn.com/v4/letter/g/0ea827/32.png) [@Generalibm\_Jang](https://discuss.elastic.co/u/Generalibm_Jang)\ **Post date:** [November 8, 2016, 6:10am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/17 "2016-11-08T06:10:08Z")
Thank a lot, I've fixed it, That is because the permissions of directory(`home/zh`) is ```auto -rwx------ ``` so the Logstash user can not work in . ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 6, 2016, 6:10am UTC](https://discuss.elastic.co/t/i-could-not-start-logstash-properly-via-rpm-installed/65120/18 "2016-12-06T06:10:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.