# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=933 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 934 --- ## [Problem accessing trust store](https://discuss.elastic.co/t/problem-accessing-trust-store/93056)
**Author:** [@Ihjaz](https://discuss.elastic.co/u/Ihjaz)\ **Replies:** 2\ **Last updated:** [August 4, 2017, 9:59am UTC](https://discuss.elastic.co/t/problem-accessing-trust-store/93056 "2017-08-04T09:59:14Z")
Hi All, I was trying out the Logstash by following the steps at the below link. https://www.elastic.co/guide/en/beats/libbeat/5.5/logstash-installation.html I'm using the same configuration mentioned there as shown … --- ## [Duplicated indices](https://discuss.elastic.co/t/duplicated-indices/95486)
**Author:** [@DFrant](https://discuss.elastic.co/u/DFrant)\ **Replies:** 6\ **Last updated:** [August 4, 2017, 8:58am UTC](https://discuss.elastic.co/t/duplicated-indices/95486 "2017-08-04T08:58:24Z")
Hello, When I do a GET \_cat/indices, I see that there is an index created twice for my data log: yellow open .kibana RE2BXwgDSnuWmXlFvEn9BA 1 1 4 0 35.3kb 35.3kb yellow open carxx-2017.07.02 yAMp8… --- ## [Logstash Capacity planning](https://discuss.elastic.co/t/logstash-capacity-planning/95807)
**Author:** [@sesaravanan](https://discuss.elastic.co/u/sesaravanan)\ **Replies:** 3\ **Last updated:** [August 4, 2017, 7:32am UTC](https://discuss.elastic.co/t/logstash-capacity-planning/95807 "2017-08-04T07:32:03Z")
I have an requirement to pull and process 1 TB of data per data using logstash and push it to elastic search. how to calculate the network bandwidth for input data to logstash. how to calculate the network bandwidt… --- ## [Mongodb input plugin for logstash missing data problem](https://discuss.elastic.co/t/mongodb-input-plugin-for-logstash-missing-data-problem/95272)
**Author:** [@TaurusD](https://discuss.elastic.co/u/TaurusD)\ **Replies:** 4\ **Last updated:** [August 4, 2017, 6:20am UTC](https://discuss.elastic.co/t/mongodb-input-plugin-for-logstash-missing-data-problem/95272 "2017-08-04T06:20:21Z")
Hi, I’m using logstash-input-mongodb to import my data from mongodb to ELK. But when it was done, I found that there is one document missing (I have 17781 docs in mongodb but only got 17780 docs in ELK). I used the “no… --- ## [Iterating through all fields in a JSON message, checking for null values](https://discuss.elastic.co/t/iterating-through-all-fields-in-a-json-message-checking-for-null-values/95776)
**Author:** [@truesecure](https://discuss.elastic.co/u/truesecure)\ **Replies:** 1\ **Last updated:** [August 4, 2017, 6:06am UTC](https://discuss.elastic.co/t/iterating-through-all-fields-in-a-json-message-checking-for-null-values/95776 "2017-08-04T06:06:53Z")
Good afternoon, I am using Logstash to read in JSON documents from a Kafka topic and indexing them into Elasticsearch. Many of the fields in the document contain “null” for value. Since I don’t know which fields in whic… --- ## [How can i pull data from hive to logstash](https://discuss.elastic.co/t/how-can-i-pull-data-from-hive-to-logstash/95703)
**Author:** [@sesaravanan](https://discuss.elastic.co/u/sesaravanan)\ **Replies:** 5\ **Last updated:** [August 4, 2017, 5:32am UTC](https://discuss.elastic.co/t/how-can-i-pull-data-from-hive-to-logstash/95703 "2017-08-04T05:32:04Z")
Hi, I’m very new to Logstash. I have access to read data from database, so I decided to pull data from a remote database to logstash. The data size will be minimum 1TB per day. How can i do capacity planning in terms o… --- ## [Want to create multiple index for multiple input](https://discuss.elastic.co/t/want-to-create-multiple-index-for-multiple-input/95538)
**Author:** [@vivekpandey564](https://discuss.elastic.co/u/vivekpandey564)\ **Replies:** 18\ **Last updated:** [August 4, 2017, 5:06am UTC](https://discuss.elastic.co/t/want-to-create-multiple-index-for-multiple-input/95538 "2017-08-04T05:06:01Z")
Hi I have multiple log file created based on user. i want to take input all user log file and create index for that. For Ex Input section nput { file { add\_field =\> \[ "host", "my-dev-host" \] path =\> "D:\\JHips… --- ## [Logstash.config not working](https://discuss.elastic.co/t/logstash-config-not-working/95696)
**Author:** [@abhishek\_chaturvedi](https://discuss.elastic.co/u/abhishek_chaturvedi)\ **Replies:** 6\ **Last updated:** [August 4, 2017, 5:30am UTC](https://discuss.elastic.co/t/logstash-config-not-working/95696 "2017-08-04T05:30:39Z")
My logstash.conf file and mysql-connector-java-5.1.38.jar both are in F:\\Software\\logstash-5.5.1\\logstash-5.5.1\\bin location Getting below error while running conf file in cmd : F:\\Software\\logstash-5.5.1\\logstash-5.5.… --- ## [Logstash URL Login giving error](https://discuss.elastic.co/t/logstash-url-login-giving-error/95783)
**Author:** [@sandeeppattem](https://discuss.elastic.co/u/sandeeppattem)\ **Replies:** 1\ **Last updated:** [August 4, 2017, 3:50am UTC](https://discuss.elastic.co/t/logstash-url-login-giving-error/95783 "2017-08-04T03:50:19Z")
Hello, I am trying to pull in data from Dynatrace. But getting the XML data required Login. I have the config file : input { http\_poller { urls =\> { test1 =\> { url =\> "https://xxx.dynatrace.com:8021/rest/managemen… --- ## [What kind of size is better to set logstash pipline?](https://discuss.elastic.co/t/what-kind-of-size-is-better-to-set-logstash-pipline/95112)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 3\ **Last updated:** [August 4, 2017, 2:28am UTC](https://discuss.elastic.co/t/what-kind-of-size-is-better-to-set-logstash-pipline/95112 "2017-08-04T02:28:26Z")
I have 3 logstash servers with 16GB memory and 4vcpu enabled in a cluster environment. so my question is that what kind of size is better to set logstash pipline ? My configure file like this as below: OS: mem:16G c… --- ## [Elasticseach input plugin - docinfo field is not respected?](https://discuss.elastic.co/t/elasticseach-input-plugin-docinfo-field-is-not-respected/95757)
**Author:** [@wmathews](https://discuss.elastic.co/u/wmathews)\ **Replies:** 2\ **Last updated:** [August 3, 2017, 11:13pm UTC](https://discuss.elastic.co/t/elasticseach-input-plugin-docinfo-field-is-not-respected/95757 "2017-08-03T23:13:55Z")
I’m trying to do some work with the elastic input plugin, and part of it is that I want to access the index for use in the eventual output. In the documentation there is the docinfo field, but (unless I don’t understand … --- ## [Logstash.inputs.file failed to open](https://discuss.elastic.co/t/logstash-inputs-file-failed-to-open/95749)
**Author:** [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\ **Replies:** 3\ **Last updated:** [August 3, 2017, 10:04pm UTC](https://discuss.elastic.co/t/logstash-inputs-file-failed-to-open/95749 "2017-08-03T22:04:52Z")
Hi, I am facing an issue where logstash is not able to read my files. I tried granted all permission to the files ( 777 ) and changing the owner from root to logstash and even to my local user which i use to login. I am… --- ## [Enriching Logs with Internal Geo Ip Location Lookups](https://discuss.elastic.co/t/enriching-logs-with-internal-geo-ip-location-lookups/95746)
**Author:** [@megadevx](https://discuss.elastic.co/u/megadevx)\ **Replies:** 3\ **Last updated:** [August 3, 2017, 9:49pm UTC](https://discuss.elastic.co/t/enriching-logs-with-internal-geo-ip-location-lookups/95746 "2017-08-03T21:49:30Z")
I have been trying to find a way to do internal Geo IP lookup to enrich logs. I have over 500 subnets with specific locations for each that I want to map the whole subnet to that location. Is there any way that this woul… --- ## [Adding tags to logs](https://discuss.elastic.co/t/adding-tags-to-logs/94545)
**Author:** [@ashvita](https://discuss.elastic.co/u/ashvita)\ **Replies:** 7\ **Last updated:** [August 3, 2017, 8:30pm UTC](https://discuss.elastic.co/t/adding-tags-to-logs/94545 "2017-08-03T20:30:55Z")
Hi, I want to add tags to my incoming logs. Below is the snippet of the log- 10.100.213.171 Jul 25 22:02:42 x86-hw user.info Cmog: 4884:UpgradeActor D {\[4 04:01:52\] DbUtil.cpp:805}\[-4052732736\]::\[StartIcsClient\]::dom… --- ## [Rabbitmq as logstash input](https://discuss.elastic.co/t/rabbitmq-as-logstash-input/95756)
**Author:** [@Daniel\_Dziubecki](https://discuss.elastic.co/u/Daniel_Dziubecki)\ **Replies:** 4\ **Last updated:** [August 3, 2017, 7:00pm UTC](https://discuss.elastic.co/t/rabbitmq-as-logstash-input/95756 "2017-08-03T19:00:23Z")
Im trying to read logs from rabbitmq queue from logstash and then pass it to elasticsearch. But with no success. Here is my logstash config. input { rabbitmq { host =\> "localhost" port =\> 15672 … --- ## [Decompose json array string in Logstash](https://discuss.elastic.co/t/decompose-json-array-string-in-logstash/95763)
**Author:** [@bitesize](https://discuss.elastic.co/u/bitesize)\ **Replies:** 1\ **Last updated:** [August 3, 2017, 6:58pm UTC](https://discuss.elastic.co/t/decompose-json-array-string-in-logstash/95763 "2017-08-03T18:58:58Z")
I have a RabbitMQ sending 3 fields in JSON format and it is being consumed by Logstash rabbitmq input plugin. One of the fields is a JSON array of strings as follows: "content": \[ "1111", "2222222", "Test 06", … --- ## [Only default time filter field name appears](https://discuss.elastic.co/t/only-default-time-filter-field-name-appears/94200)
**Author:** [@ICG](https://discuss.elastic.co/u/ICG)\ **Replies:** 1\ **Last updated:** [August 3, 2017, 6:43pm UTC](https://discuss.elastic.co/t/only-default-time-filter-field-name-appears/94200 "2017-08-03T18:43:06Z")
I run this Logstash code in Elasticstack 5.0.0 and when I open Kibana, I can see two Time Filter field names: @timestam and CapturedOn. However if I run the same code in Elasticstack 5.5.0, then I see only one Time Filte… --- ## [Logstash doesn't read an input file :/](https://discuss.elastic.co/t/logstash-doesnt-read-an-input-file/95231)
**Author:** [@abed85](https://discuss.elastic.co/u/abed85)\ **Replies:** 6\ **Last updated:** [August 3, 2017, 6:30pm UTC](https://discuss.elastic.co/t/logstash-doesnt-read-an-input-file/95231 "2017-08-03T18:30:50Z")
Hello , I am trying to use Docker ELK stack to visualize the log of openvswitch. My problem with logstash that it doesn’t read the log file. in the /logstash/pipeline/logstash.conf I added input { file { path =\> … --- ## [Not recognised pattern](https://discuss.elastic.co/t/not-recognised-pattern/95737)
**Author:** [@DFrant](https://discuss.elastic.co/u/DFrant)\ **Replies:** 1\ **Last updated:** [August 3, 2017, 5:56pm UTC](https://discuss.elastic.co/t/not-recognised-pattern/95737 "2017-08-03T17:56:41Z")
Hello, I have a log file, that I have given a \[fields\]\[type\] == carxx in my filebeat configuration to allow me to do some pattern matching with logstash. But, I have seen that some data (approximatively 1/3) are not pa… --- ## [Question on Fields created in Filebeat and matched in Logstash Input](https://discuss.elastic.co/t/question-on-fields-created-in-filebeat-and-matched-in-logstash-input/95733)
**Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 2\ **Last updated:** [August 3, 2017, 5:49pm UTC](https://discuss.elastic.co/t/question-on-fields-created-in-filebeat-and-matched-in-logstash-input/95733 "2017-08-03T17:49:52Z")
This may be a theoretical question, I am trying to configure my filebeat to send the logs to logstash. Here is the config: input\_type: log paths: /prod/logs/chassis/tomcat/app\*.log #- c:\\programdata\\elasticsearch\\lo… --- ## [Logstash (5.4.2) / Winlogbeat / SSL - File does not contain valid private key](https://discuss.elastic.co/t/logstash-5-4-2-winlogbeat-ssl-file-does-not-contain-valid-private-key/95183)
**Author:** [@jaypalexa](https://discuss.elastic.co/u/jaypalexa)\ **Replies:** 1\ **Last updated:** [August 3, 2017, 3:30pm UTC](https://discuss.elastic.co/t/logstash-5-4-2-winlogbeat-ssl-file-does-not-contain-valid-private-key/95183 "2017-08-03T15:30:17Z")
Hello -- We are attempting to implement SSL for Logstash (5.4.2) / Winlogbeat as per Securing Communication With Logstash by Using SSL. Just focusing on the server-side SSL for now; client-side / mutual authentication i… --- ## [Make extracted field visible to all events](https://discuss.elastic.co/t/make-extracted-field-visible-to-all-events/94845)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 1\ **Last updated:** [August 3, 2017, 2:53pm UTC](https://discuss.elastic.co/t/make-extracted-field-visible-to-all-events/94845 "2017-08-03T14:53:41Z")
Hello, I've the following filter configurations filter{ if "Dumping Scrapy stats" in \[message\] { grok{ match =\> \[ "message", "'item\_scraped\_count': %{NUMBER:scraped:int}" \] } grok{ … --- ## [Problem with Timezone](https://discuss.elastic.co/t/problem-with-timezone/95611)
**Author:** [@Carlos\_Barrera](https://discuss.elastic.co/u/Carlos_Barrera)\ **Replies:** 0\ **Last updated:** [August 2, 2017, 8:25pm UTC](https://discuss.elastic.co/t/problem-with-timezone/95611 "2017-08-02T20:25:17Z")
hi, I need help the problem I am using the logstash date filter, to assign the date from the log line, I need to use the timezone of my country Chile (America / Santiago) which is currently GMT-4, but when I review the… --- ## [Multiline two new lines bewteen messages](https://discuss.elastic.co/t/multiline-two-new-lines-bewteen-messages/92654)
**Author:** [@Christian\_Koller\_Nie](https://discuss.elastic.co/u/Christian_Koller_Nie)\ **Replies:** 1\ **Last updated:** [August 3, 2017, 2:05pm UTC](https://discuss.elastic.co/t/multiline-two-new-lines-bewteen-messages/92654 "2017-08-03T14:05:21Z")
I have a logfile on the form: aaa bbb aaa bbb aaa bbb and a .conf file: input { file { path =\> "..." type =\> "j4" start\_position =\> "beginning" codec =\> multiline { pattern =\> "^\[^\\r\]" what =\> "pr… --- ## [Logstash sincedb files](https://discuss.elastic.co/t/logstash-sincedb-files/95297)
**Author:** [@sreepatch](https://discuss.elastic.co/u/sreepatch)\ **Replies:** 6\ **Last updated:** [August 3, 2017, 12:09pm UTC](https://discuss.elastic.co/t/logstash-sincedb-files/95297 "2017-08-03T12:09:44Z")
Hi, I have a logstash running in unix host and since db files are getting created in so many numbers. Can you please clarify below questions 1.How often will these sincedb files be created 2.Will sincedb file be crea… --- ## [X-Pack Logstash Settings issue](https://discuss.elastic.co/t/x-pack-logstash-settings-issue/95641)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 2\ **Last updated:** [August 3, 2017, 12:03pm UTC](https://discuss.elastic.co/t/x-pack-logstash-settings-issue/95641 "2017-08-03T12:03:31Z")
I have installed the X-Pack on logstash for monitoring. I don’t see the logstash.yml file is being updated with additional xpack monitoring settings. Still I see the following settings when I run the logstash. Also the x… --- ## [Help in parsing XML data in logstash in different events](https://discuss.elastic.co/t/help-in-parsing-xml-data-in-logstash-in-different-events/95322)
**Author:** [@aashishchauhan06](https://discuss.elastic.co/u/aashishchauhan06)\ **Replies:** 1\ **Last updated:** [August 3, 2017, 11:56am UTC](https://discuss.elastic.co/t/help-in-parsing-xml-data-in-logstash-in-different-events/95322 "2017-08-03T11:56:10Z")
Hello, I need to parse a xml file data in logstash. I had been able to parse data successfully in same event but i want data in different event. My XML file looks like: \< drugbank\> \< drug type=“biotech”\> \< name\>Lepi… --- ## [\[SOLVED\] Usage of Logstash Ruby filter to calculate percentage](https://discuss.elastic.co/t/solved-usage-of-logstash-ruby-filter-to-calculate-percentage/95648)
**Author:** [@NandhaKrishna](https://discuss.elastic.co/u/NandhaKrishna)\ **Replies:** 2\ **Last updated:** [August 3, 2017, 11:33am UTC](https://discuss.elastic.co/t/solved-usage-of-logstash-ruby-filter-to-calculate-percentage/95648 "2017-08-03T11:33:59Z")
I want to know how to calculate percentage using the logstash ruby filter. I have tried this code and it did not work. code =\> “event.set(’\[ratio\_match\]’, (event.get(‘A’)/event.get(‘B’)\*100))” Kindly help me with wher… --- ## [Error on offline plugin install, verification failed](https://discuss.elastic.co/t/error-on-offline-plugin-install-verification-failed/95631)
**Author:** [@TaurusD](https://discuss.elastic.co/u/TaurusD)\ **Replies:** 4\ **Last updated:** [August 3, 2017, 10:04am UTC](https://discuss.elastic.co/t/error-on-offline-plugin-install-verification-failed/95631 "2017-08-03T10:04:21Z")
Hi, I'm using logstash 5.5.1 and trying to install some plugin offline on WINDOWS. I have already have my plugins ready on my PC, and I used bin\\logstash-plugin prepare-offline-packages logstash-input-\* logstash-filte… --- ## [Definitions of Logstash Collector, Processor and Shipper](https://discuss.elastic.co/t/definitions-of-logstash-collector-processor-and-shipper/95429)
**Author:** [@sebastus](https://discuss.elastic.co/u/sebastus)\ **Replies:** 2\ **Last updated:** [August 3, 2017, 9:40am UTC](https://discuss.elastic.co/t/definitions-of-logstash-collector-processor-and-shipper/95429 "2017-08-03T09:40:51Z")
I have seen reference to collectors, processors and shippers in various places, but I can’t find a definition of either one. Have they always existed? Did they exist for a while as important artifacts and no longer are s… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=932) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=934)