# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=294 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 295 --- ## [Logstash log10 function in ruby](https://discuss.elastic.co/t/logstash-log10-function-in-ruby/251668)
**Author:** [@Ameeruddin\_Mohammed](https://discuss.elastic.co/u/Ameeruddin_Mohammed)\ **Replies:** 2\ **Last updated:** [October 11, 2020, 8:49pm UTC](https://discuss.elastic.co/t/logstash-log10-function-in-ruby/251668 "2020-10-11T20:49:27Z")
hi, I have to use log10 function, can anyone support. tried both options but didnt work event.set("AvgRssiHigh", 10\*log10((s.sum/b.sum) / 1000000000000)) event.set("AvgRssiHigh", 10\*((s.sum/b.sum) / 1000000000000).lo… --- ## [Schedule Understanding - Logstash](https://discuss.elastic.co/t/schedule-understanding-logstash/251666)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 0\ **Last updated:** [October 11, 2020, 6:22am UTC](https://discuss.elastic.co/t/schedule-understanding-logstash/251666 "2020-10-11T06:22:28Z")
Hello I understand , now in input plugin of logstash say for example jdbc one , has schedule option, so that we can schedule it in here itself. I did read documents but did not came across the explanation of condition … --- ## [Logstash shut down immediately after starting](https://discuss.elastic.co/t/logstash-shut-down-immediately-after-starting/251642)
**Author:** [@emaravi](https://discuss.elastic.co/u/emaravi)\ **Replies:** 1\ **Last updated:** [October 10, 2020, 1:48pm UTC](https://discuss.elastic.co/t/logstash-shut-down-immediately-after-starting/251642 "2020-10-10T13:48:17Z")
\[2020-10-10T05:30:55,854\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or command line options are specified \[2020-10-10T05:30:57,935\]\[INFO \]\[org.reflections.Reflections\] R… --- ## [DIfference between grok and dissect?](https://discuss.elastic.co/t/difference-between-grok-and-dissect/251634)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 0\ **Last updated:** [October 10, 2020, 9:15am UTC](https://discuss.elastic.co/t/difference-between-grok-and-dissect/251634 "2020-10-10T09:15:30Z")
i've .net core application with use of nlog i created logs. for parsing logs to logstash which is more useful and proper filter grok filter or dissect filter --- ## [Grok can't create optional fields](https://discuss.elastic.co/t/grok-cant-create-optional-fields/251631)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 0\ **Last updated:** [October 10, 2020, 7:20am UTC](https://discuss.elastic.co/t/grok-cant-create-optional-fields/251631 "2020-10-10T07:20:41Z")
I'm trying to parse logs in logstash with use of grok. In error log there are two more optional fields like requestUrl and requestMethod. but when i use grok for parsing logs to logstash.these from the error log grok can… --- ## [Duplicated lines when parsing rabbitmq logs§](https://discuss.elastic.co/t/duplicated-lines-when-parsing-rabbitmq-logs/251613)
**Author:** [@Hamish1](https://discuss.elastic.co/u/Hamish1)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 9:37pm UTC](https://discuss.elastic.co/t/duplicated-lines-when-parsing-rabbitmq-logs/251613 "2020-10-09T21:37:52Z")
Good day everyone! We're using RabbitMQ version 3.6.10 and it has a weird log format: closing AMQP connection \<0.28817.3524\> (HIDEN:50790 -\> HIDEN:5672, vhost: '/', user: 'HIDEN') =INFO REPORT==== 9-Oct-2020::22:41:24… --- ## [Logstash :: What Would a NULL Returned from an External Program Mean?](https://discuss.elastic.co/t/logstash-what-would-a-null-returned-from-an-external-program-mean/251170)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 1\ **Last updated:** [October 9, 2020, 8:58pm UTC](https://discuss.elastic.co/t/logstash-what-would-a-null-returned-from-an-external-program-mean/251170 "2020-10-09T20:58:31Z")
Hi Logstash Gurus, I am using my Logstash filter{} configuration to use Ruby to “bounce” all my LS records off an external C program via a TCP socket. The C program accepts every record, does some processing, and then … --- ## [Logstash is not accessable outside of the docker container](https://discuss.elastic.co/t/logstash-is-not-accessable-outside-of-the-docker-container/251605)
**Author:** [@Srikanth\_S1](https://discuss.elastic.co/u/Srikanth_S1)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 6:18pm UTC](https://discuss.elastic.co/t/logstash-is-not-accessable-outside-of-the-docker-container/251605 "2020-10-09T18:18:58Z")
Hi,, I build the docker image for the logstash using the "apt-get install logstash". I can able to run the image successfully, but i am unble to access the localhost:9600/?pretty outside of the container https://ar… --- ## [Mongo Output document deletion](https://discuss.elastic.co/t/mongo-output-document-deletion/249693)
**Author:** [@arichiardi](https://discuss.elastic.co/u/arichiardi)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 6:08pm UTC](https://discuss.elastic.co/t/mongo-output-document-deletion/249693 "2020-10-09T18:08:42Z")
Hi there, I am learning how to use the MongoDB output pluing and trying to understand how to delete a document in a collection. I was wondering if this is supported. The alternative I can think of is to have a tombsto… --- ## [Cannot read Cloudtrail logs using s3-sns-sqs plugin. Uncompleted message at the end of poller loop](https://discuss.elastic.co/t/cannot-read-cloudtrail-logs-using-s3-sns-sqs-plugin-uncompleted-message-at-the-end-of-poller-loop/251603)
**Author:** [@wpitt13](https://discuss.elastic.co/u/wpitt13)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 5:11pm UTC](https://discuss.elastic.co/t/cannot-read-cloudtrail-logs-using-s3-sns-sqs-plugin-uncompleted-message-at-the-end-of-poller-loop/251603 "2020-10-09T17:11:58Z")
When I try to ingest logs using the s3snssqs input, I see no events output but no errors either. If I enable DEBUG logs, I see an "Inside Preprocess" message followed by a "Payload in Preprocess" message and then the fol… --- ## [Logstash consuming Kafka Topic but don't output document](https://discuss.elastic.co/t/logstash-consuming-kafka-topic-but-dont-output-document/251026)
**Author:** [@Coyz](https://discuss.elastic.co/u/Coyz)\ **Replies:** 3\ **Last updated:** [October 9, 2020, 2:32pm UTC](https://discuss.elastic.co/t/logstash-consuming-kafka-topic-but-dont-output-document/251026 "2020-10-09T14:32:53Z")
Hello, I'm trying to get the stack working with Filebeat =\> Kafka \<= Logstash =\> Elasticsearch Sometimes the configuration is working but sometimes logstash is consuming the kafka topic but not sending the document to… --- ## [Scientific Notation Auto Rounding](https://discuss.elastic.co/t/scientific-notation-auto-rounding/251496)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 4\ **Last updated:** [October 9, 2020, 2:04pm UTC](https://discuss.elastic.co/t/scientific-notation-auto-rounding/251496 "2020-10-09T14:04:21Z")
I am using ruby and BigDecmial to convert a scientific notation number to decimal, but it's automatically getting rouded to the nearest thousandth. How can I prevent this? I want the full number or at least n^-6 not n^-3… --- ## [Assistance with my Logstash filter](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342)
**Author:** [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Replies:** 9\ **Last updated:** [October 9, 2020, 1:21pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342 "2020-10-09T13:21:03Z")
Hello, I am trying to use a variation of the filter provided here - https://github.com/bromiley/olaf/blob/master/logstash/o365.config but I keep getting the following error, \[FATAL\]\[logstash.runner \] The given… --- ## [Stdout rubydebug not outputting to file](https://discuss.elastic.co/t/stdout-rubydebug-not-outputting-to-file/251296)
**Author:** [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\ **Replies:** 11\ **Last updated:** [October 9, 2020, 12:10pm UTC](https://discuss.elastic.co/t/stdout-rubydebug-not-outputting-to-file/251296 "2020-10-09T12:10:48Z")
Hello, I have got an nginx output with the stdout configured at the bottom as: output { if \[service\] == "nginx" and "\_grokparsefailure" in \[tags\] { elasticsearch { user =\> "user" password =\> "password" … --- ## [Unable to run Logstash 7.9.2 as a service on Ubuntu 18.04](https://discuss.elastic.co/t/unable-to-run-logstash-7-9-2-as-a-service-on-ubuntu-18-04/251005)
**Author:** [@Philip\_Colmer](https://discuss.elastic.co/u/Philip_Colmer)\ **Replies:** 15\ **Last updated:** [October 9, 2020, 7:06am UTC](https://discuss.elastic.co/t/unable-to-run-logstash-7-9-2-as-a-service-on-ubuntu-18-04/251005 "2020-10-09T07:06:44Z")
I'm getting a warning and an error when I try to start logstash with systemctl start logstash: \[2020-10-05T14:01:19,067\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or com… --- ## [Logstash Comparing Field Values Using an If Statement not working](https://discuss.elastic.co/t/logstash-comparing-field-values-using-an-if-statement-not-working/251452)
**Author:** [@bardie](https://discuss.elastic.co/u/bardie)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 6:12am UTC](https://discuss.elastic.co/t/logstash-comparing-field-values-using-an-if-statement-not-working/251452 "2020-10-09T06:12:48Z")
Hi, I am trying to compare two fields using the following method but it is not working. I have used it before using an older version of logstash and it worked but it is not working on logstash 7.9.2. Logstash cannot be… --- ## [Using gsub for multi line patterns](https://discuss.elastic.co/t/using-gsub-for-multi-line-patterns/251458)
**Author:** [@Surya\_Iriventi](https://discuss.elastic.co/u/Surya_Iriventi)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 5:26am UTC](https://discuss.elastic.co/t/using-gsub-for-multi-line-patterns/251458 "2020-10-09T05:26:38Z")
Hi, I am trying to use gsub to remove the prefix for the log line. I have a filebeat configuration which matches multiline conguration. The multi line log message (input to logstash) would look line this: \*\*\*\* Error … --- ## [How to enable proxy for google\_pubsub input plugin in logstash](https://discuss.elastic.co/t/how-to-enable-proxy-for-google-pubsub-input-plugin-in-logstash/251525)
**Author:** [@nareshahi](https://discuss.elastic.co/u/nareshahi)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 5:12am UTC](https://discuss.elastic.co/t/how-to-enable-proxy-for-google-pubsub-input-plugin-in-logstash/251525 "2020-10-09T05:12:17Z")
Hello Guys, i have enabled google\_pubsub plugin in logstash. it is up and runing. look like pipeline is looking for proxy to make API call with google cloud. please guide where have to set proxy setting for plugin . R… --- ## [String with spaces not is not searchable in logstash](https://discuss.elastic.co/t/string-with-spaces-not-is-not-searchable-in-logstash/251225)
**Author:** [@NARHAR\_DEV\_SHARMA](https://discuss.elastic.co/u/NARHAR_DEV_SHARMA)\ **Replies:** 4\ **Last updated:** [October 9, 2020, 4:51am UTC](https://discuss.elastic.co/t/string-with-spaces-not-is-not-searchable-in-logstash/251225 "2020-10-09T04:51:31Z")
Hi Team, Greeting for the day! I am trying to search a pattern that contain spaces and its not working. However, if i put something without space that's working absolutely fine; i am able to send mail to required addr… --- ## [Logstash count insta Filter aggregate](https://discuss.elastic.co/t/logstash-count-insta-filter-aggregate/251512)
**Author:** [@bbwolf](https://discuss.elastic.co/u/bbwolf)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 11:06pm UTC](https://discuss.elastic.co/t/logstash-count-insta-filter-aggregate/251512 "2020-10-08T23:06:49Z")
Hello, I would to know the best way to have count\_event as aggregate for task\_id equal to field1. I can create count as new event but notre like in the following format. Field1, field2, field3, field4, count\_event A,… --- ## [LOGSTASH: How to setup logstash to get all query's data not only a part of it?](https://discuss.elastic.co/t/logstash-how-to-setup-logstash-to-get-all-querys-data-not-only-a-part-of-it/251507)
**Author:** [@I\_Hathout](https://discuss.elastic.co/u/I_Hathout)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 10:29pm UTC](https://discuss.elastic.co/t/logstash-how-to-setup-logstash-to-get-all-querys-data-not-only-a-part-of-it/251507 "2020-10-08T22:29:03Z")
I've a database table which contains 400+ records, but each time, I use logstash and JDBC it only gets 126 records. I tried many times with different indices each time, but still the 126 only retrieving. Here is the out… --- ## [Logstash restarting by output ES](https://discuss.elastic.co/t/logstash-restarting-by-output-es/251509)
**Author:** [@dromval90](https://discuss.elastic.co/u/dromval90)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 10:35pm UTC](https://discuss.elastic.co/t/logstash-restarting-by-output-es/251509 "2020-10-08T22:35:05Z")
Hi good day community: I would like to ask if someone that you it has experiment this behavior: We have 4 servers with instances of logstash with version 7.9, into config file into output to Elasticsearch, we have 12 s… --- ## [Unable to read the txt file through Logstash](https://discuss.elastic.co/t/unable-to-read-the-txt-file-through-logstash/251057)
**Author:** [@sukumar.koganti](https://discuss.elastic.co/u/sukumar.koganti)\ **Replies:** 24\ **Last updated:** [October 8, 2020, 6:55pm UTC](https://discuss.elastic.co/t/unable-to-read-the-txt-file-through-logstash/251057 "2020-10-08T18:55:20Z")
Hi Guys, I am trying to read sample EDI log from the below path using logstash .conf file C:\\Users\\skkoganti\\Documents\\log\\EDI Log-2019-05-27.txt this is my sample input file format input { file { path =\> \["C:/Us… --- ## [Logstash pipeline.yml error](https://discuss.elastic.co/t/logstash-pipeline-yml-error/251441)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 8\ **Last updated:** [October 8, 2020, 6:08pm UTC](https://discuss.elastic.co/t/logstash-pipeline-yml-error/251441 "2020-10-08T18:08:46Z")
I've one logstash pipeline.which is given in pipeline.yml. but when i run bin/logstash.bat command in powershell i'm getting error which is look like this ERROR: Failed to read pipelines yaml file. Location: D:/Elast… --- ## [Any filter for parsing the URL request from firewalls?](https://discuss.elastic.co/t/any-filter-for-parsing-the-url-request-from-firewalls/251451)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 4:46pm UTC](https://discuss.elastic.co/t/any-filter-for-parsing-the-url-request-from-firewalls/251451 "2020-10-08T16:46:20Z")
regarding payload within firewall logs, the request field has a single line separated by \\r\\n which I've made it into individual line. Any known parsers/filter to make this into key-value pairs? (especially the User-Age… --- ## [Loading SQL table with JSON data and columnar data](https://discuss.elastic.co/t/loading-sql-table-with-json-data-and-columnar-data/251286)
**Author:** [@sanmitdesai](https://discuss.elastic.co/u/sanmitdesai)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 3:49pm UTC](https://discuss.elastic.co/t/loading-sql-table-with-json-data-and-columnar-data/251286 "2020-10-08T15:49:00Z")
Hey all, I am a bit new to logstash. I have a SQL table that looks like this. As we can see we have columns 'Address' and 'Skills' in this table that are in JSON format. I want to load all the data while maintainin… --- ## [How to parse mix json logs file](https://discuss.elastic.co/t/how-to-parse-mix-json-logs-file/251470)
**Author:** [@Purvesh\_Jaiswal](https://discuss.elastic.co/u/Purvesh_Jaiswal)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 3:09pm UTC](https://discuss.elastic.co/t/how-to-parse-mix-json-logs-file/251470 "2020-10-08T15:09:07Z")
''''''''' {"tool": {"driver": {"name": "Shell Script Analysis", "version": "1.0.0-scan", "fullName": "Shell Script Analysis"}}, "conversion": {"tool": {"driver": {"name": "@ShiftLeft/sast-scan"}}, "invocation": {"argume… --- ## [Logstash filtering. Extract data between two strings](https://discuss.elastic.co/t/logstash-filtering-extract-data-between-two-strings/251276)
**Author:** [@Igor\_Olikh](https://discuss.elastic.co/u/Igor_Olikh)\ **Replies:** 6\ **Last updated:** [October 8, 2020, 3:04pm UTC](https://discuss.elastic.co/t/logstash-filtering-extract-data-between-two-strings/251276 "2020-10-08T15:04:33Z")
I have a UNIX log looks like: ACTION started Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s finished when an un… --- ## [Logstash CIDR network range](https://discuss.elastic.co/t/logstash-cidr-network-range/251391)
**Author:** [@ParashB](https://discuss.elastic.co/u/ParashB)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 2:55pm UTC](https://discuss.elastic.co/t/logstash-cidr-network-range/251391 "2020-10-08T14:55:48Z")
Hello, How to match IP range which is not full network range? Example: 172.20.23.5 in \[172.16.0.0 -172.31.255.255 \] CIDR filter plugin specifies the full range but I need to search is specific range. --- ## [Logstash dont start if output fail](https://discuss.elastic.co/t/logstash-dont-start-if-output-fail/251467)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 2:53pm UTC](https://discuss.elastic.co/t/logstash-dont-start-if-output-fail/251467 "2020-10-08T14:53:50Z")
Using RabbitMQ as output, Logstash dont start if RabbitMQ is down: \[2020-10-08T14:45:32,102\]\[ERROR\]\[logstash.outputs.rabbitmq\]\[main\] RabbitMQ connection error, will retry. {:error\_message=\>"Connection was refused. Targe… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=293) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=295)