# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=121 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 122 --- ## [Deleted Docs - SQL SYNC](https://discuss.elastic.co/t/deleted-docs-sql-sync/312153)
**Author:** [@Berat\_Demirci](https://discuss.elastic.co/u/Berat_Demirci)\ **Replies:** 0\ **Last updated:** [August 16, 2022, 8:57am UTC](https://discuss.elastic.co/t/deleted-docs-sql-sync/312153 "2022-08-16T08:57:41Z")
I will sync data from sql to elastic. My config was running a few tries but now It is deleting my docs. Sync is successful and I dont see any error. My config input { jdbc { jdbc\_driver\_library =\> "C:\\Program Files\\… --- ## [Failed to execute action:message=\>"Unable to configure plugins](https://discuss.elastic.co/t/failed-to-execute-action-message-unable-to-configure-plugins/312106)
**Author:** [@Eina\_Zia](https://discuss.elastic.co/u/Eina_Zia)\ **Replies:** 2\ **Last updated:** [August 16, 2022, 3:35am UTC](https://discuss.elastic.co/t/failed-to-execute-action-message-unable-to-configure-plugins/312106 "2022-08-16T03:35:59Z")
Hello everyone1 I am new to logstash and I have been trying for so long to import dataset through logstash. I keep on getting error. This is my recent attempt. Without super user, it is not giving me permission even afte… --- ## [Using app.kubernetes.io/name label](https://discuss.elastic.co/t/using-app-kubernetes-io-name-label/312088)
**Author:** [@AssafKatz3](https://discuss.elastic.co/u/AssafKatz3)\ **Replies:** 0\ **Last updated:** [August 15, 2022, 11:04am UTC](https://discuss.elastic.co/t/using-app-kubernetes-io-name-label/312088 "2022-08-15T11:04:46Z")
Hi, I am trying to define service as the content of app.kubernetes.io/name label if it empty by: if !\[service\] and \[kubernetes\]\[labels\]\[app\_kubernetes\_io/name\] { mutate { add\_field =\> { "service… --- ## [Logstash match Thai timestamp](https://discuss.elastic.co/t/logstash-match-thai-timestamp/312077)
**Author:** [@Denis\_Smereka](https://discuss.elastic.co/u/Denis_Smereka)\ **Replies:** 0\ **Last updated:** [August 15, 2022, 8:22am UTC](https://discuss.elastic.co/t/logstash-match-thai-timestamp/312077 "2022-08-15T08:22:09Z")
I have certain logs which contain the following entries: 2022-08-10 12:55:58.535 Started. ๒๐๒๒-๐๘-๑๐ ๑๘:๓๔:๔๘.๒๗๑ Reader #0: ... The first line successfully matches using the pattern TIMESTAMP\_ISO8601. Second line als… --- ## [Ruby exception occurred: uninitialized constant when i am trying to devide a field value by 100](https://discuss.elastic.co/t/ruby-exception-occurred-uninitialized-constant-when-i-am-trying-to-devide-a-field-value-by-100/312058)
**Author:** [@Faiz\_Shamri](https://discuss.elastic.co/u/Faiz_Shamri)\ **Replies:** 7\ **Last updated:** [August 14, 2022, 6:19pm UTC](https://discuss.elastic.co/t/ruby-exception-occurred-uninitialized-constant-when-i-am-trying-to-devide-a-field-value-by-100/312058 "2022-08-14T18:19:08Z")
I am trying to devide filed value by 100 but i am getting Pipelines running {:count=\>1, :running\_pipelines=\>\[:main\], :non\_running\_pipelines=\>\[\]} \[ERROR\] 2022-08-14 00:10:07.738 \[\[main\]\>worker2\] ruby - Ruby exception occu… --- ## [\[Logstash\] \[Filebeat\] Using codec Plain and JSON for the same input](https://discuss.elastic.co/t/logstash-filebeat-using-codec-plain-and-json-for-the-same-input/310836)
**Author:** [@hdryx](https://discuss.elastic.co/u/hdryx)\ **Replies:** 8\ **Last updated:** [August 14, 2022, 4:55pm UTC](https://discuss.elastic.co/t/logstash-filebeat-using-codec-plain-and-json-for-the-same-input/310836 "2022-08-14T16:55:46Z")
Hi, Actually we have this architecture : Filebeat --\> Kafaka --\> Logstash The Logstash is using this input config : file1.conf input { kafka { codec =\> json bootstrap\_servers =\> "....." topics\_pattern =\>… --- ## [Logstatsh throws cluster - Exception in monitor thread while connecting to server](https://discuss.elastic.co/t/logstatsh-throws-cluster-exception-in-monitor-thread-while-connecting-to-server/312049)
**Author:** [@momarious](https://discuss.elastic.co/u/momarious)\ **Replies:** 4\ **Last updated:** [August 14, 2022, 2:10pm UTC](https://discuss.elastic.co/t/logstatsh-throws-cluster-exception-in-monitor-thread-while-connecting-to-server/312049 "2022-08-14T14:10:17Z")
Hello everyone this is my first post in this forum, well I would like to synchronize a MongoDB database with Elasticsearch thanks to Logstash, I used version 8.3.3 of ELK with a MongoDriver4.4. I have configured the pipe… --- ## [Help to parse date time](https://discuss.elastic.co/t/help-to-parse-date-time/312045)
**Author:** [@Claudio\_Ract\_Costa](https://discuss.elastic.co/u/Claudio_Ract_Costa)\ **Replies:** 1\ **Last updated:** [August 13, 2022, 4:15pm UTC](https://discuss.elastic.co/t/help-to-parse-date-time/312045 "2022-08-13T16:15:47Z")
Hi, I am trying to parse datetime bellow on logstash but i couldnt. The line is: "Aug 13, 2022 02:24:02.532177318 -03"|10.191.69.248|60542|10.110.62.45|3868|||PABLM01-SAECIS01-Gx-Openet;787252731;575065246;62f6e893-9… --- ## [How to remove brackets on grok patterns?](https://discuss.elastic.co/t/how-to-remove-brackets-on-grok-patterns/310547)
**Author:** [@SamuelSMendes](https://discuss.elastic.co/u/SamuelSMendes)\ **Replies:** 1\ **Last updated:** [August 12, 2022, 11:34pm UTC](https://discuss.elastic.co/t/how-to-remove-brackets-on-grok-patterns/310547 "2022-08-12T23:34:44Z")
Continuing the discussion from Remove characters in grok: On the previous post is an example of how to remove "," in grok. I was wondering if there is something to remove the "\[,\]" as well. I still don't know if is it'… --- ## [Logstash-to-Logstash Communication, Beats input?](https://discuss.elastic.co/t/logstash-to-logstash-communication-beats-input/312022)
**Author:** [@6igwig](https://discuss.elastic.co/u/6igwig)\ **Replies:** 2\ **Last updated:** [August 12, 2022, 5:29pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-communication-beats-input/312022 "2022-08-12T17:29:33Z")
I see that Logstash has a lumberjack input plug-in. I would have thought that the standard way for one logstash server to send logs to another logstash server would be to setup a lumberjack output and a lumberjack input.… --- ## [Referring a multi level field in Logstash](https://discuss.elastic.co/t/referring-a-multi-level-field-in-logstash/311999)
**Author:** [@yuanchuan](https://discuss.elastic.co/u/yuanchuan)\ **Replies:** 1\ **Last updated:** [August 12, 2022, 9:36am UTC](https://discuss.elastic.co/t/referring-a-multi-level-field-in-logstash/311999 "2022-08-12T09:36:49Z")
Hi, I would like to to do an output to a syslog when a fieldname with the name symantec\_endpoint.log.event\_description equals to "The....successfully" The way that I refer the field in my code is not working, need help… --- ## [Problem with my personal index creation for platform logs](https://discuss.elastic.co/t/problem-with-my-personal-index-creation-for-platform-logs/311998)
**Author:** [@SOFIA\_FERNANDEZ\_MORE](https://discuss.elastic.co/u/SOFIA_FERNANDEZ_MORE)\ **Replies:** 0\ **Last updated:** [August 12, 2022, 9:08am UTC](https://discuss.elastic.co/t/problem-with-my-personal-index-creation-for-platform-logs/311998 "2022-08-12T09:08:11Z")
Hello, i'm trying to deploy the ECK with Beat (Filebeat) to ingest logs and Logstah to collect, for example platform logs or namespaces logs of Kubernetes cluster. I'm using eck-operator 2.3.0 and Elastic stack in 8.3.3… --- ## [Create document with values from parent aggregation and sub aggregation](https://discuss.elastic.co/t/create-document-with-values-from-parent-aggregation-and-sub-aggregation/311954)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [August 11, 2022, 5:50pm UTC](https://discuss.elastic.co/t/create-document-with-values-from-parent-aggregation-and-sub-aggregation/311954 "2022-08-11T17:50:38Z")
Hi, Im queriyng elastic with logstash, exec input, and a bash script with curl. this is the response: "aggregations" : { "parent-agg" : { "buckets" : \[ { "key" : "one", "sub-aggs-t… --- ## [Install & configuration of live logs with logstash (docker)](https://discuss.elastic.co/t/install-configuration-of-live-logs-with-logstash-docker/311814)
**Author:** [@Dreinale](https://discuss.elastic.co/u/Dreinale)\ **Replies:** 2\ **Last updated:** [August 11, 2022, 1:43pm UTC](https://discuss.elastic.co/t/install-configuration-of-live-logs-with-logstash-docker/311814 "2022-08-11T13:43:06Z")
HI, I'm trying to install elk on the docker host and filebeat on another docker. So fo the first docker i followed that: Elasticsearch kibana & logstash with the ELK image. (i don't want security s i enabled it) I… --- ## [Logstash WARN: Could not index event to Elasticsearch .... Can't get text on a START\_OBJECT](https://discuss.elastic.co/t/logstash-warn-could-not-index-event-to-elasticsearch-cant-get-text-on-a-start-object/311910)
**Author:** [@d.silwon](https://discuss.elastic.co/u/d.silwon)\ **Replies:** 2\ **Last updated:** [August 11, 2022, 1:32pm UTC](https://discuss.elastic.co/t/logstash-warn-could-not-index-event-to-elasticsearch-cant-get-text-on-a-start-object/311910 "2022-08-11T13:32:46Z")
Dears, I have some problem with json. When I send json log to logstash there are many errors like this: Could not index event to Elasticsearch .... Can't get text on a START\_OBJECT at 1:670 My json log looks like: {… --- ## [How to use http input logstash](https://discuss.elastic.co/t/how-to-use-http-input-logstash/311928)
**Author:** [@cihady](https://discuss.elastic.co/u/cihady)\ **Replies:** 1\ **Last updated:** [August 11, 2022, 1:19pm UTC](https://discuss.elastic.co/t/how-to-use-http-input-logstash/311928 "2022-08-11T13:19:55Z")
I am trying to api call logstash by using http input but when I run logstash it failse and throws the error below Here is my config file input { http { host=\>0.0.0.0 port =\>2020 } } output { elasticsearch { … --- ## [Add new document using logstash](https://discuss.elastic.co/t/add-new-document-using-logstash/311833)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 2\ **Last updated:** [August 11, 2022, 7:23am UTC](https://discuss.elastic.co/t/add-new-document-using-logstash/311833 "2022-08-11T07:23:52Z")
Hi, i have a file so i read this file using filebeat and i do the multilne like this filebeat.yml: - type: log enabled: true paths: - D:\\elastic\_stack\\logs\\\* fields: kafka\_topic: "kafka-topic-1" ta… --- ## [Veeam Logstash Grok](https://discuss.elastic.co/t/veeam-logstash-grok/311774)
**Author:** [@rcraigncs](https://discuss.elastic.co/u/rcraigncs)\ **Replies:** 16\ **Last updated:** [August 10, 2022, 10:57pm UTC](https://discuss.elastic.co/t/veeam-logstash-grok/311774 "2022-08-10T22:57:30Z")
Hello, I'm trying to grok logs from Veeam logs \> filebeat file stream \> logstash to create a better filter on logs collected from Veeam. I'm relatively new when it comes to grok and I'm stuck on how I would go about gro… --- ## [Merge two csv files into single csv file using logstash](https://discuss.elastic.co/t/merge-two-csv-files-into-single-csv-file-using-logstash/311478)
**Author:** [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Replies:** 4\ **Last updated:** [August 10, 2022, 2:21pm UTC](https://discuss.elastic.co/t/merge-two-csv-files-into-single-csv-file-using-logstash/311478 "2022-08-10T14:21:31Z")
Hi All, I am trying to merge two csv files into single csv file using logstash and below is the sample data and config I have tried, logstash-1.csv Sample,Gender,Age 1,dentist,10 2,doctor,20 3,rep,30 logstash-2.csv S… --- ## [Logstash conf file will not work after filter section is added](https://discuss.elastic.co/t/logstash-conf-file-will-not-work-after-filter-section-is-added/311770)
**Author:** [@Lori\_Wallace](https://discuss.elastic.co/u/Lori_Wallace)\ **Replies:** 6\ **Last updated:** [August 10, 2022, 1:02pm UTC](https://discuss.elastic.co/t/logstash-conf-file-will-not-work-after-filter-section-is-added/311770 "2022-08-10T13:02:28Z")
I am trying to ingest logs from a checkpoint firewall. My logstash config file checks out and works as long as I don't have the filter section in it. However, I need the data filtered so can you guys help me figure out w… --- ## [Logstash 6.8.23 Cannot put data into elastic index, mapping issue](https://discuss.elastic.co/t/logstash-6-8-23-cannot-put-data-into-elastic-index-mapping-issue/310940)
**Author:** [@Akhil\_Chandran](https://discuss.elastic.co/u/Akhil_Chandran)\ **Replies:** 8\ **Last updated:** [August 10, 2022, 12:59pm UTC](https://discuss.elastic.co/t/logstash-6-8-23-cannot-put-data-into-elastic-index-mapping-issue/310940 "2022-08-10T12:59:26Z")
HI, I am trying to pass data from logstash to ES index, but it throws error as shown below 2022-07-28T10:28:17,909\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\[… --- ## [To control the log flows in kibana end through logstash config](https://discuss.elastic.co/t/to-control-the-log-flows-in-kibana-end-through-logstash-config/310499)
**Author:** [@yasar](https://discuss.elastic.co/u/yasar)\ **Replies:** 16\ **Last updated:** [August 10, 2022, 8:57am UTC](https://discuss.elastic.co/t/to-control-the-log-flows-in-kibana-end-through-logstash-config/310499 "2022-08-10T08:57:32Z")
HI team, We have getting the huge logs when application team doing the Performance testing. At that time, Our cluster is not stable due to unable to handle the huge much of load. Could you please help us to control the… --- ## [Elasticsearch - Could not index event to Elasticsearch status=\>400](https://discuss.elastic.co/t/elasticsearch-could-not-index-event-to-elasticsearch-status-400/311101)
**Author:** [@Maruthappan\_Muthu](https://discuss.elastic.co/u/Maruthappan_Muthu)\ **Replies:** 4\ **Last updated:** [August 10, 2022, 6:38am UTC](https://discuss.elastic.co/t/elasticsearch-could-not-index-event-to-elasticsearch-status-400/311101 "2022-08-10T06:38:35Z")
We are trying to poll the data from a device (PDU) through SNMP Input Plugin. The device MIB file has been imported to ELK logstash, as per SNMP input plugin | Logstash Reference \[8.3\] | Elastic. When executing the snmp… --- ## [Where is logstash.service file?](https://discuss.elastic.co/t/where-is-logstash-service-file/311784)
**Author:** [@hellocomputer](https://discuss.elastic.co/u/hellocomputer)\ **Replies:** 0\ **Last updated:** [August 10, 2022, 3:47am UTC](https://discuss.elastic.co/t/where-is-logstash-service-file/311784 "2022-08-10T03:47:00Z")
Hello I'm using ubuntu20.04 and logstash 8.3.2 version. I want to set TimeoutStopSec=300 this command! but I couldn't find logstash.service file in /etc/systemd/system/logstash.service in this location. so, wh… --- ## [How to filter json object field class, java.lang.ClassCastException: class org.jruby.RubyHash cannot be cast to class org.jruby.RubyIO](https://discuss.elastic.co/t/how-to-filter-json-object-field-class-java-lang-classcastexception-class-org-jruby-rubyhash-cannot-be-cast-to-class-org-jruby-rubyio/311779)
**Author:** [@HeChuanXUPT](https://discuss.elastic.co/u/HeChuanXUPT)\ **Replies:** 0\ **Last updated:** [August 10, 2022, 1:20am UTC](https://discuss.elastic.co/t/how-to-filter-json-object-field-class-java-lang-classcastexception-class-org-jruby-rubyhash-cannot-be-cast-to-class-org-jruby-rubyio/311779 "2022-08-10T01:20:45Z")
input file content: {"\_source": {"timestamp": 1612256372000, "date": "2021-02-02 16:59:32", "ip": "127.175.208.130"}} logstash.conf: input { file { path =\> \["/tmp/test\_file"\] start\_position =\> "beginning" … --- ## [How to set start and end point of a prebuild grok pattern](https://discuss.elastic.co/t/how-to-set-start-and-end-point-of-a-prebuild-grok-pattern/311749)
**Author:** [@cihady](https://discuss.elastic.co/u/cihady)\ **Replies:** 6\ **Last updated:** [August 9, 2022, 8:43pm UTC](https://discuss.elastic.co/t/how-to-set-start-and-end-point-of-a-prebuild-grok-pattern/311749 "2022-08-09T20:43:59Z")
My grok patter broken when there is a |(pipe) in my data . here is my log \> 16:29:52.143 \[kafka-producer-network-thread | producer-1\] INFO c.h.h.d.e.ApiMessageProducer - ===============================================… --- ## [Logstash deletes log files after processing](https://discuss.elastic.co/t/logstash-deletes-log-files-after-processing/311743)
**Author:** [@Indrajit](https://discuss.elastic.co/u/Indrajit)\ **Replies:** 1\ **Last updated:** [August 9, 2022, 3:23pm UTC](https://discuss.elastic.co/t/logstash-deletes-log-files-after-processing/311743 "2022-08-09T15:23:34Z")
logstash delete log files after processing. My input follows below. Is there a way to force logstash not to delete files after processing? input { file { path =\> "/sftpshare/dplogs/dplogs/prod/ext01/… --- ## [Wirting Filter](https://discuss.elastic.co/t/wirting-filter/311752)
**Author:** [@AbdulRahman\_Mahmoud](https://discuss.elastic.co/u/AbdulRahman_Mahmoud)\ **Replies:** 0\ **Last updated:** [August 9, 2022, 3:02pm UTC](https://discuss.elastic.co/t/wirting-filter/311752 "2022-08-09T15:02:45Z")
Hi, I have tried to use following repo to create a custom filter, for the step of "Package and Deploy" I see it is written that the snippet of "plugin info" - that contatins"group, version, description and pluginInfo.\*"… --- ## [Logstash filter add new calculated float value](https://discuss.elastic.co/t/logstash-filter-add-new-calculated-float-value/311724)
**Author:** [@azeiner](https://discuss.elastic.co/u/azeiner)\ **Replies:** 3\ **Last updated:** [August 9, 2022, 1:07pm UTC](https://discuss.elastic.co/t/logstash-filter-add-new-calculated-float-value/311724 "2022-08-09T13:07:21Z")
I've got Data from a Json Object and what i want is to calculate some Values of the same Object e.g. filter { json { source =\> "message" } mutate { add\_field =\> { "xxx1" =\> "Float(%{\[… --- ## [Use nested field names in Grok custom patterns](https://discuss.elastic.co/t/use-nested-field-names-in-grok-custom-patterns/311728)
**Author:** [@vilman](https://discuss.elastic.co/u/vilman)\ **Replies:** 1\ **Last updated:** [August 9, 2022, 12:49pm UTC](https://discuss.elastic.co/t/use-nested-field-names-in-grok-custom-patterns/311728 "2022-08-09T12:49:40Z")
Hi, I'm trying to map a nested field from a custom grok expression. So, instead of using: (?=.\*?Message Type: (?\\[^;\]+))? I would like to map test1 to \[event\]\[name\]. Tried things like: (?=.\*?Message Type: (?\<\[… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=120) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=122)