# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=588 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 589 --- ## [How to escape character in logstash](https://discuss.elastic.co/t/how-to-escape-character-in-logstash/171327)
**Author:** [@mandar.raj](https://discuss.elastic.co/u/mandar.raj)\ **Replies:** 2\ **Last updated:** [March 7, 2019, 3:28pm UTC](https://discuss.elastic.co/t/how-to-escape-character-in-logstash/171327 "2019-03-07T15:28:58Z")
I want to execute below sql statement in input of logstash configuration file but it fails to compile statement =\> "select ALERT\_ID, regexp\_replace(regexp\_substr(html\_file\_key,'(.?)'),'|','') as alertDate, regexp\_replac… --- ## [Dynamic createlogstash config file](https://discuss.elastic.co/t/dynamic-createlogstash-config-file/170270)
**Author:** [@supermario](https://discuss.elastic.co/u/supermario)\ **Replies:** 1\ **Last updated:** [March 7, 2019, 3:20pm UTC](https://discuss.elastic.co/t/dynamic-createlogstash-config-file/170270 "2019-03-07T15:20:06Z")
logstash was started. I create some jdbc input config file to conf.d directory. whether logstash can exec new config. --- ## [Fresh install on debian 9 - where is the bynary?](https://discuss.elastic.co/t/fresh-install-on-debian-9-where-is-the-bynary/171323)
**Author:** [@realtebo](https://discuss.elastic.co/u/realtebo)\ **Replies:** 0\ **Last updated:** [March 7, 2019, 2:22pm UTC](https://discuss.elastic.co/t/fresh-install-on-debian-9-where-is-the-bynary/171323 "2019-03-07T14:22:14Z")
I succesfully installed logstash on a debian 9 using apt following the official doc.. I am able to see logstash.service loaded and running using sudo systemctl status logstash.service Now I must execute bin/logstash bu… --- ## [AWS SES Parsing Nested json](https://discuss.elastic.co/t/aws-ses-parsing-nested-json/169423)
**Author:** [@neo](https://discuss.elastic.co/u/neo)\ **Replies:** 1\ **Last updated:** [March 7, 2019, 12:42pm UTC](https://discuss.elastic.co/t/aws-ses-parsing-nested-json/169423 "2019-03-07T12:42:45Z")
Hi Guys, I need your help I get logs from S3 buckets SES, the logs change, one for a few, I get field with key value structure, I'm trying to parse it but I'm getting errors. here is my conf file,result and error log… --- ## [Grok pattern not working in Logstast](https://discuss.elastic.co/t/grok-pattern-not-working-in-logstast/171289)
**Author:** [@Vijay\_Khadia](https://discuss.elastic.co/u/Vijay_Khadia)\ **Replies:** 0\ **Last updated:** [March 7, 2019, 11:33am UTC](https://discuss.elastic.co/t/grok-pattern-not-working-in-logstast/171289 "2019-03-07T11:33:08Z")
I have to parse the java Error logs and pass the result to elastic-search. The pattern that I created is working fine when tested on the site 'https://grokdebug.herokuapp.com/'. However the same grok-pattern when I am c… --- ## [Add field dynamically based on different match](https://discuss.elastic.co/t/add-field-dynamically-based-on-different-match/171286)
**Author:** [@tsuijie](https://discuss.elastic.co/u/tsuijie)\ **Replies:** 0\ **Last updated:** [March 7, 2019, 11:19am UTC](https://discuss.elastic.co/t/add-field-dynamically-based-on-different-match/171286 "2019-03-07T11:19:51Z")
We'd like to add an index based on which pattern was actually hit, which looks like this, but it won't work: filter { grok { patterns\_dir =\> "./patterns" match =\> { "message" =\> "%{A}" } add\_fiel… --- ## [Join Data from a database to an event based on field value](https://discuss.elastic.co/t/join-data-from-a-database-to-an-event-based-on-field-value/171190)
**Author:** [@Simon\_Becker](https://discuss.elastic.co/u/Simon_Becker)\ **Replies:** 2\ **Last updated:** [March 7, 2019, 10:43am UTC](https://discuss.elastic.co/t/join-data-from-a-database-to-an-event-based-on-field-value/171190 "2019-03-07T10:43:40Z")
Hi everyone, i have a pretty complicated problem to solve: I would like enrich my datastream that I recieve from applications with metadata. I need this later to create on joined fields user restrictions and so on. Ev… --- ## [Handle Large Messages in Logstash](https://discuss.elastic.co/t/handle-large-messages-in-logstash/171229)
**Author:** [@AMITH1284](https://discuss.elastic.co/u/AMITH1284)\ **Replies:** 2\ **Last updated:** [March 7, 2019, 9:46am UTC](https://discuss.elastic.co/t/handle-large-messages-in-logstash/171229 "2019-03-07T09:46:05Z")
Hello. i am using multiline patterns in my logstash input, it works well. sometimes i have very large logs say 20MB, logstash is crashing when this logs comes in. is there a way to handle it? i am also using grok patter… --- ## [Multiple if else problem in output in Logstash](https://discuss.elastic.co/t/multiple-if-else-problem-in-output-in-logstash/170473)
**Author:** [@Varun\_Mahajan](https://discuss.elastic.co/u/Varun_Mahajan)\ **Replies:** 6\ **Last updated:** [March 7, 2019, 8:56am UTC](https://discuss.elastic.co/t/multiple-if-else-problem-in-output-in-logstash/170473 "2019-03-07T08:56:11Z")
Hi Team, I am new to elastic search and logstash so i have one basic small query, Hope i will get quick reply here I am trying to apply multiple if else in my logstash but i am failed to do it , Here's the example out… --- ## [SNMP data to visualisations not easy possible like metricbeats](https://discuss.elastic.co/t/snmp-data-to-visualisations-not-easy-possible-like-metricbeats/167432)
**Author:** [@pw64](https://discuss.elastic.co/u/pw64)\ **Replies:** 2\ **Last updated:** [March 7, 2019, 8:54am UTC](https://discuss.elastic.co/t/snmp-data-to-visualisations-not-easy-possible-like-metricbeats/167432 "2019-03-07T08:54:13Z")
Hi All, With ES 6.6 it is great to see SNMP input. Typically something we would like to see is trending information of dataflow of routers, switches etc. And the trap host functionality looks great. But... All the gath… --- ## [How to filter Data with Spaces in Logstash](https://discuss.elastic.co/t/how-to-filter-data-with-spaces-in-logstash/170939)
**Author:** [@samyo](https://discuss.elastic.co/u/samyo)\ **Replies:** 19\ **Last updated:** [March 7, 2019, 8:09am UTC](https://discuss.elastic.co/t/how-to-filter-data-with-spaces-in-logstash/170939 "2019-03-07T08:09:33Z")
Hello Folks, i need some Help here to read this Example Data in my Text editor with its Spaces in Logstash. Example Data in my Text editor as follows: Hello World Hello back Plea… --- ## [Parse xml from database and index into elastic seasrch](https://discuss.elastic.co/t/parse-xml-from-database-and-index-into-elastic-seasrch/171070)
**Author:** [@mandar.raj](https://discuss.elastic.co/u/mandar.raj)\ **Replies:** 4\ **Last updated:** [March 7, 2019, 3:32am UTC](https://discuss.elastic.co/t/parse-xml-from-database-and-index-into-elastic-seasrch/171070 "2019-03-07T03:32:07Z")
Please help me out to read xml from database column and store indexes in elastic search by xml tag name My XML looks like below. \ 2019-01-10 01:56:43 100 … --- ## [Changing dotted json field names to valid json form for Logstash](https://discuss.elastic.co/t/changing-dotted-json-field-names-to-valid-json-form-for-logstash/171026)
**Author:** [@Sanghyeon\_Park](https://discuss.elastic.co/u/Sanghyeon_Park)\ **Replies:** 2\ **Last updated:** [March 7, 2019, 12:24am UTC](https://discuss.elastic.co/t/changing-dotted-json-field-names-to-valid-json-form-for-logstash/171026 "2019-03-07T00:24:07Z")
Hello! I'm new to Logstash, and I'm trying filtering and mutating input logs from Filebeat. First of all, I have read this article, and I know that I cannot use dot separated field names. Field name cannot contain dot… --- ## [Parsing Text file with Pipe(|) separated fields](https://discuss.elastic.co/t/parsing-text-file-with-pipe-separated-fields/171024)
**Author:** [@RaTheDev](https://discuss.elastic.co/u/RaTheDev)\ **Replies:** 2\ **Last updated:** [March 6, 2019, 10:55pm UTC](https://discuss.elastic.co/t/parsing-text-file-with-pipe-separated-fields/171024 "2019-03-06T22:55:30Z")
I am a newbie and started with Logstash I have raw text file with data in the below format, with first row as Column headers and following with data separated with Pipe as delimiter. I was trying to read the data and f… --- ## [Logstash Jenkins plugin - Do I need to set logstashSend failBuild to false to avoid failing entire build?](https://discuss.elastic.co/t/logstash-jenkins-plugin-do-i-need-to-set-logstashsend-failbuild-to-false-to-avoid-failing-entire-build/171210)
**Author:** [@ps895r](https://discuss.elastic.co/u/ps895r)\ **Replies:** 1\ **Last updated:** [March 6, 2019, 10:27pm UTC](https://discuss.elastic.co/t/logstash-jenkins-plugin-do-i-need-to-set-logstashsend-failbuild-to-false-to-avoid-failing-entire-build/171210 "2019-03-06T22:27:12Z")
Hi everyone, I have this line in a shared library that's used by Jenkins. logstashSend failBuild: true, maxLines: 1000 My current understanding is this will fail the build if ELK crashes or this line doesn't work. Ple… --- ## [Logstash posting messages to syslog](https://discuss.elastic.co/t/logstash-posting-messages-to-syslog/171203)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 0\ **Last updated:** [March 6, 2019, 8:41pm UTC](https://discuss.elastic.co/t/logstash-posting-messages-to-syslog/171203 "2019-03-06T20:41:34Z")
I have set my log dir in logstash.yml as follows: path.logs: /var/log/logstash However, important messages like the following are going to /var/log/messages, i.e. the system log. Mar 3 03:36:33 elklog01 logstash\[6342\]… --- ## [Logstash's grok GREEDYDATA pattern takes word before it starts](https://discuss.elastic.co/t/logstashs-grok-greedydata-pattern-takes-word-before-it-starts/170973)
**Author:** [@Marko\_Todoric](https://discuss.elastic.co/u/Marko_Todoric)\ **Replies:** 2\ **Last updated:** [March 6, 2019, 5:22pm UTC](https://discuss.elastic.co/t/logstashs-grok-greedydata-pattern-takes-word-before-it-starts/170973 "2019-03-06T17:22:14Z")
Hello everyone, I've been modifying GROK patterns for postfix from GitHub - ActionScripted/elastic-kibana-postfix: Kibana dashboards, visualizations and searches for Postfix so that it can include "warning" from postfi… --- ## [Logstash S3 Output Plugin not working](https://discuss.elastic.co/t/logstash-s3-output-plugin-not-working/171000)
**Author:** [@hiteshs](https://discuss.elastic.co/u/hiteshs)\ **Replies:** 3\ **Last updated:** [March 6, 2019, 4:28pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-not-working/171000 "2019-03-06T16:28:08Z")
My goal is to send all of the logs from all the remote servers to a S3 bucket. I decided to go with Logstash S3 output plugin since we're already running a ELK setup. So each remote server is sending the log files to the… --- ## [ComMysqlJdbcExceptionsJdbc4::CommunicationsException: Communications link failure, if i schedule query every system hour](https://discuss.elastic.co/t/commysqljdbcexceptionsjdbc4-communications-link-failure-if-i-schedule-query-every-system-hour/170330)
**Author:** [@ApekshaB](https://discuss.elastic.co/u/ApekshaB)\ **Replies:** 4\ **Last updated:** [March 6, 2019, 2:49pm UTC](https://discuss.elastic.co/t/commysqljdbcexceptionsjdbc4-communications-link-failure-if-i-schedule-query-every-system-hour/170330 "2019-03-06T14:49:24Z")
Kindly find my conf file as below: input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://172.23.125.52:3304/faredge" jdbc\_user =\> "faredge" jdbc\_password =\> "faredge" jdbc\_driver\_library =\> "D:\\Apeksha\\ELK\_… --- ## [Messages/Lines are coming more than once in log stash output](https://discuss.elastic.co/t/messages-lines-are-coming-more-than-once-in-log-stash-output/171121)
**Author:** [@Mayurbiw](https://discuss.elastic.co/u/Mayurbiw)\ **Replies:** 0\ **Last updated:** [March 6, 2019, 1:26pm UTC](https://discuss.elastic.co/t/messages-lines-are-coming-more-than-once-in-log-stash-output/171121 "2019-03-06T13:26:37Z")
Here is my configuration file input { file { path =\> "myFile.xml" start\_position =\> "beginning" codec =\> multiline { pattern =\> "((^\)|(^\))" negate =\> … --- ## [How i can create index on xml tags](https://discuss.elastic.co/t/how-i-can-create-index-on-xml-tags/170175)
**Author:** [@mandar.raj](https://discuss.elastic.co/u/mandar.raj)\ **Replies:** 12\ **Last updated:** [March 6, 2019, 2:09pm UTC](https://discuss.elastic.co/t/how-i-can-create-index-on-xml-tags/170175 "2019-03-06T14:09:57Z")
Hello, I am new to logstash, i have one requirement like below. i want to create index on xml tag. this xml is present in database table. I am able to index on column which having this xml. but the requirement is to in… --- ## [UDP netflow packets arrive, netflow does not process them](https://discuss.elastic.co/t/udp-netflow-packets-arrive-netflow-does-not-process-them/171001)
**Author:** [@jpbourke](https://discuss.elastic.co/u/jpbourke)\ **Replies:** 5\ **Last updated:** [March 6, 2019, 1:50pm UTC](https://discuss.elastic.co/t/udp-netflow-packets-arrive-netflow-does-not-process-them/171001 "2019-03-06T13:50:57Z")
I have two servers, getting logs from the same sources, parallel streams of netflow and syslog. 64Gb RAM, 8 cores, 500Gb SSD VMs running on ESXI 6.7. I build the servers using Ansible and Cobbler so they are "identical… --- ## [One row from postgres table inserted into elasticsearch](https://discuss.elastic.co/t/one-row-from-postgres-table-inserted-into-elasticsearch/170883)
**Author:** [@rhari](https://discuss.elastic.co/u/rhari)\ **Replies:** 2\ **Last updated:** [March 6, 2019, 1:27pm UTC](https://discuss.elastic.co/t/one-row-from-postgres-table-inserted-into-elasticsearch/170883 "2019-03-06T13:27:56Z")
New to Logstash, ElasticSearch etc. Created new LogStash configuration for pulling records from Postgres DB and push to elasticsearch, however seeing only only row is pushed to elasticsearch, input { jdbc { # Postgre… --- ## [Persistent queue and kafka output](https://discuss.elastic.co/t/persistent-queue-and-kafka-output/171120)
**Author:** [@JohanRask](https://discuss.elastic.co/u/JohanRask)\ **Replies:** 0\ **Last updated:** [March 6, 2019, 1:19pm UTC](https://discuss.elastic.co/t/persistent-queue-and-kafka-output/171120 "2019-03-06T13:19:26Z")
Hi, We are reading from kafka topics, performing different transformations and then write back to other kafka topics. Since logstash does periodic commits to kafka we where not able to use inmemory queues and we are i… --- ## [How to let Logstash start indexing into Elasticsearch on specific Word](https://discuss.elastic.co/t/how-to-let-logstash-start-indexing-into-elasticsearch-on-specific-word/171112)
**Author:** [@samyo](https://discuss.elastic.co/u/samyo)\ **Replies:** 0\ **Last updated:** [March 6, 2019, 12:41pm UTC](https://discuss.elastic.co/t/how-to-let-logstash-start-indexing-into-elasticsearch-on-specific-word/171112 "2019-03-06T12:41:33Z")
hello Folks, i am trying to insert(indexing) some unstructured Sentences from my Text editor into Elasticsearch, and only when Logstash arraive a specific Word than start to index all other Words and sentences after thi… --- ## [Reverse\_mapping for cef input codec not working](https://discuss.elastic.co/t/reverse-mapping-for-cef-input-codec-not-working/169184)
**Author:** [@FreddyFernando](https://discuss.elastic.co/u/FreddyFernando)\ **Replies:** 3\ **Last updated:** [March 6, 2019, 12:41pm UTC](https://discuss.elastic.co/t/reverse-mapping-for-cef-input-codec-not-working/169184 "2019-03-06T12:41:19Z")
Cef reverse\_mapping does not work. I am expecting logstash to give me short form of cef, instead it gives me the long form. Version: Logstash 6.5.0 (cef codec version 5.0.6) Operating System: Ubuntu 16.04 Config File … --- ## [Logstash container crashes with weird error](https://discuss.elastic.co/t/logstash-container-crashes-with-weird-error/171105)
**Author:** [@Ryukote](https://discuss.elastic.co/u/Ryukote)\ **Replies:** 0\ **Last updated:** [March 6, 2019, 12:07pm UTC](https://discuss.elastic.co/t/logstash-container-crashes-with-weird-error/171105 "2019-03-06T12:07:27Z")
Hello, I am trying to make an integration between Elasticsearch and SQL Server via Logstash. I have created my dockerfile for that integration where base image is logstash. First the error I am getting is: elasticsq… --- ## [Field references sometimes resolved and sometimes not](https://discuss.elastic.co/t/field-references-sometimes-resolved-and-sometimes-not/170511)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 1\ **Last updated:** [March 6, 2019, 11:59am UTC](https://discuss.elastic.co/t/field-references-sometimes-resolved-and-sometimes-not/170511 "2019-03-06T11:59:38Z")
Elasticsearch + LogStash 6.5 / FileBeat 6.6 / Windows Server In FileBeat we define the fields "source", "server" and "system" and add them together in Logstash to a field logfile according to the template %{\[fields\]\[sou… --- ## [Overriding timestamp, now logs are out of order](https://discuss.elastic.co/t/overriding-timestamp-now-logs-are-out-of-order/170720)
**Author:** [@BKG](https://discuss.elastic.co/u/BKG)\ **Replies:** 1\ **Last updated:** [March 6, 2019, 11:22am UTC](https://discuss.elastic.co/t/overriding-timestamp-now-logs-are-out-of-order/170720 "2019-03-06T11:22:36Z")
Hi All, I have successfully extracted our date and used it to override the @timestamp, however because our application uses microseconds and Elasticsearch only supports milliseconds, my logs for some events are now appe… --- ## [Perfomance/Best Practice Question: Translate plugin vs ES Plugin](https://discuss.elastic.co/t/perfomance-best-practice-question-translate-plugin-vs-es-plugin/171087)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 2\ **Last updated:** [March 6, 2019, 11:19am UTC](https://discuss.elastic.co/t/perfomance-best-practice-question-translate-plugin-vs-es-plugin/171087 "2019-03-06T11:19:29Z")
Hi All, I have to enrich in LS data related to phone calls events based on prefixes of the of the from number (A number) and of the to number (B Number) I have two options Using translate filter plugin I need 5 YAM… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=587) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=589)