# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=695 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 696 --- ## [Parsed JSON and Grokked Strings in the same pipeline](https://discuss.elastic.co/t/parsed-json-and-grokked-strings-in-the-same-pipeline/146211)
**Author:** [@isaacsanders](https://discuss.elastic.co/u/isaacsanders)\ **Replies:** 2\ **Last updated:** [August 27, 2018, 9:32pm UTC](https://discuss.elastic.co/t/parsed-json-and-grokked-strings-in-the-same-pipeline/146211 "2018-08-27T21:32:16Z")
The flow of data in my system is described below: Files on my filesystem, some log, some jsonlog Filebeats a. Human readable log lines b. JSON log lines Logstash a. grok Something is coercing the JSON object from m… --- ## [All Running but not Parsing or sending to Elasticsearch](https://discuss.elastic.co/t/all-running-but-not-parsing-or-sending-to-elasticsearch/145894)
**Author:** [@LloydA](https://discuss.elastic.co/u/LloydA)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 8:16pm UTC](https://discuss.elastic.co/t/all-running-but-not-parsing-or-sending-to-elasticsearch/145894 "2018-08-27T20:16:15Z")
Hi All. I've set up an Logstash and Elasticsearch on a DigitalOcean Droplet. I am looking to index a CSV file in Elasticsearch by putting it through Logstash. Just to get it running, I'm using GREEDYDATA for my grok par… --- ## [Want to save elasticsearch data index to oracle database](https://discuss.elastic.co/t/want-to-save-elasticsearch-data-index-to-oracle-database/145846)
**Author:** [@sadhu](https://discuss.elastic.co/u/sadhu)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 8:12pm UTC](https://discuss.elastic.co/t/want-to-save-elasticsearch-data-index-to-oracle-database/145846 "2018-08-27T20:12:12Z")
Is it possible to store elasticsearch data into oracle database using logstash --- ## [Send data in elasticsearch parts](https://discuss.elastic.co/t/send-data-in-elasticsearch-parts/145739)
**Author:** [@Olha1](https://discuss.elastic.co/u/Olha1)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 8:11pm UTC](https://discuss.elastic.co/t/send-data-in-elasticsearch-parts/145739 "2018-08-27T20:11:34Z")
Hi! Please, give me advice, can i have download data in elasticsearch, from database, in a few parts? --- ## [Not seeing .conf execution logs in Ubuntu](https://discuss.elastic.co/t/not-seeing-conf-execution-logs-in-ubuntu/145592)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 8:08pm UTC](https://discuss.elastic.co/t/not-seeing-conf-execution-logs-in-ubuntu/145592 "2018-08-27T20:08:56Z")
Hi - Am trying to execute a .conf file in my ubuntu server as below - sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf -l /var/log/logstash/test.log I'd see the conf file is executing fine, but… --- ## [Parsing web hook data in logstash](https://discuss.elastic.co/t/parsing-web-hook-data-in-logstash/145224)
**Author:** [@iamlearner123](https://discuss.elastic.co/u/iamlearner123)\ **Replies:** 10\ **Last updated:** [August 27, 2018, 8:00pm UTC](https://discuss.elastic.co/t/parsing-web-hook-data-in-logstash/145224 "2018-08-27T20:00:08Z")
Hello, I am bringing the data thru webhook into logstash but can i please know how the parse that data and send it to separate index. I have provided the http://logstashIp/\_hooks/gitlab in the gitlab webhook section… --- ## [Accessing log files on a remote Unix server from a Windows server](https://discuss.elastic.co/t/accessing-log-files-on-a-remote-unix-server-from-a-windows-server/145623)
**Author:** [@cbril](https://discuss.elastic.co/u/cbril)\ **Replies:** 5\ **Last updated:** [August 27, 2018, 7:56pm UTC](https://discuss.elastic.co/t/accessing-log-files-on-a-remote-unix-server-from-a-windows-server/145623 "2018-08-27T19:56:16Z")
\---------- BACKGROUND ---------- I am setting up a production cluster that has a mix of Windows and Unix servers. The cluster will involve dozens of servers. I have one master Windows server that is running a single in… --- ## [Jenkins server log tracking in logstash](https://discuss.elastic.co/t/jenkins-server-log-tracking-in-logstash/146216)
**Author:** [@prangan](https://discuss.elastic.co/u/prangan)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 7:49pm UTC](https://discuss.elastic.co/t/jenkins-server-log-tracking-in-logstash/146216 "2018-08-27T19:49:37Z")
Hi, I'm setting up elasticsearch suite for the first time. I managed to log the syslog files. Now, trying to source Jenkins server log file. In Jenkins server log, not every line is printed with the date. For example,… --- ## [Logstash java.lang.OutOfMemoryError when reading yml file](https://discuss.elastic.co/t/logstash-java-lang-outofmemoryerror-when-reading-yml-file/146014)
**Author:** [@blueren](https://discuss.elastic.co/u/blueren)\ **Replies:** 3\ **Last updated:** [August 27, 2018, 6:01pm UTC](https://discuss.elastic.co/t/logstash-java-lang-outofmemoryerror-when-reading-yml-file/146014 "2018-08-27T18:01:48Z")
I'm currently running logstash with elastiflow configured within a docker container. When logstash starts up, it attemps to read from ip\_rep\_basic.yml which is approx 48MB and 1559862 lines long. Logstash seems to get … --- ## [How to connect AWS Redis servers (elastic-cache service) with SSL in-transit true as input plugin?](https://discuss.elastic.co/t/how-to-connect-aws-redis-servers-elastic-cache-service-with-ssl-in-transit-true-as-input-plugin/146208)
**Author:** [@Nikhil\_Chabukswar](https://discuss.elastic.co/u/Nikhil_Chabukswar)\ **Replies:** 0\ **Last updated:** [August 27, 2018, 3:55pm UTC](https://discuss.elastic.co/t/how-to-connect-aws-redis-servers-elastic-cache-service-with-ssl-in-transit-true-as-input-plugin/146208 "2018-08-27T15:55:08Z")
I need to ship logs from dumped in redis queue with logstash. for that, I have used the following configuration in logstash.conf file: input { redis { host =\> "master.qa-logging-redis.\*\*\*\*\*\*.use1.cache.amazonaws.com"… --- ## [GeoIP Field Selection](https://discuss.elastic.co/t/geoip-field-selection/145968)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [August 27, 2018, 3:26pm UTC](https://discuss.elastic.co/t/geoip-field-selection/145968 "2018-08-27T15:26:50Z")
Trying to figure out the syntax of the Geoip filter field option. It says it accepts an array but I don't understand how you format a group of unrelated strings inside of an array....at least going by the example linked… --- ## [Can't disable logstash.codecs.json logging](https://discuss.elastic.co/t/cant-disable-logstash-codecs-json-logging/146199)
**Author:** [@eperry](https://discuss.elastic.co/u/eperry)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 2:58pm UTC](https://discuss.elastic.co/t/cant-disable-logstash-codecs-json-logging/146199 "2018-08-27T14:58:23Z")
Logstash 6.3 I am trying to disable logging from the port 9600 and seems to reject every value though I can change other loggers just not logstash.codecs.json "error" : "unrecognized option \[logstash.codecs.json\]" D… --- ## [HTTP parse error, malformed request (): #\ **Author:** [@krpatel19](https://discuss.elastic.co/u/krpatel19)\ **Replies:** 0\ **Last updated:** [August 27, 2018, 1:39pm UTC](https://discuss.elastic.co/t/http-parse-error-malformed-request-puma-invalid-http-format-parsing-fails/146184 "2018-08-27T13:39:33Z") Hi , I am getting error logs in logstash 5.2.1 with below error. 2018-08-27 15:36:01 +0200: HTTP parse error, malformed request (): #\ 2018-08-27 15:36:01 +02… --- ## [Split Filter Parse Failure](https://discuss.elastic.co/t/split-filter-parse-failure/145958)
**Author:** [@Rahul\_Javadekar](https://discuss.elastic.co/u/Rahul_Javadekar)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 12:55pm UTC](https://discuss.elastic.co/t/split-filter-parse-failure/145958 "2018-08-27T12:55:39Z")
Hi, please help with splitting the following JSON at "value": {"count":3, "value":\[ {"id":232, "buildNumber":"20180706.7", "status":"completed"}, {"id":231, "buildNumber":"20180706.6", "status":"completed"}, {"id":2… --- ## [Logstash Exec Input Plugin throws OutofMemory Error](https://discuss.elastic.co/t/logstash-exec-input-plugin-throws-outofmemory-error/146172)
**Author:** [@Dheeraj\_Gupta](https://discuss.elastic.co/u/Dheeraj_Gupta)\ **Replies:** 0\ **Last updated:** [August 27, 2018, 12:04pm UTC](https://discuss.elastic.co/t/logstash-exec-input-plugin-throws-outofmemory-error/146172 "2018-08-27T12:04:15Z")
Hi, We are running a program to read some events off our API and send them to Elasticsearch. Since http poller plugin does not support dynamic parameters (in our case timestamp when last poll was done), we use a custom … --- ## [Ignore Error Messages](https://discuss.elastic.co/t/ignore-error-messages/145911)
**Author:** [@pavan1991](https://discuss.elastic.co/u/pavan1991)\ **Replies:** 3\ **Last updated:** [August 27, 2018, 11:02am UTC](https://discuss.elastic.co/t/ignore-error-messages/145911 "2018-08-27T11:02:19Z")
Team, While analyzing system log files in Linux, I came up with the below entry. How do we ignore such huge blocks in Logstash ?? I believe if we do not ignore this, it will be considered as a log and we will get inco… --- ## [SQLRecoverableException: No more data to read from socket](https://discuss.elastic.co/t/sqlrecoverableexception-no-more-data-to-read-from-socket/146158)
**Author:** [@seungtae](https://discuss.elastic.co/u/seungtae)\ **Replies:** 0\ **Last updated:** [August 27, 2018, 10:45am UTC](https://discuss.elastic.co/t/sqlrecoverableexception-no-more-data-to-read-from-socket/146158 "2018-08-27T10:45:01Z")
Anyone looks like this error? Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#\ **Author:** [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Replies:** 5\ **Last updated:** [August 27, 2018, 10:19am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error-creating-index/146141 "2018-08-27T10:19:20Z") I am trying to parse my logs and trying to create a dynamic index. But somehow logstash is itself creating the index with a , in the pattern hence its failing. Can you please help me remove the comma. filter{ grok{ mat… --- ## [Data is not proper post transformation](https://discuss.elastic.co/t/data-is-not-proper-post-transformation/144716)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 17\ **Last updated:** [August 27, 2018, 10:02am UTC](https://discuss.elastic.co/t/data-is-not-proper-post-transformation/144716 "2018-08-27T10:02:35Z")
Hi All, I'm pulling data from service now through logstash, i have converted few fields like created on, resolved at and closed at, to convert into date values using filter option. Lets say there are a total of 100 tic… --- ## [RUBY Logstash Remove field without suffix](https://discuss.elastic.co/t/ruby-logstash-remove-field-without-suffix/146132)
**Author:** [@Arnodl](https://discuss.elastic.co/u/Arnodl)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 9:34am UTC](https://discuss.elastic.co/t/ruby-logstash-remove-field-without-suffix/146132 "2018-08-27T09:34:48Z")
Hey. I need to write a filter in the logstash which deletes all fields except Hostname and Username and all fields with suffix and \_key, for example platform\_txt --- ## [3 grok filter in same file](https://discuss.elastic.co/t/3-grok-filter-in-same-file/145228)
**Author:** [@Ludovic9](https://discuss.elastic.co/u/Ludovic9)\ **Replies:** 15\ **Last updated:** [August 27, 2018, 9:31am UTC](https://discuss.elastic.co/t/3-grok-filter-in-same-file/145228 "2018-08-27T09:31:26Z")
Hi, I have installed filebeat on a windows server to send logs of filezilla server to logstash. At this moment, it works, I see my logs. But it seems, only one filter works. Here my configuration grok { matc… --- ## [How configure pipeline to process DLQ](https://discuss.elastic.co/t/how-configure-pipeline-to-process-dlq/146140)
**Author:** [@dorinand](https://discuss.elastic.co/u/dorinand)\ **Replies:** 0\ **Last updated:** [August 27, 2018, 9:30am UTC](https://discuss.elastic.co/t/how-configure-pipeline-to-process-dlq/146140 "2018-08-27T09:30:21Z")
I have default-pipeline configured and I would like to have pipeline to process events from DLQ. I have configured two pipelines: - pipeline.id: default-pipeline path.config: "/etc/logstash/conf.d/default-pipeline.co… --- ## [\[INFO \]\[logstash.filters.multiline\] Grok loading patterns from file](https://discuss.elastic.co/t/info-logstash-filters-multiline-grok-loading-patterns-from-file/146115)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 2\ **Last updated:** [August 27, 2018, 8:44am UTC](https://discuss.elastic.co/t/info-logstash-filters-multiline-grok-loading-patterns-from-file/146115 "2018-08-27T08:44:59Z")
when this message is coming, does it take more cpu ? --- ## [How can we create auto alias of index in logstash?](https://discuss.elastic.co/t/how-can-we-create-auto-alias-of-index-in-logstash/146113)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 6:38am UTC](https://discuss.elastic.co/t/how-can-we-create-auto-alias-of-index-in-logstash/146113 "2018-08-27T06:38:30Z")
please help me . --- ## [How to monitor LOGSTASH logs in Kibana](https://discuss.elastic.co/t/how-to-monitor-logstash-logs-in-kibana/145909)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [August 27, 2018, 6:01am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-logs-in-kibana/145909 "2018-08-27T06:01:35Z")
I am using HTTP poller as input and HTTP poller as output. Using get for input and post for output. some time records are failure to load in HTTP output. This failure and reason for failure I am able to see in logstas… --- ## [Redis plugin crash, bug?](https://discuss.elastic.co/t/redis-plugin-crash-bug/146107)
**Author:** [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Replies:** 0\ **Last updated:** [August 27, 2018, 6:00am UTC](https://discuss.elastic.co/t/redis-plugin-crash-bug/146107 "2018-08-27T06:00:38Z")
Hi All, Last night on of the redis servers disappeared with the result that the redis input plugin stopped working. Now this is a expected behavior. However Looking at my log I found the plugin trying to restart over an… --- ## [Collecting multiline message from http input](https://discuss.elastic.co/t/collecting-multiline-message-from-http-input/146019)
**Author:** [@frankshad](https://discuss.elastic.co/u/frankshad)\ **Replies:** 1\ **Last updated:** [August 26, 2018, 2:06pm UTC](https://discuss.elastic.co/t/collecting-multiline-message-from-http-input/146019 "2018-08-26T14:06:47Z")
I'm experimenting with logstash to collect the output of various updates of things (for example, letsencrypt daily certificate renewals) on multiple servers using the http input. These are almost all multiple line text i… --- ## [Approximate match between two fields](https://discuss.elastic.co/t/approximate-match-between-two-fields/145709)
**Author:** [@scch](https://discuss.elastic.co/u/scch)\ **Replies:** 3\ **Last updated:** [August 26, 2018, 9:33am UTC](https://discuss.elastic.co/t/approximate-match-between-two-fields/145709 "2018-08-26T09:33:47Z")
is there a way of approximate match in Logstash, between two fields in a document. what i want to achieve is.. if, sender and recipient name is matching even approximately(30%..) add field "sender\_id" = personal --- ## [\[SOLVED\] Grok or not Grok on SophosXG](https://discuss.elastic.co/t/solved-grok-or-not-grok-on-sophosxg/145896)
**Author:** [@tardis](https://discuss.elastic.co/u/tardis)\ **Replies:** 4\ **Last updated:** [August 26, 2018, 8:02am UTC](https://discuss.elastic.co/t/solved-grok-or-not-grok-on-sophosxg/145896 "2018-08-26T08:02:35Z")
Hi, apologies but I'm struggling to find a good reference document with examples, and I'm basing this on test and fail from a pattern test tool. I'm trying to parse a SophosXG log message, as this one below: device="S… --- ## [Securing Logstash Communication without CA](https://discuss.elastic.co/t/securing-logstash-communication-without-ca/146045)
**Author:** [@lucasjkr](https://discuss.elastic.co/u/lucasjkr)\ **Replies:** 0\ **Last updated:** [August 25, 2018, 7:24pm UTC](https://discuss.elastic.co/t/securing-logstash-communication-without-ca/146045 "2018-08-25T19:24:12Z")
I'm trying to figure out how to secure communication with Logstash without a CA? Can't I just generate a key pair on the Logstash instance, SCP the public cert it to each of my servers that are feeding into it, and tell… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=694) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=696)