# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=275 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 276 --- ## [Logstash date format - Converting timestamp without timezone to timetamp with timezone](https://discuss.elastic.co/t/logstash-date-format-converting-timestamp-without-timezone-to-timetamp-with-timezone/257426)
**Author:** [@jasoninmel](https://discuss.elastic.co/u/jasoninmel)\ **Replies:** 2\ **Last updated:** [December 3, 2020, 12:32am UTC](https://discuss.elastic.co/t/logstash-date-format-converting-timestamp-without-timezone-to-timetamp-with-timezone/257426 "2020-12-03T00:32:24Z")
I got an event timestamp field without timezone stored in Elasticsearch as follows; "open-date": "2016-05-28T00:00:00" This time is in Australia/Melbourne timezone (AEDT/AEST). I used the following filter to covert … --- ## [Logstash Java filter plugin example "./gradlew gem" error](https://discuss.elastic.co/t/logstash-java-filter-plugin-example-gradlew-gem-error/257110)
**Author:** [@lakechat](https://discuss.elastic.co/u/lakechat)\ **Replies:** 4\ **Last updated:** [December 3, 2020, 12:18am UTC](https://discuss.elastic.co/t/logstash-java-filter-plugin-example-gradlew-gem-error/257110 "2020-12-03T00:18:57Z")
I was following Elasticsearch document (https://www.elastic.co/guide/en/logstash/current/java-filter-plugin.html) to write a Java filter plugin for Logtash. Somehow, I could not even get the provided example work. I stuc… --- ## [Error: Cannot allocate memory](https://discuss.elastic.co/t/error-cannot-allocate-memory/257423)
**Author:** [@manunc](https://discuss.elastic.co/u/manunc)\ **Replies:** 2\ **Last updated:** [December 2, 2020, 11:06pm UTC](https://discuss.elastic.co/t/error-cannot-allocate-memory/257423 "2020-12-02T23:06:23Z")
Hello, I am faced to an issue since a long time without finding the solution. I have a dedicated server with 12 Gb RAM but I can only allocate to Logstash 2Gb otherwise the process fall into an error. It seems linked t… --- ## [Cannot connect locally run Logstash to influxdb in docker container](https://discuss.elastic.co/t/cannot-connect-locally-run-logstash-to-influxdb-in-docker-container/257280)
**Author:** [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\ **Replies:** 0\ **Last updated:** [December 1, 2020, 9:52pm UTC](https://discuss.elastic.co/t/cannot-connect-locally-run-logstash-to-influxdb-in-docker-container/257280 "2020-12-01T21:52:05Z")
Hello, I'm running Influxdb in docker container and trying to connect from logstash which is running locally. I've enabled the influxdb-logstash plugin to connect to influxdb output { stdout { codec =\> rubydebug … --- ## [How to extract data from filename using logstash?](https://discuss.elastic.co/t/how-to-extract-data-from-filename-using-logstash/257371)
**Author:** [@reillye](https://discuss.elastic.co/u/reillye)\ **Replies:** 2\ **Last updated:** [December 2, 2020, 4:46pm UTC](https://discuss.elastic.co/t/how-to-extract-data-from-filename-using-logstash/257371 "2020-12-02T16:46:30Z")
I am using filebeat to monitor a directory for new files that get dropped in every hour. The files are then sent to logstash to be parsed before being sent to ES. The filenames are in this format aldb-E6-20201015232941.… --- ## [LogStash Not Receiving Logs](https://discuss.elastic.co/t/logstash-not-receiving-logs/257370)
**Author:** [@darkmatter](https://discuss.elastic.co/u/darkmatter)\ **Replies:** 0\ **Last updated:** [December 2, 2020, 2:48pm UTC](https://discuss.elastic.co/t/logstash-not-receiving-logs/257370 "2020-12-02T14:48:45Z")
I'm looking for a little help troubleshooting a LogStash Docker issues. I'm using Docker ELK and having trouble with Logstash receiving any data and more specifically syslog data. I've confirmed that the host is receivi… --- ## [Logstash pipelines - 2 simple questions](https://discuss.elastic.co/t/logstash-pipelines-2-simple-questions/257306)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 1\ **Last updated:** [December 2, 2020, 1:12pm UTC](https://discuss.elastic.co/t/logstash-pipelines-2-simple-questions/257306 "2020-12-02T13:12:25Z")
Hi there, I have 2 simple questions regarding logstash pipelines: Is it possible in a pipeline - if I made a change and need this change a filter e.g. to take effect - to restart only this single pipeline.id? Or can I… --- ## [Postgresql queries in ELK](https://discuss.elastic.co/t/postgresql-queries-in-elk/256371)
**Author:** [@Charakterny](https://discuss.elastic.co/u/Charakterny)\ **Replies:** 7\ **Last updated:** [December 2, 2020, 12:43pm UTC](https://discuss.elastic.co/t/postgresql-queries-in-elk/256371 "2020-12-02T12:43:23Z")
Hi, I installed metricbeat to provide metrics from postgresql and it works, but main goal was to analize queries in elastic and build dashboard for custom queries. As I started to configure, any statements from databas… --- ## [Logstash splitting and tagging events](https://discuss.elastic.co/t/logstash-splitting-and-tagging-events/256571)
**Author:** [@John\_Smith](https://discuss.elastic.co/u/John_Smith)\ **Replies:** 7\ **Last updated:** [December 2, 2020, 12:02pm UTC](https://discuss.elastic.co/t/logstash-splitting-and-tagging-events/256571 "2020-12-02T12:02:15Z")
I need to ingest following json as separated events { "test1": {some nested json here}, "test2": {some nested json here}, "test3": {some nested json here}, "test4": {some nested json here} } I have 3 problems: When … --- ## [Is there a good way to run benchmark to logstash on GKE?](https://discuss.elastic.co/t/is-there-a-good-way-to-run-benchmark-to-logstash-on-gke/257350)
**Author:** [@iooi](https://discuss.elastic.co/u/iooi)\ **Replies:** 0\ **Last updated:** [December 2, 2020, 11:37am UTC](https://discuss.elastic.co/t/is-there-a-good-way-to-run-benchmark-to-logstash-on-gke/257350 "2020-12-02T11:37:50Z")
Here is an official tool but seems old: https://github.com/elastic/logstash-benchmark-tools Google provides a way but for security: https://cloud.google.com/blog/products/containers-kubernetes/gke-cis-benchmarks-deliver… --- ## [Split a string to array/list, greedy match](https://discuss.elastic.co/t/split-a-string-to-array-list-greedy-match/255305)
**Author:** [@blukit](https://discuss.elastic.co/u/blukit)\ **Replies:** 2\ **Last updated:** [December 2, 2020, 10:58am UTC](https://discuss.elastic.co/t/split-a-string-to-array-list-greedy-match/255305 "2020-12-02T10:58:11Z")
Hi all... I have a message field that contains multiline string. Need to split it to an array for each line, because will do some logic based on a certain lines. But so far only able to catch the first line. A simplifi… --- ## [Recent data not being indexed](https://discuss.elastic.co/t/recent-data-not-being-indexed/257343)
**Author:** [@jafri6](https://discuss.elastic.co/u/jafri6)\ **Replies:** 0\ **Last updated:** [December 2, 2020, 10:45am UTC](https://discuss.elastic.co/t/recent-data-not-being-indexed/257343 "2020-12-02T10:45:22Z")
My ELK system has been running for over a week now. All components - beats, logstash, ES and Kibana are up and running. I suddenly stopped receiving data for the past 2 days. The log files exist but I don't see those in… --- ## [Aggregation filter half work](https://discuss.elastic.co/t/aggregation-filter-half-work/257038)
**Author:** [@Andex](https://discuss.elastic.co/u/Andex)\ **Replies:** 3\ **Last updated:** [December 2, 2020, 9:54am UTC](https://discuss.elastic.co/t/aggregation-filter-half-work/257038 "2020-12-02T09:54:19Z")
Hi, i need to aggregate 6 log file , i have 6 grok parser, each for one of my logs. This is my conf filter { if \[type\] == "application\_log" { grok { match =\> \[ "message", "%{DATA:jcaption\_id}\\s+%{TIME:orari… --- ## [How to run logstash config file automatically after system restarts?](https://discuss.elastic.co/t/how-to-run-logstash-config-file-automatically-after-system-restarts/257177)
**Author:** [@Rahul\_Chougule](https://discuss.elastic.co/u/Rahul_Chougule)\ **Replies:** 4\ **Last updated:** [December 2, 2020, 7:26am UTC](https://discuss.elastic.co/t/how-to-run-logstash-config-file-automatically-after-system-restarts/257177 "2020-12-02T07:26:37Z")
I want to load/run my logstash config file automatically after system restarts everytime (after updates). How to do that? Thanks in advance. --- ## [Not properly formated XML](https://discuss.elastic.co/t/not-properly-formated-xml/257003)
**Author:** [@shani](https://discuss.elastic.co/u/shani)\ **Replies:** 0\ **Last updated:** [November 29, 2020, 7:20pm UTC](https://discuss.elastic.co/t/not-properly-formated-xml/257003 "2020-11-29T19:20:22Z")
Hi All, I have raw event logs like this \\\ **Author:** [@robderon](https://discuss.elastic.co/u/robderon)\ **Replies:** 1\ **Last updated:** [December 2, 2020, 12:48am UTC](https://discuss.elastic.co/t/best-way-to-ingest-data-from-windows-c-app/257093 "2020-12-02T00:48:31Z") Hello, I must create a Kibana dashboard to analyze a windows c++ application usage : screen viewed, actions, app launching, subscription etc.. Our plan is to send data from the clients to our server (not elastic) once … --- ## [Logstash exits after successfully installing plugin with docker-compose](https://discuss.elastic.co/t/logstash-exits-after-successfully-installing-plugin-with-docker-compose/255456)
**Author:** [@luka.klaric](https://discuss.elastic.co/u/luka.klaric)\ **Replies:** 2\ **Last updated:** [December 1, 2020, 9:50pm UTC](https://discuss.elastic.co/t/logstash-exits-after-successfully-installing-plugin-with-docker-compose/255456 "2020-12-01T21:50:42Z")
I am trying to run an Elastic stack and I am trying to install a logstash plugin within the docker file. I am getting the message that the plugin is installed successful and right after that the container exits with code… --- ## [Multiple logstash output](https://discuss.elastic.co/t/multiple-logstash-output/257272)
**Author:** [@stijnvr](https://discuss.elastic.co/u/stijnvr)\ **Replies:** 1\ **Last updated:** [December 1, 2020, 9:09pm UTC](https://discuss.elastic.co/t/multiple-logstash-output/257272 "2020-12-01T21:09:30Z")
I would like to forward all source logs to Logstash. Logstash should have 2 outputs. Elasticsearch, for formatted logs S3, for untampered logs Is this possible ? Or do I need to configuring 2 outputs for the source l… --- ## [Doubts about Grok](https://discuss.elastic.co/t/doubts-about-grok/257129)
**Author:** [@rildo](https://discuss.elastic.co/u/rildo)\ **Replies:** 2\ **Last updated:** [December 1, 2020, 8:37pm UTC](https://discuss.elastic.co/t/doubts-about-grok/257129 "2020-12-01T20:37:15Z")
Hello People , I have a doubt about my filter I have this content in my log file {"other\_logs": "127.0.0.1|28614|2018-11-28 23:51:57|srcport 50389 mwtype AvalancheBotnet-andromeda destaddr 2.2.2.2 destinyurl: dogs.ru|M… --- ## [Multiline data to correlate into single line](https://discuss.elastic.co/t/multiline-data-to-correlate-into-single-line/257253)
**Author:** [@Ramesh535](https://discuss.elastic.co/u/Ramesh535)\ **Replies:** 1\ **Last updated:** [December 1, 2020, 6:38pm UTC](https://discuss.elastic.co/t/multiline-data-to-correlate-into-single-line/257253 "2020-12-01T18:38:58Z")
Hi Guys, Need some help I have the data in this format \[2020-11-27\] \[data\] \[country:India\] \[Capital:Delhi\] \[2020-11-27\] \[data\] \[country:India\] \[Currency:INR\] \[2020-11-27\] \[data\] \[country:India\] \[PM:Modi\] \[2020-11-2… --- ## [Grok parse failures](https://discuss.elastic.co/t/grok-parse-failures/257230)
**Author:** [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\ **Replies:** 3\ **Last updated:** [December 1, 2020, 4:12pm UTC](https://discuss.elastic.co/t/grok-parse-failures/257230 "2020-12-01T16:12:21Z")
Hello I'm playing around with grok filters and i'm running into parse failures..Any idea whats wrong ? Filters: filter { if \[type\] == "app-data" { mutate { rename =\> \["env", "environment"\] } grok { … --- ## [How can I drop preflight http requst( OPTIONS request)in Logstash](https://discuss.elastic.co/t/how-can-i-drop-preflight-http-requst-options-request-in-logstash/257235)
**Author:** [@abdoegy83](https://discuss.elastic.co/u/abdoegy83)\ **Replies:** 0\ **Last updated:** [December 1, 2020, 4:09pm UTC](https://discuss.elastic.co/t/how-can-i-drop-preflight-http-requst-options-request-in-logstash/257235 "2020-12-01T16:09:51Z")
how can I drop preflight HTTP request( OPTIONS request)in Logstash, how can I get access to the HTTP method (post,get,...) --- ## [Logstash Grok](https://discuss.elastic.co/t/logstash-grok/256682)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 3\ **Last updated:** [December 1, 2020, 3:51pm UTC](https://discuss.elastic.co/t/logstash-grok/256682 "2020-12-01T15:51:45Z")
I need to start a grok at \<190\> I dont care about anything before that . What is the way to do this ? \<13\>Nov 25 15:36:31 servername.abc.com LOGSTASH\[-\]: \<190\>Nov 25 07:36:31 --- ## [No geopoint for destination.ip but latitude and longitude field - 7.10](https://discuss.elastic.co/t/no-geopoint-for-destination-ip-but-latitude-and-longitude-field-7-10/256905)
**Author:** [@Camille](https://discuss.elastic.co/u/Camille)\ **Replies:** 3\ **Last updated:** [December 1, 2020, 3:23pm UTC](https://discuss.elastic.co/t/no-geopoint-for-destination-ip-but-latitude-and-longitude-field-7-10/256905 "2020-12-01T15:23:53Z")
Hi, i'm trying to use "geoip" module on a destination.ip. My logstash code is : if \[destination.ip\] { geoip { source =\> "destination.ip" target =\> "destination.geo" } … --- ## [Logstash starts then stops after enabling tcp output config](https://discuss.elastic.co/t/logstash-starts-then-stops-after-enabling-tcp-output-config/257224)
**Author:** [@gprimm](https://discuss.elastic.co/u/gprimm)\ **Replies:** 0\ **Last updated:** [December 1, 2020, 3:11pm UTC](https://discuss.elastic.co/t/logstash-starts-then-stops-after-enabling-tcp-output-config/257224 "2020-12-01T15:11:46Z")
I am trying to send logs to an address listening on a tcp port, but when I add the lines to an output config and restart the logstash service logstash will start for a minute and then fail. There are no errors anywhere a… --- ## [\[Pipeline to Pipeline communication\] - passing pipeline id dinamically to send\_to =\>](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-passing-pipeline-id-dinamically-to-send-to/257222)
**Author:** [@Flavio1](https://discuss.elastic.co/u/Flavio1)\ **Replies:** 1\ **Last updated:** [December 1, 2020, 3:08pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-passing-pipeline-id-dinamically-to-send-to/257222 "2020-12-01T15:08:30Z")
Hi everybody, does exist a method to pass the value of pipeline id dinamically in send\_to. in my environment every log has a field with a value corresponding to a pipeline id present in my pipelines.yml. A hoped that wit… --- ## [Logstash not restarting/stoping](https://discuss.elastic.co/t/logstash-not-restarting-stoping/257153)
**Author:** [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Replies:** 1\ **Last updated:** [December 1, 2020, 2:55pm UTC](https://discuss.elastic.co/t/logstash-not-restarting-stoping/257153 "2020-12-01T14:55:12Z")
Logstash is not restarting/stoping? How do we stop/restart? OS is CentOS 8 Logstash version using : 7.10 ( on previous versions also it is not stoping). \[root@INHY-PAPP-ELK01 conf.d\]# \[root@INHY-PAPP-ELK01 conf.d\]# … --- ## [Truncate filter adds tag even if no fileds were truncated](https://discuss.elastic.co/t/truncate-filter-adds-tag-even-if-no-fileds-were-truncated/257179)
**Author:** [@sarabande](https://discuss.elastic.co/u/sarabande)\ **Replies:** 1\ **Last updated:** [December 1, 2020, 2:34pm UTC](https://discuss.elastic.co/t/truncate-filter-adds-tag-even-if-no-fileds-were-truncated/257179 "2020-12-01T14:34:40Z")
Hi, I'm using truncate filter to truncate messages longer than 32766 bytes. In that case I'd like the tag 'truncated\_message' to be added to the message. This is how I use truncate filter in my logstash configuration: … --- ## [Logstash parse csv file with a column that contains multiple comma separated data points?](https://discuss.elastic.co/t/logstash-parse-csv-file-with-a-column-that-contains-multiple-comma-separated-data-points/256874)
**Author:** [@reillye](https://discuss.elastic.co/u/reillye)\ **Replies:** 7\ **Last updated:** [December 1, 2020, 2:33pm UTC](https://discuss.elastic.co/t/logstash-parse-csv-file-with-a-column-that-contains-multiple-comma-separated-data-points/256874 "2020-12-01T14:33:10Z")
I have a CSV file with multiple entries in the following format (the headers are not included in the actual data): Time, FT Data (2048 entries), SNR, Frequency 430201865, 1000, 2000, 8500,... 4.50, 1266.255 2… --- ## [Message Parsing](https://discuss.elastic.co/t/message-parsing/256572)
**Author:** [@droidus](https://discuss.elastic.co/u/droidus)\ **Replies:** 5\ **Last updated:** [December 1, 2020, 1:32pm UTC](https://discuss.elastic.co/t/message-parsing/256572 "2020-12-01T13:32:25Z")
Here is what my logstash output looks like: output { elasticsearch { index =\> "%{\[@metadata\]\[beat\]}" hosts =\> "192.168.0.103" } } Logstash errors are as follows: \[2020-11-18T13:08:12,824\]\[WARN \]\[logstash.c… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=274) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=276)