# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=323 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 324 --- ## [Merge date and time field](https://discuss.elastic.co/t/merge-date-and-time-field/242862)
**Author:** [@Bigranawab](https://discuss.elastic.co/u/Bigranawab)\ **Replies:** 4\ **Last updated:** [July 28, 2020, 10:07am UTC](https://discuss.elastic.co/t/merge-date-and-time-field/242862 "2020-07-28T10:07:34Z")
So, I have been trying to parse fortigate logs using logstash, I came across date and time fields, in fortigate there are two different fields, I tried to parse those fileds using mutate {add\_field =\> { "@timestamp" =\> … --- ## [Parse multi line json](https://discuss.elastic.co/t/parse-multi-line-json/242677)
**Author:** [@laxmikanth](https://discuss.elastic.co/u/laxmikanth)\ **Replies:** 11\ **Last updated:** [July 28, 2020, 9:57am UTC](https://discuss.elastic.co/t/parse-multi-line-json/242677 "2020-07-28T09:57:40Z")
Hi, When I try to parse multi line json as below, I am seeing in ELK as multiline, \_jsonparsefailure, \_grokparsefailure from below json file content, want to remove KEY4 & host and send rest of the fields to elastic se… --- ## [Logstash - Retry with exponential backoff code=\>400,](https://discuss.elastic.co/t/logstash-retry-with-exponential-backoff-code-400/242849)
**Author:** [@GraceYu](https://discuss.elastic.co/u/GraceYu)\ **Replies:** 0\ **Last updated:** [July 28, 2020, 7:34am UTC](https://discuss.elastic.co/t/logstash-retry-with-exponential-backoff-code-400/242849 "2020-07-28T07:34:15Z")
Hi, I have started logstash with the following error: \[2020-07-28T15:19:51,466\]\[ERROR\]\[logstash.outputs.elasticsearch\]\[main\]\[ac34975042de783740a7fba68d15a74cca99c92e561d5fc2f692e54892613f0d\] Encountered a retryable err… --- ## [Logstash jdbc sqlite file in readonly directory](https://discuss.elastic.co/t/logstash-jdbc-sqlite-file-in-readonly-directory/242838)
**Author:** [@KoettingSimon](https://discuss.elastic.co/u/KoettingSimon)\ **Replies:** 0\ **Last updated:** [July 28, 2020, 6:34am UTC](https://discuss.elastic.co/t/logstash-jdbc-sqlite-file-in-readonly-directory/242838 "2020-07-28T06:34:34Z")
Hi all, I need your help for a Logstash-Pipeline. We have an third party application which stores is logs in a sqlite .db File (Windows Server). We want to integrate these Logs in our Stack. What we done so far: Mou… --- ## [Logstash not connecting to elasticsearch after enabling xpack.security on single.node](https://discuss.elastic.co/t/logstash-not-connecting-to-elasticsearch-after-enabling-xpack-security-on-single-node/242741)
**Author:** [@Bhanu\_Praveen](https://discuss.elastic.co/u/Bhanu_Praveen)\ **Replies:** 3\ **Last updated:** [July 28, 2020, 6:05am UTC](https://discuss.elastic.co/t/logstash-not-connecting-to-elasticsearch-after-enabling-xpack-security-on-single-node/242741 "2020-07-28T06:05:07Z")
Hello, I have configured elastic stack on Docker with below docker-compose on single node model. Initially disabled xpack-security and everything working fine. Post that enabled, xpack-security and restarted stack. Log… --- ## [Logstash filter fields in if conditions](https://discuss.elastic.co/t/logstash-filter-fields-in-if-conditions/242717)
**Author:** [@vape](https://discuss.elastic.co/u/vape)\ **Replies:** 4\ **Last updated:** [July 28, 2020, 5:38am UTC](https://discuss.elastic.co/t/logstash-filter-fields-in-if-conditions/242717 "2020-07-28T05:38:26Z")
Hello, I am learning about logstash, esspecially about logstash filter with if condition. Something not clear to me is what are those fields used in if condition? How can I get the list of those fields? For example, I … --- ## [Index logstash-\* blocked by TOO\_MANY\_REQUESTS/12/index read-only](https://discuss.elastic.co/t/index-logstash-blocked-by-too-many-requests-12-index-read-only/242690)
**Author:** [@Mozhganch](https://discuss.elastic.co/u/Mozhganch)\ **Replies:** 4\ **Last updated:** [July 28, 2020, 5:16am UTC](https://discuss.elastic.co/t/index-logstash-blocked-by-too-many-requests-12-index-read-only/242690 "2020-07-28T05:16:42Z")
Hi my logstash service starts without error but it stops with this error "Failed with result 'exit-code'" then I run it without logstash service. so after sending 5 entry point, the following error is occured: \[INFO … --- ## [HTTP\_POLLER plugin failing to connect to the REST API via proxy](https://discuss.elastic.co/t/http-poller-plugin-failing-to-connect-to-the-rest-api-via-proxy/242824)
**Author:** [@Gaurav\_Deoras](https://discuss.elastic.co/u/Gaurav_Deoras)\ **Replies:** 0\ **Last updated:** [July 28, 2020, 4:29am UTC](https://discuss.elastic.co/t/http-poller-plugin-failing-to-connect-to-the-rest-api-via-proxy/242824 "2020-07-28T04:29:08Z")
Hi Team, I'm trying to use logstash to pull the log through rest API and plan to further forward it to the SIEM. The setup is pretty straight forward. SIEM \<== Logstash (server) ==\> Proxy ==\> FW ==\> API Host. The have … --- ## [Logstash multiple outputs](https://discuss.elastic.co/t/logstash-multiple-outputs/238428)
**Author:** [@mycloudcms](https://discuss.elastic.co/u/mycloudcms)\ **Replies:** 5\ **Last updated:** [July 27, 2020, 10:05pm UTC](https://discuss.elastic.co/t/logstash-multiple-outputs/238428 "2020-07-27T22:05:27Z")
Hi, once I try to add multiple output.logstash as array, I don't get any data, can you please assist? Thank you ex. #----------------------------- Logstash output -------------------------------- output.logstash: # T… --- ## [Different config files for different log sources](https://discuss.elastic.co/t/different-config-files-for-different-log-sources/242734)
**Author:** [@Bigranawab](https://discuss.elastic.co/u/Bigranawab)\ **Replies:** 3\ **Last updated:** [July 27, 2020, 9:32pm UTC](https://discuss.elastic.co/t/different-config-files-for-different-log-sources/242734 "2020-07-27T21:32:08Z")
Hi, I am new to ELK, and I have tried to integrate some sources with ELk, I have integrated 'Microsoft DNS logs' and 'Fortigate' using Logstash config. My question here is that what is best practice for logstash config… --- ## [Auto\_flush\_interval not working inside my TCP plugin](https://discuss.elastic.co/t/auto-flush-interval-not-working-inside-my-tcp-plugin/242614)
**Author:** [@dhanshake](https://discuss.elastic.co/u/dhanshake)\ **Replies:** 2\ **Last updated:** [July 27, 2020, 6:12pm UTC](https://discuss.elastic.co/t/auto-flush-interval-not-working-inside-my-tcp-plugin/242614 "2020-07-27T18:12:16Z")
I'm using Multiline codec, along with my TCP input plugin. And the problem I've is logstash is not processing the last line (it waits for the next line always). Tried using auto\_flush\_interval option as well. But didn'… --- ## [Logstash not starting when using "codec =\> multiline"](https://discuss.elastic.co/t/logstash-not-starting-when-using-codec-multiline/242441)
**Author:** [@saurabhc](https://discuss.elastic.co/u/saurabhc)\ **Replies:** 6\ **Last updated:** [July 27, 2020, 5:52pm UTC](https://discuss.elastic.co/t/logstash-not-starting-when-using-codec-multiline/242441 "2020-07-27T17:52:28Z")
My objective is to parse a multiline string. For that, I tried to use “codec =\> multiline”. But whenever I try to start logstash using “codec =\> multiline”, it fails to start. But it starts successfully without the “cod… --- ## [How processing logs with different structure with logstash](https://discuss.elastic.co/t/how-processing-logs-with-different-structure-with-logstash/242401)
**Author:** [@Bilal2](https://discuss.elastic.co/u/Bilal2)\ **Replies:** 2\ **Last updated:** [July 27, 2020, 4:44pm UTC](https://discuss.elastic.co/t/how-processing-logs-with-different-structure-with-logstash/242401 "2020-07-27T16:44:48Z")
Hi, I'm fairly new to logstash; and I want to ship the logs to elasticsearch but I have different structures of logs in my log file: such as: 00000078 1 57118 20200708040053402 6408 2 admin IP-AC1FF2CB.io rsession21.ex… --- ## [SSL Handshake failure](https://discuss.elastic.co/t/ssl-handshake-failure/242771)
**Author:** [@Abdul\_Hameed](https://discuss.elastic.co/u/Abdul_Hameed)\ **Replies:** 0\ **Last updated:** [July 27, 2020, 3:04pm UTC](https://discuss.elastic.co/t/ssl-handshake-failure/242771 "2020-07-27T15:04:10Z")
Hi, I get an error in logstash \[logstash.inputs.tcp \] Error in Netty pipeline: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:100000d7:SSL routines:OPENSSL\_internal:SSL\_HANDSHA… --- ## [How does aggregate with different task\_id in one filter part works](https://discuss.elastic.co/t/how-does-aggregate-with-different-task-id-in-one-filter-part-works/242066)
**Author:** [@Aberry](https://discuss.elastic.co/u/Aberry)\ **Replies:** 1\ **Last updated:** [July 27, 2020, 2:34pm UTC](https://discuss.elastic.co/t/how-does-aggregate-with-different-task-id-in-one-filter-part-works/242066 "2020-07-27T14:34:56Z")
Could anyone help explain how the aggregate with different task\_id in one filter part works? Will the second aggregate walk through again on all existed events? Or any memory limit setting when perform the aggregate? Ta… --- ## [Elastic/logstash puppet forge module config\_dir not being set](https://discuss.elastic.co/t/elastic-logstash-puppet-forge-module-config-dir-not-being-set/242566)
**Author:** [@cgutshal](https://discuss.elastic.co/u/cgutshal)\ **Replies:** 2\ **Last updated:** [July 27, 2020, 2:33pm UTC](https://discuss.elastic.co/t/elastic-logstash-puppet-forge-module-config-dir-not-being-set/242566 "2020-07-27T14:33:18Z")
I am working with the following module in puppet forge: This is my current profile: https://gist.github.com/cgutshal/116d2cfd1bd21b9083088bfbacd1a804 The issue I am running into is that while my pipeline configs ar… --- ## [Http-poller plugin](https://discuss.elastic.co/t/http-poller-plugin/242736)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 0\ **Last updated:** [July 27, 2020, 10:54am UTC](https://discuss.elastic.co/t/http-poller-plugin/242736 "2020-07-27T10:54:01Z")
Hi, I use ELK 7.7.0 I would like to use logstash's http-poller input to consume a webservice returning an array of json documents to load data into elasticsearch. the result return the first page, and the response inc… --- ## [Pagination for REST API Via Logstash](https://discuss.elastic.co/t/pagination-for-rest-api-via-logstash/242714)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 0\ **Last updated:** [July 27, 2020, 8:49am UTC](https://discuss.elastic.co/t/pagination-for-rest-api-via-logstash/242714 "2020-07-27T08:49:23Z")
Hi, How can I import from a REST API to Elasticsearch via Logstash using Pagination? I was able to import using http\_poller, how can I write force a custom script or "for loop" to check for other records to import. C… --- ## [Logstash jdbc sybase connect fails](https://discuss.elastic.co/t/logstash-jdbc-sybase-connect-fails/242217)
**Author:** [@xcitan](https://discuss.elastic.co/u/xcitan)\ **Replies:** 3\ **Last updated:** [July 27, 2020, 8:02am UTC](https://discuss.elastic.co/t/logstash-jdbc-sybase-connect-fails/242217 "2020-07-27T08:02:57Z")
Working on: SMP Debian 4.19.118-2+deb10u1 (2020-06-07) x86\_64 GNU/Linux logstash 7.8.0 I need to connect to a sybase ASE16, using jconn4 driver. The driver for example works in squirrel without any problems. input {… --- ## [Change elastic ip from localhost to x.x.x.x](https://discuss.elastic.co/t/change-elastic-ip-from-localhost-to-x-x-x-x/242534)
**Author:** [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\ **Replies:** 2\ **Last updated:** [July 27, 2020, 7:16am UTC](https://discuss.elastic.co/t/change-elastic-ip-from-localhost-to-x-x-x-x/242534 "2020-07-27T07:16:38Z")
Hi, Elastic search by default running on 127.0.0.1:9200. I change it to some IP address. Question is how I can change or tell logstash to connect with given IP address. Thanks in Advance --- ## [ELK Multi index in signle logstash.conf file](https://discuss.elastic.co/t/elk-multi-index-in-signle-logstash-conf-file/242548)
**Author:** [@jeetugangwar11](https://discuss.elastic.co/u/jeetugangwar11)\ **Replies:** 4\ **Last updated:** [July 27, 2020, 5:21am UTC](https://discuss.elastic.co/t/elk-multi-index-in-signle-logstash-conf-file/242548 "2020-07-27T05:21:45Z")
Hi, I'm trying to push data from logstash to more then 2 index using single logstash.conf file. below is my configuration: input { beats { port =\> 5044 } } input { file { type =\> "java" path =\> "C:/ELK/Projects… --- ## [Add frequency to elatize data logstash](https://discuss.elastic.co/t/add-frequency-to-elatize-data-logstash/242552)
**Author:** [@Gabriel\_Mare](https://discuss.elastic.co/u/Gabriel_Mare)\ **Replies:** 5\ **Last updated:** [July 27, 2020, 7:09am UTC](https://discuss.elastic.co/t/add-frequency-to-elatize-data-logstash/242552 "2020-07-27T07:09:15Z")
Hi, I need to add frequency to elastize data in my elasticsearch. I'm using stdin input and I don't know which the parameter/variable relationated with frequency. Somebody can help me? --- ## [Logstash-forwarder](https://discuss.elastic.co/t/logstash-forwarder/240204)
**Author:** [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)\ **Replies:** 3\ **Last updated:** [July 27, 2020, 7:07am UTC](https://discuss.elastic.co/t/logstash-forwarder/240204 "2020-07-27T07:07:19Z")
I am using logstash forwarder to ship my logs from AIX machine to my ELK stack on linux machine I am trying to make an index as (hostname-ip) but it is not working my config is as below input { lumberjack { port =\> … --- ## [Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of \[A-Za-z0-9\_-\], \[ \\\\t\\\\r\\\\n\]](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline-id-main-exception-logstash-configurationerror-message-expected-one-of-a-za-z0-9-t-r-n/242613)
**Author:** [@abhijeet\_sinha](https://discuss.elastic.co/u/abhijeet_sinha)\ **Replies:** 3\ **Last updated:** [July 26, 2020, 4:40pm UTC](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline-id-main-exception-logstash-configurationerror-message-expected-one-of-a-za-z0-9-t-r-n/242613 "2020-07-26T16:40:59Z")
Hi, I'm trying to set up ELK stack and while setting up pipeline in logstash is gave me error: \[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :ex… --- ## [Using "pg" gem for ruby filter plugin](https://discuss.elastic.co/t/using-pg-gem-for-ruby-filter-plugin/242637)
**Author:** [@Ayan\_Saha](https://discuss.elastic.co/u/Ayan_Saha)\ **Replies:** 0\ **Last updated:** [July 26, 2020, 7:54am UTC](https://discuss.elastic.co/t/using-pg-gem-for-ruby-filter-plugin/242637 "2020-07-26T07:54:49Z")
Hi, I am very new to logstash and facing problem when tried to use 'pg' in ruby filter plugin for sql query. This is my config file input { jdbc { # Postgres jdbc connection string to our datab… --- ## [Trouble with logstash pipeline creating es index](https://discuss.elastic.co/t/trouble-with-logstash-pipeline-creating-es-index/242397)
**Author:** [@jwade](https://discuss.elastic.co/u/jwade)\ **Replies:** 4\ **Last updated:** [July 25, 2020, 11:31am UTC](https://discuss.elastic.co/t/trouble-with-logstash-pipeline-creating-es-index/242397 "2020-07-25T11:31:02Z")
I am attempting to ingest this logfile: ---,36.25, 30.14, 0.01, 0.01, 26.36, 23.92, 23.68 ---,36.25, 30.15, 0.01, 0.01, 26.36, 24.04, 23.68 ---,36.26, 30.14, 0.01, 0.01, 26.36, 24.04, 23.68 ---,36.25,… --- ## [How to filter beat data by version on logstash?](https://discuss.elastic.co/t/how-to-filter-beat-data-by-version-on-logstash/242591)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 2\ **Last updated:** [July 25, 2020, 9:08am UTC](https://discuss.elastic.co/t/how-to-filter-beat-data-by-version-on-logstash/242591 "2020-07-25T09:08:39Z")
How to filter beat data by version on logstash ? I tried code as belw on logstash config , but it did not work. I want to filter out version 6.8.6 data input { beats { port =\> 5044 } } filter { if \[agent.versi… --- ## [Modelling to Elastic Common Schema (ECS): Best practices in logstash](https://discuss.elastic.co/t/modelling-to-elastic-common-schema-ecs-best-practices-in-logstash/242597)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 0\ **Last updated:** [July 25, 2020, 8:22am UTC](https://discuss.elastic.co/t/modelling-to-elastic-common-schema-ecs-best-practices-in-logstash/242597 "2020-07-25T08:22:15Z")
hi Being a big fan of ECS, I understand most of them are built-in for known products and technology if beats does it, but how to tackle it at logstash? My scenario is like Data arrives to data lake via syslog and is … --- ## [Docker-elk\_logstash\_1 exited with code 1 after enabling xpack.security](https://discuss.elastic.co/t/docker-elk-logstash-1-exited-with-code-1-after-enabling-xpack-security/242554)
**Author:** [@Bhanu\_Praveen](https://discuss.elastic.co/u/Bhanu_Praveen)\ **Replies:** 0\ **Last updated:** [July 24, 2020, 6:08pm UTC](https://discuss.elastic.co/t/docker-elk-logstash-1-exited-with-code-1-after-enabling-xpack-security/242554 "2020-07-24T18:08:33Z")
Hello, I set-up docker image in production server with single node. Stack came up and working fine. As a part of next step, enabled "xpack.security.enabled: true" in elasticsearch.yml. Generated random passwords for bui… --- ## [Logstash https plugin not working](https://discuss.elastic.co/t/logstash-https-plugin-not-working/242386)
**Author:** [@rajkumar.m](https://discuss.elastic.co/u/rajkumar.m)\ **Replies:** 9\ **Last updated:** [July 24, 2020, 5:45pm UTC](https://discuss.elastic.co/t/logstash-https-plugin-not-working/242386 "2020-07-24T17:45:29Z")
Hi Team, I am unable to send data with logstash plugin https, someone please help me. I can send data with http without security successfully, but when try to enable ssl, it is failing, please find the below command s… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=322) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=324)