# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=167 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 168 --- ## [Logstash OSS 7.16.3 and Elasticsearch OSS 7.10.2](https://discuss.elastic.co/t/logstash-oss-7-16-3-and-elasticsearch-oss-7-10-2/294658)
**Author:** [@Thilak1](https://discuss.elastic.co/u/Thilak1)\ **Replies:** 2\ **Last updated:** [January 18, 2022, 4:00pm UTC](https://discuss.elastic.co/t/logstash-oss-7-16-3-and-elasticsearch-oss-7-10-2/294658 "2022-01-18T16:00:34Z")
Hi Team , I am getting the below error when using Elasticsearch output plugin in logstash OSS 7.16.3 \[2021-04-16T05:31:15,846\]\[ERROR\]\[logstash.javapipeline \] Pipeline error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 5\ **Last updated:** [January 18, 2022, 3:15pm UTC](https://discuss.elastic.co/t/conditional-in-the-output-is-ignored/294343 "2022-01-18T15:15:22Z") The conditional statement in my logstash configuration is being ignored. It is being skipped over no matter what I put in it. I have 4 elasticsearch options in the output divided by an if and else statement. I've tried u… --- ## [K8s (kubernetes) sidecar considerations](https://discuss.elastic.co/t/k8s-kubernetes-sidecar-considerations/294393)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 5\ **Last updated:** [January 18, 2022, 2:29pm UTC](https://discuss.elastic.co/t/k8s-kubernetes-sidecar-considerations/294393 "2022-01-18T14:29:05Z")
Hi, At this moment we have a myriad of microservices running on-prem. However, we are preparing to move everything to tha cloud (k8s on aws). As a result I am redesigning our elastic platform since this will move to aws… --- ## [Unable to use filters with grok expression](https://discuss.elastic.co/t/unable-to-use-filters-with-grok-expression/294662)
**Author:** [@mohammed.sabil](https://discuss.elastic.co/u/mohammed.sabil)\ **Replies:** 0\ **Last updated:** [January 18, 2022, 8:02am UTC](https://discuss.elastic.co/t/unable-to-use-filters-with-grok-expression/294662 "2022-01-18T08:02:04Z")
Hello Team, I need to use grok expression for ELK to filter the message data I have elk configured using docker compose and filebeat on client server using docker container. I have following files configured for filte… --- ## [I'm unable to create fields and unable to parse the below logs with split filter please suggest me the correct logstash input](https://discuss.elastic.co/t/im-unable-to-create-fields-and-unable-to-parse-the-below-logs-with-split-filter-please-suggest-me-the-correct-logstash-input/294682)
**Author:** [@aravindpatel](https://discuss.elastic.co/u/aravindpatel)\ **Replies:** 0\ **Last updated:** [January 18, 2022, 10:45am UTC](https://discuss.elastic.co/t/im-unable-to-create-fields-and-unable-to-parse-the-below-logs-with-split-filter-please-suggest-me-the-correct-logstash-input/294682 "2022-01-18T10:45:29Z")
log file { "id": "AAA", "num": "-17.3595", "num": "-145.494", "name": "Anaa", "author": "Anaaauthor", "place": "Tuamotu-Gambier", "landmark": "French Polynesia", "spcid": "12512819", "target": "Pacific/Midway", … --- ## [Cant install jdbc plugin](https://discuss.elastic.co/t/cant-install-jdbc-plugin/294665)
**Author:** [@peledev](https://discuss.elastic.co/u/peledev)\ **Replies:** 0\ **Last updated:** [January 18, 2022, 8:33am UTC](https://discuss.elastic.co/t/cant-install-jdbc-plugin/294665 "2022-01-18T08:33:12Z")
hi i cant install plugin in my server is get error : \[claudede@server logstash\]$ sudo bin/logstash-plugin install logstash-input-jdbc Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option … --- ## [How to fix \[logstash.filters.geoip.databasemanager\] Connection reset](https://discuss.elastic.co/t/how-to-fix-logstash-filters-geoip-databasemanager-connection-reset/294237)
**Author:** [@Frank\_Kuo](https://discuss.elastic.co/u/Frank_Kuo)\ **Replies:** 2\ **Last updated:** [January 18, 2022, 7:03am UTC](https://discuss.elastic.co/t/how-to-fix-logstash-filters-geoip-databasemanager-connection-reset/294237 "2022-01-18T07:03:21Z")
hi guys, I meet the issue and I don't know why? Does anybody know the reason? Jan 14 10:23:18 OANWELKL1 logstash\[4874\]: \[2022-01-14T10:23:18,644\]\[WARN \]\[org.logstash.instrument.metrics.gauge.LazyDelegatingGauge\]\[main\] … --- ## [Issue of Configuring in Logstash](https://discuss.elastic.co/t/issue-of-configuring-in-logstash/294387)
**Author:** [@leyleynewt](https://discuss.elastic.co/u/leyleynewt)\ **Replies:** 7\ **Last updated:** [January 18, 2022, 2:16am UTC](https://discuss.elastic.co/t/issue-of-configuring-in-logstash/294387 "2022-01-18T02:16:07Z")
This is the configuration in logstash-sample.conf: input{ stdin{} } output{ elasticsearch{ hosts =\> \["localhost:9200"\] index =\> "my-index-name" user =\> "elastic" passwo… --- ## [Error installing Logstash on Linux Server](https://discuss.elastic.co/t/error-installing-logstash-on-linux-server/294621)
**Author:** [@uday22](https://discuss.elastic.co/u/uday22)\ **Replies:** 0\ **Last updated:** [January 17, 2022, 6:40pm UTC](https://discuss.elastic.co/t/error-installing-logstash-on-linux-server/294621 "2022-01-17T18:40:28Z")
OS: Linux i used rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch command to start installation process i am getting errors curl: (7) Failed connect to artifacts.elastic.co:443; connection timed out er… --- ## [Aggregate filter plugin output in a single document - Logstash](https://discuss.elastic.co/t/aggregate-filter-plugin-output-in-a-single-document-logstash/294606)
**Author:** [@Eduard\_Abril](https://discuss.elastic.co/u/Eduard_Abril)\ **Replies:** 1\ **Last updated:** [January 17, 2022, 4:28pm UTC](https://discuss.elastic.co/t/aggregate-filter-plugin-output-in-a-single-document-logstash/294606 "2022-01-17T16:28:21Z")
Hi guys, I'm using an aggregate filter plugin to map certain information to the main document that I'm processing with logstash but I don't understand why at the end I'm getting 2 documents instead of 1, the first of the… --- ## [Output conditions with logstash plugin http\_poller](https://discuss.elastic.co/t/output-conditions-with-logstash-plugin-http-poller/294608)
**Author:** [@redaER7](https://discuss.elastic.co/u/redaER7)\ **Replies:** 0\ **Last updated:** [January 17, 2022, 4:26pm UTC](https://discuss.elastic.co/t/output-conditions-with-logstash-plugin-http-poller/294608 "2022-01-17T16:26:49Z")
I would like to implement a conditional output to Elasticsearch index using logstash http\_poller input plugin. I am using a GET request and sending the response to the index only if the response is not already in the ind… --- ## [After Upgrade Logstash is running but not sending logs to elasticsearch](https://discuss.elastic.co/t/after-upgrade-logstash-is-running-but-not-sending-logs-to-elasticsearch/294324)
**Author:** [@Othmane\_CHHAIBI](https://discuss.elastic.co/u/Othmane_CHHAIBI)\ **Replies:** 7\ **Last updated:** [January 17, 2022, 4:19pm UTC](https://discuss.elastic.co/t/after-upgrade-logstash-is-running-but-not-sending-logs-to-elasticsearch/294324 "2022-01-17T16:19:15Z")
hello everyone, After upgrading logstash from version 6.8 to 7.16.2, logstash is running but it didn't send logs to Elasticsearch. In the logfile of logstash server i have this error : "\[2022-01-03Tl3:50:23,910)\[FATAL)… --- ## [How to change "message" value in index](https://discuss.elastic.co/t/how-to-change-message-value-in-index/294548)
**Author:** [@mhzr](https://discuss.elastic.co/u/mhzr)\ **Replies:** 1\ **Last updated:** [January 17, 2022, 3:38pm UTC](https://discuss.elastic.co/t/how-to-change-message-value-in-index/294548 "2022-01-17T15:38:45Z")
Hello, In logstash pipeline or indexpattern how to change the following part of CDN log in "message" field to seperate or extract some data then aggrigate them. original "message" value: \<40\> 2022-01-17T08:31:22Z logs… --- ## [Date incorrectly parsed](https://discuss.elastic.co/t/date-incorrectly-parsed/294571)
**Author:** [@nino](https://discuss.elastic.co/u/nino)\ **Replies:** 4\ **Last updated:** [January 17, 2022, 2:03pm UTC](https://discuss.elastic.co/t/date-incorrectly-parsed/294571 "2022-01-17T14:03:09Z")
Hello, i can't get this date parsed correctly 20220113T21:00:18.965901 date { match =\> \["fill\_date", "yyyMMdd'T'HH:mm:ss.SSSSSS"\] target =\> "@timestamp" } Best --- ## [Logstash parse strings before a valid json](https://discuss.elastic.co/t/logstash-parse-strings-before-a-valid-json/294578)
**Author:** [@Calin\_Tatar](https://discuss.elastic.co/u/Calin_Tatar)\ **Replies:** 0\ **Last updated:** [January 17, 2022, 11:59am UTC](https://discuss.elastic.co/t/logstash-parse-strings-before-a-valid-json/294578 "2022-01-17T11:59:58Z")
Hello, i have the following log : Received stock info for v2, payload: {'id': '1234dads', 'sku': '1QA0QWQ-1JFQQQ'} and i would like to add a new field with phase before json ,i tried the following config but is now worki… --- ## [How to run multiple instances in docker container?](https://discuss.elastic.co/t/how-to-run-multiple-instances-in-docker-container/294390)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 4\ **Last updated:** [January 17, 2022, 10:18am UTC](https://discuss.elastic.co/t/how-to-run-multiple-instances-in-docker-container/294390 "2022-01-17T10:18:48Z")
I want to run multiple config files in docker container. pipelines.yml file : - pipeline.id: main path.config: "/usr/share/logstash/config/conf.d/\*.conf" - pipeline.id: i2c\_req\_res path.config: "/usr/share/logstas… --- ## [Date filter in logstash](https://discuss.elastic.co/t/date-filter-in-logstash/294431)
**Author:** [@MKH](https://discuss.elastic.co/u/MKH)\ **Replies:** 6\ **Last updated:** [January 16, 2022, 7:16pm UTC](https://discuss.elastic.co/t/date-filter-in-logstash/294431 "2022-01-16T19:16:06Z")
Hi, I am trying to use Date filter of the logstash to create a timestamp on my data stored in Elasticsearch. The timestamp in my data is in Linux format and I want to change it to some standard format like as "dd MMM yy… --- ## [How to check if a field has any subfields](https://discuss.elastic.co/t/how-to-check-if-a-field-has-any-subfields/294482)
**Author:** [@dantamsdb](https://discuss.elastic.co/u/dantamsdb)\ **Replies:** 1\ **Last updated:** [January 15, 2022, 5:03pm UTC](https://discuss.elastic.co/t/how-to-check-if-a-field-has-any-subfields/294482 "2022-01-15T17:03:08Z")
I get input that sometimes looks like this: { "baz": { "foo": "bar", "data": {} } } sometimes it looks like this: { "baz": { "foo": "bar", "data": { "fizz": "buzz", "fizzy": "buzzy" … --- ## [How to transform data through ruby code](https://discuss.elastic.co/t/how-to-transform-data-through-ruby-code/293913)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 12\ **Last updated:** [January 15, 2022, 1:10pm UTC](https://discuss.elastic.co/t/how-to-transform-data-through-ruby-code/293913 "2022-01-15T13:10:07Z")
Hi I need a help for extract data from one of filed from CSV. the header for these data presents like below: EPS\_BEARER\_ID-BEARER\_QCI-ARP\_PL-ARP\_PCI-ARP\_PVI-GBR\_UL-GBR\_DL-BEARER\_CAUSE-DEFAULT\_BEARER\_ID 5-7-undefin… --- ## [Logstash config change suggestion](https://discuss.elastic.co/t/logstash-config-change-suggestion/294466)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 0\ **Last updated:** [January 15, 2022, 4:08am UTC](https://discuss.elastic.co/t/logstash-config-change-suggestion/294466 "2022-01-15T04:08:40Z")
Currently, if you run multiple pipelines, you have to list them in pipelines.yml. In some cases, it would be easier to have a pipelines.d directory with individual files for each pipeline instead of the single pipelines… --- ## [Logstash CSV output opening/closing file bottleneck](https://discuss.elastic.co/t/logstash-csv-output-opening-closing-file-bottleneck/294446)
**Author:** [@tkirui](https://discuss.elastic.co/u/tkirui)\ **Replies:** 0\ **Last updated:** [January 14, 2022, 8:41pm UTC](https://discuss.elastic.co/t/logstash-csv-output-opening-closing-file-bottleneck/294446 "2022-01-14T20:41:48Z")
I am running logstash to parse many files with output to a single CSV file. This is working ok but is taking long with the bottleneck being the opening file/closing file. Im i missing something? Any faster way of doin th… --- ## [Read timed out when using Elasticsearch filter in pipeline](https://discuss.elastic.co/t/read-timed-out-when-using-elasticsearch-filter-in-pipeline/294438)
**Author:** [@SamW](https://discuss.elastic.co/u/SamW)\ **Replies:** 0\ **Last updated:** [January 14, 2022, 5:29pm UTC](https://discuss.elastic.co/t/read-timed-out-when-using-elasticsearch-filter-in-pipeline/294438 "2022-01-14T17:29:28Z")
As part of my project's workflow, we have a pipeline which uses the Elasticsearch filter plugin a few times in order to find a set of items which need to be deleted from several indices. This always works perfectly when … --- ## [Logstash Issue: Got response code '401' contacting Elasticsearch at URL 'http://localhost:9200/'](https://discuss.elastic.co/t/logstash-issue-got-response-code-401-contacting-elasticsearch-at-url-http-localhost-9200/294395)
**Author:** [@Abhishek\_Gautam](https://discuss.elastic.co/u/Abhishek_Gautam)\ **Replies:** 1\ **Last updated:** [January 14, 2022, 5:08pm UTC](https://discuss.elastic.co/t/logstash-issue-got-response-code-401-contacting-elasticsearch-at-url-http-localhost-9200/294395 "2022-01-14T17:08:34Z")
while trying to run logstash 7.16.3 with this current version of Elasticsearch 8.1.0 . I get the following error. {:url=\>"http://localhost:9200/", :exception=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResp… --- ## [Best method to get 5 fields from inputs and duplicate in index2](https://discuss.elastic.co/t/best-method-to-get-5-fields-from-inputs-and-duplicate-in-index2/294429)
**Author:** [@aaleman](https://discuss.elastic.co/u/aaleman)\ **Replies:** 1\ **Last updated:** [January 14, 2022, 5:06pm UTC](https://discuss.elastic.co/t/best-method-to-get-5-fields-from-inputs-and-duplicate-in-index2/294429 "2022-01-14T17:06:32Z")
Hello, I was reading a lot because I want to now: What is the best way that masters recommend to next case?: I have index1 with a lot of fields I need this data into another index to have history, but this index2 jus… --- ## [Logstash sql\_last\_value not compare correctly](https://discuss.elastic.co/t/logstash-sql-last-value-not-compare-correctly/294398)
**Author:** [@vikram\_singh](https://discuss.elastic.co/u/vikram_singh)\ **Replies:** 1\ **Last updated:** [January 14, 2022, 3:02pm UTC](https://discuss.elastic.co/t/logstash-sql-last-value-not-compare-correctly/294398 "2022-01-14T15:02:29Z")
Hi, I am using logstash to insert data from sql to index. I am using a date column compare\_date to compare for sql\_last\_value field. When I am running, logstash automatically reduce 1 day from compare\_date. Suppose … --- ## [Jdbc integration plugin output documentation](https://discuss.elastic.co/t/jdbc-integration-plugin-output-documentation/294421)
**Author:** [@patrikpihlstrom](https://discuss.elastic.co/u/patrikpihlstrom)\ **Replies:** 1\ **Last updated:** [January 14, 2022, 2:27pm UTC](https://discuss.elastic.co/t/jdbc-integration-plugin-output-documentation/294421 "2022-01-14T14:27:07Z")
I've been using the jdbc integration plugin to insert records in my mariadb server for some time now. However, as requirements have changed, I've been tasked with finding a solution to acknowledge messages after they've … --- ## [Remove fields - plugin http\_poller\_plugin with json](https://discuss.elastic.co/t/remove-fields-plugin-http-poller-plugin-with-json/294415)
**Author:** [@guilhermealano](https://discuss.elastic.co/u/guilhermealano)\ **Replies:** 1\ **Last updated:** [January 14, 2022, 1:28pm UTC](https://discuss.elastic.co/t/remove-fields-plugin-http-poller-plugin-with-json/294415 "2022-01-14T13:28:09Z")
Hi, I'm making a request in logstash through http\_poller\_plugin and the data that return in full in json. I'm trying to make a filter to display in the json only the necessary fields. What would be the correct way to c… --- ## [Logstash Configuration File](https://discuss.elastic.co/t/logstash-configuration-file/294266)
**Author:** [@\_Thomas](https://discuss.elastic.co/u/_Thomas)\ **Replies:** 7\ **Last updated:** [January 14, 2022, 12:27pm UTC](https://discuss.elastic.co/t/logstash-configuration-file/294266 "2022-01-14T12:27:54Z")
Hi Guys, first of all, I'm sorry in case I'm asking stupid questions - I'm pretty new to this ELK Topic, and I'm trying to get my head around it. At the moment I'm working on a Logstash config file, that should import … --- ## [Syncing SQL data with Elastic](https://discuss.elastic.co/t/syncing-sql-data-with-elastic/294372)
**Author:** [@Satyan\_Gupta](https://discuss.elastic.co/u/Satyan_Gupta)\ **Replies:** 0\ **Last updated:** [January 14, 2022, 7:27am UTC](https://discuss.elastic.co/t/syncing-sql-data-with-elastic/294372 "2022-01-14T07:27:02Z")
Hello Everyone, I am new to elastic and facing issue to keep my sql database in sync with elastic. Here is my logstash config file. input { jdbc { jdbc\_driver\_library =\> "C:\\Users\\satyang.sw\\Desktop\\ELK\_Stack\\log… --- ## [Logstash User Survey](https://discuss.elastic.co/t/logstash-user-survey/294248)
**Author:** [@katja1](https://discuss.elastic.co/u/katja1)\ **Replies:** 2\ **Last updated:** [January 14, 2022, 6:48am UTC](https://discuss.elastic.co/t/logstash-user-survey/294248 "2022-01-14T06:48:19Z")
Hi, I know it's been a while ago, but I ran into the post about Logstash user survey Logstash User Survey Results | Elastic Blog recently and I found the blog really useful, since it also includes the link to the raw fi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=166) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=168)