# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=331 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 332 --- ## [URIPATH with other languages failing Grok parse](https://discuss.elastic.co/t/uripath-with-other-languages-failing-grok-parse/240225)
**Author:** [@andrew.campbell](https://discuss.elastic.co/u/andrew.campbell)\ **Replies:** 2\ **Last updated:** [July 10, 2020, 5:13pm UTC](https://discuss.elastic.co/t/uripath-with-other-languages-failing-grok-parse/240225 "2020-07-10T17:13:32Z")
We have some URIs with 8bit characters that are not getting parsed by Grok but are valid URIs. I've validated with the debugger that it is failing at this URIPATH field in our logs. /uk-UA/Properties/бц-«леонардо»-фаза-… --- ## [Transfering data from elasticsearch to mongodb](https://discuss.elastic.co/t/transfering-data-from-elasticsearch-to-mongodb/240689)
**Author:** [@sandeepsingh](https://discuss.elastic.co/u/sandeepsingh)\ **Replies:** 1\ **Last updated:** [July 10, 2020, 4:55pm UTC](https://discuss.elastic.co/t/transfering-data-from-elasticsearch-to-mongodb/240689 "2020-07-10T16:55:17Z")
Hi, I am trying to extract data from elasticsearch to mongodb. I am using the inbuilt elasticsearch input plugin and 3.1.5 mongodb output plugin. I have multiple indexes in elasticsearch whose data needs to be extracted… --- ## [Logstash 7.8.0 Pipeline Authentication on MongoDB 4 Failure](https://discuss.elastic.co/t/logstash-7-8-0-pipeline-authentication-on-mongodb-4-failure/240706)
**Author:** [@leoper](https://discuss.elastic.co/u/leoper)\ **Replies:** 0\ **Last updated:** [July 10, 2020, 1:36pm UTC](https://discuss.elastic.co/t/logstash-7-8-0-pipeline-authentication-on-mongodb-4-failure/240706 "2020-07-10T13:36:10Z")
Hi everyone. I'm trying to read a collection on MongoDB 4 (installed on an AWS machine) via the Logstash 7.8.0 pipeline (installed on a GCP machine). In the pipeline configuration file I tried both to specify the authe… --- ## [Concatenate two columns](https://discuss.elastic.co/t/concatenate-two-columns/240662)
**Author:** [@Jasmin](https://discuss.elastic.co/u/Jasmin)\ **Replies:** 1\ **Last updated:** [July 10, 2020, 9:24am UTC](https://discuss.elastic.co/t/concatenate-two-columns/240662 "2020-07-10T09:24:01Z")
Hi , i want to select only new entries , and i am gonna use last sql value , but in my case i am wokring on shops , so i have two dates , date of creation ( if somebody buy something ) and date of modification ( if som… --- ## [Logstash not forwarding messages to elasticsearch but able to write to file locally](https://discuss.elastic.co/t/logstash-not-forwarding-messages-to-elasticsearch-but-able-to-write-to-file-locally/240111)
**Author:** [@satish92](https://discuss.elastic.co/u/satish92)\ **Replies:** 4\ **Last updated:** [July 10, 2020, 7:29am UTC](https://discuss.elastic.co/t/logstash-not-forwarding-messages-to-elasticsearch-but-able-to-write-to-file-locally/240111 "2020-07-10T07:29:51Z")
Im facing strange issue and im new to elastic stack this is my flow logstash A (input application logs , output kafka topics) =\> kafka (topics , all topics to logstash consumer group) =\> logstash B ( input kafka , outp… --- ## [How to create a new field combining values of other field values](https://discuss.elastic.co/t/how-to-create-a-new-field-combining-values-of-other-field-values/238087)
**Author:** [@Matish\_Bhuyan](https://discuss.elastic.co/u/Matish_Bhuyan)\ **Replies:** 36\ **Last updated:** [July 10, 2020, 5:32am UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-combining-values-of-other-field-values/238087 "2020-07-10T05:32:08Z")
Hi All, I am using elk version 7.2 I have two keywords 1- result 2 - service using the values of these keywords i want to create another field which will met the condition and keep the data according to the conditio… --- ## [Logstash 7.6.2 (EPIPE) Broken pipe Error (pipeline stopped processing new events)](https://discuss.elastic.co/t/logstash-7-6-2-epipe-broken-pipe-error-pipeline-stopped-processing-new-events/238366)
**Author:** [@iscorpio23](https://discuss.elastic.co/u/iscorpio23)\ **Replies:** 3\ **Last updated:** [July 10, 2020, 4:48am UTC](https://discuss.elastic.co/t/logstash-7-6-2-epipe-broken-pipe-error-pipeline-stopped-processing-new-events/238366 "2020-07-10T04:48:22Z")
Hi All, Just upgrade Logstash version from version 6.2.3 to 7.6.2 by RPM upgrade and i have 9 pipeline. Logstash will push log to Elasticsearch cloud version 7.4. After the upgrade, log can be seen in Kibana, but daily … --- ## [How to run or test Logstash pipeline?](https://discuss.elastic.co/t/how-to-run-or-test-logstash-pipeline/240638)
**Author:** [@prabhpahul](https://discuss.elastic.co/u/prabhpahul)\ **Replies:** 0\ **Last updated:** [July 10, 2020, 4:05am UTC](https://discuss.elastic.co/t/how-to-run-or-test-logstash-pipeline/240638 "2020-07-10T04:05:24Z")
I am trying to build a logstash pipeline to pool data from sql server continuously. But I am not able to test the pipeline on the cloud? --- ## [Logstash since\_db =\> "NUL" not forcing reload](https://discuss.elastic.co/t/logstash-since-db-nul-not-forcing-reload/240566)
**Author:** [@noamc](https://discuss.elastic.co/u/noamc)\ **Replies:** 7\ **Last updated:** [July 9, 2020, 10:11pm UTC](https://discuss.elastic.co/t/logstash-since-db-nul-not-forcing-reload/240566 "2020-07-09T22:11:30Z")
Hello all, I've built a ELK stack that loads a CSV on an hourly basis. The CSV is of all our tasks from jira, and so every change in an issue gets updated in the CSV, and needs to be reloaded to Kibana. The stack works… --- ## [Index two parts of document in two different index](https://discuss.elastic.co/t/index-two-parts-of-document-in-two-different-index/240609)
**Author:** [@Bouraoui\_Kacem](https://discuss.elastic.co/u/Bouraoui_Kacem)\ **Replies:** 1\ **Last updated:** [July 9, 2020, 9:53pm UTC](https://discuss.elastic.co/t/index-two-parts-of-document-in-two-different-index/240609 "2020-07-09T21:53:51Z")
I received a document which contains an array of objects , i want to index all the document in index1 and the content of the array in indexTwo , how can i configure that with logstash please with filter and output ? --- ## [Mysterious \_dateparsefailure on identical fields](https://discuss.elastic.co/t/mysterious-dateparsefailure-on-identical-fields/240597)
**Author:** [@pgervais](https://discuss.elastic.co/u/pgervais)\ **Replies:** 6\ **Last updated:** [July 9, 2020, 8:53pm UTC](https://discuss.elastic.co/t/mysterious-dateparsefailure-on-identical-fields/240597 "2020-07-09T20:53:48Z")
I have included below 3 csv input lines that are virtually identical to one another. The date field in the first and last parse properly whereas I get a \_dateparsefailure on the middle line. The logstash date field is s… --- ## [Printing stdout {codec =\> rubydebug} only on error](https://discuss.elastic.co/t/printing-stdout-codec-rubydebug-only-on-error/240598)
**Author:** [@pgervais](https://discuss.elastic.co/u/pgervais)\ **Replies:** 2\ **Last updated:** [July 9, 2020, 8:36pm UTC](https://discuss.elastic.co/t/printing-stdout-codec-rubydebug-only-on-error/240598 "2020-07-09T20:36:16Z")
How can I control the output section of a logstash config to only print the stdout {codec =\> rubydebug} on an error? for example , on a \_dateparsefailure. --- ## [Question about logstash and kafka](https://discuss.elastic.co/t/question-about-logstash-and-kafka/240553)
**Author:** [@Maurizio\_Tarducci](https://discuss.elastic.co/u/Maurizio_Tarducci)\ **Replies:** 12\ **Last updated:** [July 9, 2020, 6:26pm UTC](https://discuss.elastic.co/t/question-about-logstash-and-kafka/240553 "2020-07-09T18:26:23Z")
Hi i would like to configure logstash with kafka in a way that based on the content of the message that logstash collect send to kafka in a different topic : for example if the message contains the string XXX i want send… --- ## [Problem in SSL between logstash and filebeat](https://discuss.elastic.co/t/problem-in-ssl-between-logstash-and-filebeat/240579)
**Author:** [@azteker](https://discuss.elastic.co/u/azteker)\ **Replies:** 1\ **Last updated:** [July 9, 2020, 5:47pm UTC](https://discuss.elastic.co/t/problem-in-ssl-between-logstash-and-filebeat/240579 "2020-07-09T17:47:46Z")
Here is the logstash's error log: Jul 09 17:34:26 elk.novalocal logstash\[15657\]: Caused by: java.security.cert.CertificateException: could not find certificate file: /etc/logstash/certs/logstash.crt Jul 09 17:34:26 elk… --- ## [Receiving remote logs in logstash](https://discuss.elastic.co/t/receiving-remote-logs-in-logstash/240234)
**Author:** [@orlapa](https://discuss.elastic.co/u/orlapa)\ **Replies:** 4\ **Last updated:** [July 9, 2020, 5:36pm UTC](https://discuss.elastic.co/t/receiving-remote-logs-in-logstash/240234 "2020-07-09T17:36:23Z")
I have configured a HSM to send lodgs via TCP to logstash listening in pot 5004, but nothing arrive. I did a test with Rsyslog in port 514 and the logs arrive. Where should I focus the serach for error, the services are … --- ## [Referencing multiple pipelines using env var](https://discuss.elastic.co/t/referencing-multiple-pipelines-using-env-var/240578)
**Author:** [@earl\_potter](https://discuss.elastic.co/u/earl_potter)\ **Replies:** 0\ **Last updated:** [July 9, 2020, 5:36pm UTC](https://discuss.elastic.co/t/referencing-multiple-pipelines-using-env-var/240578 "2020-07-09T17:36:14Z")
Using CPM and I'm currently referencing my pipeline w/the env var XPACK\_MANAGEMENT\_PIPELINE\_ID. Can I reference multiple pipelines by separating them with a comma? export XPACK\_MANAGEMENT\_PIPELINE\_ID=mypipeline01,mypip… --- ## [How to replace Logstash timestamp with URL request timestamp in Elasticsearch?](https://discuss.elastic.co/t/how-to-replace-logstash-timestamp-with-url-request-timestamp-in-elasticsearch/240525)
**Author:** [@Rahul\_Chougule](https://discuss.elastic.co/u/Rahul_Chougule)\ **Replies:** 1\ **Last updated:** [July 9, 2020, 5:01pm UTC](https://discuss.elastic.co/t/how-to-replace-logstash-timestamp-with-url-request-timestamp-in-elasticsearch/240525 "2020-07-09T17:01:56Z")
I have created .conf file where I am ingesting HTTP endpoints output data in Elasticsearch with the help of http\_poller plugin. I want to replace timestamp which is provided by Logstash with the timestamp of at which act… --- ## [Looks like you don't have any metrics indices](https://discuss.elastic.co/t/looks-like-you-dont-have-any-metrics-indices/239933)
**Author:** [@DucretJe](https://discuss.elastic.co/u/DucretJe)\ **Replies:** 2\ **Last updated:** [July 9, 2020, 4:57pm UTC](https://discuss.elastic.co/t/looks-like-you-dont-have-any-metrics-indices/239933 "2020-07-09T16:57:50Z")
Hi there, I'm trying to set up a monitoring system with ELK. I deploy Beats on my infra Beats output to Logstash server Logstash server output to Elasticsearch Server And I want to visualize my data with Kibana. On… --- ## [Unable to connect to database. The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption](https://discuss.elastic.co/t/unable-to-connect-to-database-the-driver-could-not-establish-a-secure-connection-to-sql-server-by-using-secure-sockets-layer-ssl-encryption/240562)
**Author:** [@Andrew\_Foxis](https://discuss.elastic.co/u/Andrew_Foxis)\ **Replies:** 1\ **Last updated:** [July 9, 2020, 4:41pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-database-the-driver-could-not-establish-a-secure-connection-to-sql-server-by-using-secure-sockets-layer-ssl-encryption/240562 "2020-07-09T16:41:34Z")
Hi all! I configure connect to database jdbc and get error( \[2020-07-09T19:07:06,569\]\[ERROR\]\[logstash.inputs.jdbc \] Unable to connect to database. Tried 1 times {:error\_message=\>"Java::ComMicrosoftSqlserverJdbc::SQ… --- ## [Logstash csv loading broken duplicate of last row](https://discuss.elastic.co/t/logstash-csv-loading-broken-duplicate-of-last-row/238295)
**Author:** [@noamc](https://discuss.elastic.co/u/noamc)\ **Replies:** 5\ **Last updated:** [July 9, 2020, 4:18pm UTC](https://discuss.elastic.co/t/logstash-csv-loading-broken-duplicate-of-last-row/238295 "2020-07-09T16:18:34Z")
Hello hello, Quite new to Elastic Stack, and I'm encountering a problem which also experienced colleagues can't figure our the reason for this event happening. Bottom line - my data loads fine, except that almost every… --- ## [Split field values in logstash](https://discuss.elastic.co/t/split-field-values-in-logstash/240285)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 7\ **Last updated:** [July 9, 2020, 3:54pm UTC](https://discuss.elastic.co/t/split-field-values-in-logstash/240285 "2020-07-09T15:54:56Z")
Hi All, I'm ingesting data into elasticsearch using logstash http\_poller plugin. WE have a value which contains the location field like city, country e.g: Location : Chennai, India Is there a possible way to split th… --- ## [Problem with parse json in strings](https://discuss.elastic.co/t/problem-with-parse-json-in-strings/240435)
**Author:** [@Vladimir\_Martyshin](https://discuss.elastic.co/u/Vladimir_Martyshin)\ **Replies:** 1\ **Last updated:** [July 9, 2020, 3:51pm UTC](https://discuss.elastic.co/t/problem-with-parse-json-in-strings/240435 "2020-07-09T15:51:29Z")
Hello! I am trying to parse logs. The log have field "message". When I have message: Connection lost after 18 seconds all ok. But, when I get the log with message: CMD: exit. I get the following error in logstash: o… --- ## [How logstash memory behaves with lot of beats connections?](https://discuss.elastic.co/t/how-logstash-memory-behaves-with-lot-of-beats-connections/240365)
**Author:** [@obuez](https://discuss.elastic.co/u/obuez)\ **Replies:** 7\ **Last updated:** [July 9, 2020, 1:47pm UTC](https://discuss.elastic.co/t/how-logstash-memory-behaves-with-lot-of-beats-connections/240365 "2020-07-09T13:47:19Z")
Hi all, We got some little issues in production around Logstash and we are a bit worried for the future, we would like to know the community's opinion on this. Let me explain: We have ran Logstash in production so far … --- ## [How to generate \_id field manually](https://discuss.elastic.co/t/how-to-generate-id-field-manually/240302)
**Author:** [@onkark](https://discuss.elastic.co/u/onkark)\ **Replies:** 8\ **Last updated:** [July 9, 2020, 12:06pm UTC](https://discuss.elastic.co/t/how-to-generate-id-field-manually/240302 "2020-07-09T12:06:32Z")
I have imported a data of 3.5 crore documents in elasticsearch by using logstash. After I found that the unique field "\_id" is not generated. Now, I want to generate the unique field as "\_id" on all the documents. Can a… --- ## [Logstash Output dosent publishes logs while using systemctl](https://discuss.elastic.co/t/logstash-output-dosent-publishes-logs-while-using-systemctl/240410)
**Author:** [@Rahul\_Ravichandran](https://discuss.elastic.co/u/Rahul_Ravichandran)\ **Replies:** 6\ **Last updated:** [July 9, 2020, 11:55am UTC](https://discuss.elastic.co/t/logstash-output-dosent-publishes-logs-while-using-systemctl/240410 "2020-07-09T11:55:19Z")
If I start the logstash service using service or systemctl, it starts there are no errors started successfully but logs dosen't get published there in kibana. If i use " bin/logstash -f /etc/logstash/conf.d/logsta… --- ## [Error while injecting data from sqlite3 database](https://discuss.elastic.co/t/error-while-injecting-data-from-sqlite3-database/240509)
**Author:** [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Replies:** 5\ **Last updated:** [July 9, 2020, 11:19am UTC](https://discuss.elastic.co/t/error-while-injecting-data-from-sqlite3-database/240509 "2020-07-09T11:19:55Z")
Hi I am trying to create index for sqlite3 database but I am getting error saying Pipeline\_id:main Plugin: \"/home/kagamee/liveshield/api-sqlite/Yogesh/SimpleSocketIO/db/networkdata.db", … --- ## [Useragent plugin - android 9 and 10](https://discuss.elastic.co/t/useragent-plugin-android-9-and-10/239175)
**Author:** [@holiveira](https://discuss.elastic.co/u/holiveira)\ **Replies:** 4\ **Last updated:** [July 9, 2020, 10:49am UTC](https://discuss.elastic.co/t/useragent-plugin-android-9-and-10/239175 "2020-07-09T10:49:32Z")
The parse of an Android 9 or 10 useragent is partial. For example, the fields user\_agentos\_major, user\_agentos\_minor do not exist. Example useragent: Mozilla/5.0 (Linux; Android 9; Redmi Note 7 Build/PKQ1.180904.001; w… --- ## [LogStash is running but Not processing Logs](https://discuss.elastic.co/t/logstash-is-running-but-not-processing-logs/240491)
**Author:** [@singh.piyush862](https://discuss.elastic.co/u/singh.piyush862)\ **Replies:** 0\ **Last updated:** [July 9, 2020, 8:23am UTC](https://discuss.elastic.co/t/logstash-is-running-but-not-processing-logs/240491 "2020-07-09T08:23:36Z")
Hi All, We started Logstash service & it process the syslog data but suddenly it stopped processing logs (I could not see new logs in Kibaan/elastic search index) however Syslog is running. Elastic search is rusnning … --- ## [Need to integrate Logstash HTTP input plugin with my android application](https://discuss.elastic.co/t/need-to-integrate-logstash-http-input-plugin-with-my-android-application/240484)
**Author:** [@sheetalg](https://discuss.elastic.co/u/sheetalg)\ **Replies:** 0\ **Last updated:** [July 9, 2020, 8:01am UTC](https://discuss.elastic.co/t/need-to-integrate-logstash-http-input-plugin-with-my-android-application/240484 "2020-07-09T08:01:25Z")
I need to integrate Logstash HTTP input plugin with my android application so that I can errors logs to logstash server. Please guide me with example or sample android app --- ## [Logstash on Centos 6 with non-default java](https://discuss.elastic.co/t/logstash-on-centos-6-with-non-default-java/240361)
**Author:** [@Nekasas](https://discuss.elastic.co/u/Nekasas)\ **Replies:** 2\ **Last updated:** [July 9, 2020, 8:02am UTC](https://discuss.elastic.co/t/logstash-on-centos-6-with-non-default-java/240361 "2020-07-09T08:02:27Z")
Hello, We have an old Centos 6 server where java 1.7 is crucial and we can't update it to java 1.8 so we are using alternatives. Configure alternative with logstash is quite difficult ( at least for me) Problem is that… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=330) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=332)