# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=629 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 630 --- ## [Logstash output influxdb problem](https://discuss.elastic.co/t/logstash-output-influxdb-problem/161509)
**Author:** [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Replies:** 0\ **Last updated:** [December 19, 2018, 12:18pm UTC](https://discuss.elastic.co/t/logstash-output-influxdb-problem/161509 "2018-12-19T12:18:22Z")
input { beats { port =\> 5044 ssl =\> true type =\> "json" ssl\_certificate =\> "/etc/pki/tls/certs/abc.crt" ssl\_key =\> "/etc/pki/tls/private/abc.key" } } filter { if \[type\] == "wineventlog" { ruby { code =\> " event.get('… --- ## [Location to geo\_point mapping not working with logtash and kibana](https://discuss.elastic.co/t/location-to-geo-point-mapping-not-working-with-logtash-and-kibana/160980)
**Author:** [@pravinnair](https://discuss.elastic.co/u/pravinnair)\ **Replies:** 14\ **Last updated:** [December 19, 2018, 12:10pm UTC](https://discuss.elastic.co/t/location-to-geo-point-mapping-not-working-with-logtash-and-kibana/160980 "2018-12-19T12:10:06Z")
Hello, I am trying to plot a group of store locations on a map, i have zip and lat and long of the store. version being used is 6.4.2 I am covert the location feild to geo\_point using logstash output for elastic and s… --- ## [Sflow support](https://discuss.elastic.co/t/sflow-support/161506)
**Author:** [@craigmcfly](https://discuss.elastic.co/u/craigmcfly)\ **Replies:** 0\ **Last updated:** [December 19, 2018, 11:59am UTC](https://discuss.elastic.co/t/sflow-support/161506 "2018-12-19T11:59:20Z")
Can someone clarify the current support for sflow in Logstash? It's been removed from the list of current codecs on the documentation, but it still installs. Specifically, I'm trying to work out why the AS Router/Source… --- ## [CEF Codec issues](https://discuss.elastic.co/t/cef-codec-issues/160695)
**Author:** [@QwerFact](https://discuss.elastic.co/u/QwerFact)\ **Replies:** 1\ **Last updated:** [December 19, 2018, 11:18am UTC](https://discuss.elastic.co/t/cef-codec-issues/160695 "2018-12-19T11:18:14Z")
Hello, I have some Warning and Error with the CEF codec on input pipeline. We have some special character (like French accent é à è) not correctly decoded by the default charset UTF-8. The CEF codec don't accept the c… --- ## [Create a custom id from a a database columns](https://discuss.elastic.co/t/create-a-custom-id-from-a-a-database-columns/161350)
**Author:** [@tunigas](https://discuss.elastic.co/u/tunigas)\ **Replies:** 8\ **Last updated:** [December 19, 2018, 10:21am UTC](https://discuss.elastic.co/t/create-a-custom-id-from-a-a-database-columns/161350 "2018-12-19T10:21:01Z")
Hello everyone, I'm new to logstash. Currently, I'm trying to create a custom document\_id from the concatenation of two database columns. My sql query inside the jdbc is as follows: statement =\> "SELECT TOP 10 name, a… --- ## [Logstash. Version Conflict](https://discuss.elastic.co/t/logstash-version-conflict/161485)
**Author:** [@shakir](https://discuss.elastic.co/u/shakir)\ **Replies:** 0\ **Last updated:** [December 19, 2018, 10:09am UTC](https://discuss.elastic.co/t/logstash-version-conflict/161485 "2018-12-19T10:09:05Z")
Version\_conflict\_engine\_exception. Version conflict, current version \[64\] is different than the one provided \[63\] I am trying to update document using logstash, getting below error. Tried retry\_on\_conflict with value … --- ## [Can not re-use field value for calculating new value](https://discuss.elastic.co/t/can-not-re-use-field-value-for-calculating-new-value/160931)
**Author:** [@Hryhorii\_Koval](https://discuss.elastic.co/u/Hryhorii_Koval)\ **Replies:** 2\ **Last updated:** [December 19, 2018, 10:07am UTC](https://discuss.elastic.co/t/can-not-re-use-field-value-for-calculating-new-value/160931 "2018-12-19T10:07:40Z")
Depends on some condition I want to re-calculate field value. Please, give me a hand. ruby { code =\> "event.set('my\_field', event.get('my\_field')\*1024)" } --- ## [Logstash - update or delete records after fetching in db2luw work table](https://discuss.elastic.co/t/logstash-update-or-delete-records-after-fetching-in-db2luw-work-table/161417)
**Author:** [@Florian\_B1](https://discuss.elastic.co/u/Florian_B1)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 9:35pm UTC](https://discuss.elastic.co/t/logstash-update-or-delete-records-after-fetching-in-db2luw-work-table/161417 "2018-12-18T21:35:49Z")
We have following set up: data for elasticsearch is offered in a work table. There we get the records with insert/update/delete and fetch these records via logstash. Thats working. But how can we delete/update the proce… --- ## [Logstash generate timely csv output](https://discuss.elastic.co/t/logstash-generate-timely-csv-output/158471)
**Author:** [@Manoj\_Hettiarachchi](https://discuss.elastic.co/u/Manoj_Hettiarachchi)\ **Replies:** 1\ **Last updated:** [December 19, 2018, 5:48am UTC](https://discuss.elastic.co/t/logstash-generate-timely-csv-output/158471 "2018-12-19T05:48:45Z")
Hello, In my system, I use the csv output plugin. The output block is as follows. output { csv { path =\> "/var/csv\_reports/%{+YYYY}-%{+MM}-%{+dd}/transaction-report.csv" fields =\> \["timest… --- ## [How to use aggregate filter with multiple workers](https://discuss.elastic.co/t/how-to-use-aggregate-filter-with-multiple-workers/157621)
**Author:** [@Manoj\_Hettiarachchi](https://discuss.elastic.co/u/Manoj_Hettiarachchi)\ **Replies:** 3\ **Last updated:** [December 19, 2018, 5:36am UTC](https://discuss.elastic.co/t/how-to-use-aggregate-filter-with-multiple-workers/157621 "2018-12-19T05:36:54Z")
I am using logstash aggregate filter to aggregate two log lines with same "uuid" But in the documentation, it is mentioned that " You should be very careful to set Logstash filter workers to 1 ( -w 1 flag) for this fil… --- ## [Logstash save/ modify configuration in environment](https://discuss.elastic.co/t/logstash-save-modify-configuration-in-environment/159464)
**Author:** [@Manoj\_Hettiarachchi](https://discuss.elastic.co/u/Manoj_Hettiarachchi)\ **Replies:** 2\ **Last updated:** [December 19, 2018, 5:32am UTC](https://discuss.elastic.co/t/logstash-save-modify-configuration-in-environment/159464 "2018-12-19T05:32:35Z")
Hello, In my system, I use logstash, filebeat and elasticsearch Filebeat reads the logs, required fields in the logs are filtered with logstash and saved in elasticsearch. I have a customer requirement to switch on/of… --- ## [Logstash configuration issue in VPN server](https://discuss.elastic.co/t/logstash-configuration-issue-in-vpn-server/161017)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 1\ **Last updated:** [December 19, 2018, 5:29am UTC](https://discuss.elastic.co/t/logstash-configuration-issue-in-vpn-server/161017 "2018-12-19T05:29:20Z")
I have a project in (A.B.C.D server). it is fetching data from third-party API. Now I want to store this data to my ES which is on a different server(P.L.M.N server). ELK server is connected with VPN ( openVPN ) and its … --- ## [Remove % from the field](https://discuss.elastic.co/t/remove-from-the-field/161441)
**Author:** [@Sridhar](https://discuss.elastic.co/u/Sridhar)\ **Replies:** 0\ **Last updated:** [December 19, 2018, 5:13am UTC](https://discuss.elastic.co/t/remove-from-the-field/161441 "2018-12-19T05:13:55Z")
Hi.. Below is the json output { "\_index": "itrs-source-mantas2018-12-344", "\_type": "doc", "\_id": "AWeZyKeDpSGftotd3dLO", "\_version": 1, "\_score": null, "\_source": { "operation": "update", "@version": "1", "da… --- ## [Logstash delete double quotes and parsing](https://discuss.elastic.co/t/logstash-delete-double-quotes-and-parsing/160698)
**Author:** [@Denis\_Kazinachikov](https://discuss.elastic.co/u/Denis_Kazinachikov)\ **Replies:** 0\ **Last updated:** [December 13, 2018, 9:37am UTC](https://discuss.elastic.co/t/logstash-delete-double-quotes-and-parsing/160698 "2018-12-13T09:37:59Z")
I made a grok filter for the string: 2018-12-10T16:35:50+06:00 2018 ZyWall src="192.168.0.202:57092" dst="173.194.32.197:443" msg="Match default rule, DROP" note="ACCESS BLOCK" user="unknown" devID="04bf6d1fb9a1" cat="S… --- ## [What happens if ES has a index issue?](https://discuss.elastic.co/t/what-happens-if-es-has-a-index-issue/161099)
**Author:** [@rsasportes](https://discuss.elastic.co/u/rsasportes)\ **Replies:** 1\ **Last updated:** [December 19, 2018, 1:03am UTC](https://discuss.elastic.co/t/what-happens-if-es-has-a-index-issue/161099 "2018-12-19T01:03:42Z")
Hi, I'm a newbie with ES, and I try hardening it in order to avoid failure. Last week, I encountered the famous issue where ES locks indexes when it reaches less than 5% of disk storage. I've deleted old indexes, put t… --- ## [CSV Date Filter not working](https://discuss.elastic.co/t/csv-date-filter-not-working/161218)
**Author:** [@kb2295](https://discuss.elastic.co/u/kb2295)\ **Replies:** 2\ **Last updated:** [December 18, 2018, 9:45pm UTC](https://discuss.elastic.co/t/csv-date-filter-not-working/161218 "2018-12-18T21:45:35Z")
Hello. I have a CSV file that looks a bit like this: Date,Year,Month,Client,Execution Account,Source,SharedBook,I/E/H,FirmId,TraderLogin,Trader,Region,City,Exchange,ProductName,AssetClass,CTM,P&L,CFOXType,TradeType,Han… --- ## [Daily index rotation bug](https://discuss.elastic.co/t/daily-index-rotation-bug/161220)
**Author:** [@justinw](https://discuss.elastic.co/u/justinw)\ **Replies:** 2\ **Last updated:** [December 18, 2018, 9:29pm UTC](https://discuss.elastic.co/t/daily-index-rotation-bug/161220 "2018-12-18T21:29:34Z")
Hi, Every now and then a few documents (less than 1% of total logs) get placed into a daily index where the 20 in 2018 is replaced by 00. For example, logstash-6.4.3-0018.12.08 Has anyone encountered this? Seems like… --- ## [Multiline log - How to parse?](https://discuss.elastic.co/t/multiline-log-how-to-parse/161414)
**Author:** [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 9:09pm UTC](https://discuss.elastic.co/t/multiline-log-how-to-parse/161414 "2018-12-18T21:09:28Z")
Have several beats now working, but running up against this trainwreck of a log. My understanding is that since I have several beats going I need to "transform" the log on the filebeat side before it gets shipped out? H… --- ## [Logstash http poller question](https://discuss.elastic.co/t/logstash-http-poller-question/161409)
**Author:** [@Ryan\_Ivis](https://discuss.elastic.co/u/Ryan_Ivis)\ **Replies:** 1\ **Last updated:** [December 18, 2018, 8:41pm UTC](https://discuss.elastic.co/t/logstash-http-poller-question/161409 "2018-12-18T20:41:41Z")
Curious if anybody here has used the logstash input http poller to reach out and pull info from google api's. I have generated a API project as well as enabled a few api's and generated credentials for those API's which… --- ## [Logstash 6.5.1 Ruby filter how to Modify nested json array](https://discuss.elastic.co/t/logstash-6-5-1-ruby-filter-how-to-modify-nested-json-array/161389)
**Author:** [@alon.eldi](https://discuss.elastic.co/u/alon.eldi)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 5:59pm UTC](https://discuss.elastic.co/t/logstash-6-5-1-ruby-filter-how-to-modify-nested-json-array/161389 "2018-12-18T17:59:12Z")
Hi , I have like following json {"id":"5", "name":"John Barns", "address":"Medison road", "Message": {"Header": {"Item": \[{"Field\_Name":"Status", "Field\_Value":"Ok"}, {"Field\_Name":"Type", "Field\_Value":"Short… --- ## [Optional fields](https://discuss.elastic.co/t/optional-fields/160827)
**Author:** [@glennharm](https://discuss.elastic.co/u/glennharm)\ **Replies:** 2\ **Last updated:** [December 18, 2018, 5:57pm UTC](https://discuss.elastic.co/t/optional-fields/160827 "2018-12-18T17:57:58Z")
I've looked at several posts regarding optional fields in grok match filter and so far I've been unsuccessful in getting it to work using the parenthesis (with and without ? in various places, depending on what was advis… --- ## [Empty configuration for pipeline\_id](https://discuss.elastic.co/t/empty-configuration-for-pipeline-id/160968)
**Author:** [@perryparktung](https://discuss.elastic.co/u/perryparktung)\ **Replies:** 1\ **Last updated:** [December 18, 2018, 4:55pm UTC](https://discuss.elastic.co/t/empty-configuration-for-pipeline-id/160968 "2018-12-18T16:55:21Z")
Hi, I am trying to configure for centralized pipeline management but failed. Would you please help check why? Thanks.. pipeline.yml - pipeline.id: mainpipe path.config: "/etc/logstash/conf.d/\*.conf" ll path.confi… --- ## [Get Logstash PID](https://discuss.elastic.co/t/get-logstash-pid/161339)
**Author:** [@Elango-mas](https://discuss.elastic.co/u/Elango-mas)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 1:56pm UTC](https://discuss.elastic.co/t/get-logstash-pid/161339 "2018-12-18T13:56:02Z")
Hi, I trying to get the process id of the logstash service running on my server. What to do? Thanks in Advance!! Elango --- ## [Translate v3.2.3 on LS 6.5.0: does not work anymore with .csv file](https://discuss.elastic.co/t/translate-v3-2-3-on-ls-6-5-0-does-not-work-anymore-with-csv-file/159161)
**Author:** [@Michel99\_7](https://discuss.elastic.co/u/Michel99_7)\ **Replies:** 2\ **Last updated:** [December 14, 2018, 9:07am UTC](https://discuss.elastic.co/t/translate-v3-2-3-on-ls-6-5-0-does-not-work-anymore-with-csv-file/159161 "2018-12-14T09:07:42Z")
Dear all, I'm trying to migrate a pipeline from LS 6.3.2 to 6.5.0 but there's a problem during its execution due to an error on the translate filter (used with a .csv file): "Unquoted fields do not allow \\r or \\n". \[20… --- ## [Trouble connecting Logstash to secure cluster](https://discuss.elastic.co/t/trouble-connecting-logstash-to-secure-cluster/161337)
**Author:** [@bigdamhero](https://discuss.elastic.co/u/bigdamhero)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 1:41pm UTC](https://discuss.elastic.co/t/trouble-connecting-logstash-to-secure-cluster/161337 "2018-12-18T13:41:28Z")
Logstash version 6.5.2, openjdk. Trying to get my logstash to connect with a secured cluster. Recieving this peskey error which I cannot seem to locate. \[2018-12-18T07:24:30,909\]\[ERROR\]\[logstash.pipeline \] Err… --- ## [Logstash 6.5.2 does not start with openjdk11](https://discuss.elastic.co/t/logstash-6-5-2-does-not-start-with-openjdk11/160334)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 2\ **Last updated:** [December 18, 2018, 12:57pm UTC](https://discuss.elastic.co/t/logstash-6-5-2-does-not-start-with-openjdk11/160334 "2018-12-18T12:57:06Z")
Hello, Since it was announced that ELK stack has support of Java 11 since 6.5.0 release I thought that I could start it on openjdk11. When starting I can see in the logs the following error: Dec 10 17:18:28 mgt-wazuh-0… --- ## [Multiple keys one value : translate filter](https://discuss.elastic.co/t/multiple-keys-one-value-translate-filter/161309)
**Author:** [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 11:02am UTC](https://discuss.elastic.co/t/multiple-keys-one-value-translate-filter/161309 "2018-12-18T11:02:33Z")
Hi i have tried to combined multiple keys and one value in different formats with no success. eg. ip,host,true I need a translate filter for dropping events when message contains a specific ip and host but logstash ou… --- ## [Pipelines.yml RemoteConfigError - Where should pipeline configs reside?](https://discuss.elastic.co/t/pipelines-yml-remoteconfigerror-where-should-pipeline-configs-reside/161298)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 9:55am UTC](https://discuss.elastic.co/t/pipelines-yml-remoteconfigerror-where-should-pipeline-configs-reside/161298 "2018-12-18T09:55:28Z")
We're trying to use the multiple pipelines feature described here in Logstash/ES 6.5 on Windows but LogStash can apparently not see/find/read the .conf files. We're starting logstash.bat with no parameters and our pipel… --- ## [Unzip with ruby](https://discuss.elastic.co/t/unzip-with-ruby/161296)
**Author:** [@00a45f9a00ca005db54c](https://discuss.elastic.co/u/00a45f9a00ca005db54c)\ **Replies:** 0\ **Last updated:** [December 18, 2018, 9:49am UTC](https://discuss.elastic.co/t/unzip-with-ruby/161296 "2018-12-18T09:49:38Z")
I have flow python gzip script -\> rabbitMQ -\> logstash -\> ES If i subscribe to RMQ directly with python or ruby script - i can catch the message and decode it easily. But when it comes to logstash - ruby says that messa… --- ## [DELETE all data feom logstash](https://discuss.elastic.co/t/delete-all-data-feom-logstash/161284)
**Author:** [@saeed](https://discuss.elastic.co/u/saeed)\ **Replies:** 3\ **Last updated:** [December 18, 2018, 8:56am UTC](https://discuss.elastic.co/t/delete-all-data-feom-logstash/161284 "2018-12-18T08:56:34Z")
Hi How can i delete data from an elasticsearch index (just data, not setting) using logstash? I want to reindex data from SQLServer. If GET my\_index/\_count is \> 0 then delete all of them and reload that index from my … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=628) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=630)