# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=342 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 343 --- ## [Problem sincedb\_path](https://discuss.elastic.co/t/problem-sincedb-path/237222)
**Author:** [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\ **Replies:** 1\ **Last updated:** [June 16, 2020, 6:01am UTC](https://discuss.elastic.co/t/problem-sincedb-path/237222 "2020-06-16T06:01:56Z")
Hello, I use logstash to ingest CSV file in elastic, i have a problem with sincedb\_path input { file { path =\> "/data/volumes/monitoring/logstash/logCtrlM/stock.csv" start\_position =\> "beginning" sincedb\_path =\> "/usr… --- ## [Need to Split the Value in the Message](https://discuss.elastic.co/t/need-to-split-the-value-in-the-message/237217)
**Author:** [@Prithviraj\_Tantubay](https://discuss.elastic.co/u/Prithviraj_Tantubay)\ **Replies:** 0\ **Last updated:** [June 16, 2020, 5:02am UTC](https://discuss.elastic.co/t/need-to-split-the-value-in-the-message/237217 "2020-06-16T05:02:54Z")
I need help in splitting the value in message field and update the result in the remaining respective fields. I have a row data in .csv file in the following format: "NA","GIXRG051/Genpact/Consultant","5/1/2020 2:10","… --- ## [Calculating new fields using ruby filter](https://discuss.elastic.co/t/calculating-new-fields-using-ruby-filter/237163)
**Author:** [@inchirah](https://discuss.elastic.co/u/inchirah)\ **Replies:** 2\ **Last updated:** [June 15, 2020, 9:13pm UTC](https://discuss.elastic.co/t/calculating-new-fields-using-ruby-filter/237163 "2020-06-15T21:13:28Z")
Hi evryone i tried o use ruby filter to calculate 3 new fields: The first is the total number of observation in may table. the second id the number of observations where(Contrainte\>=-78),Contrainte is a field in my cs… --- ## [Decode and Add derived/custom fields](https://discuss.elastic.co/t/decode-and-add-derived-custom-fields/237175)
**Author:** [@Mike\_Labman](https://discuss.elastic.co/u/Mike_Labman)\ **Replies:** 1\ **Last updated:** [June 15, 2020, 8:57pm UTC](https://discuss.elastic.co/t/decode-and-add-derived-custom-fields/237175 "2020-06-15T20:57:54Z")
I am brand new to elastic. I have regularly incoming data that has an encoded field, i.e. { id: 1, time: "2020-06-15 13:45:00", encoded: "AAIBdQYmaSUAoPy2Bbx7bP9QX4V3UKCHbYGfxjz" } I have external javascript that I us… --- ## [Getting info from website with docker's logstash image](https://discuss.elastic.co/t/getting-info-from-website-with-dockers-logstash-image/236991)
**Author:** [@brandon1](https://discuss.elastic.co/u/brandon1)\ **Replies:** 4\ **Last updated:** [June 15, 2020, 4:13pm UTC](https://discuss.elastic.co/t/getting-info-from-website-with-dockers-logstash-image/236991 "2020-06-15T16:13:35Z")
I'm new to logstash, and I've been trying to get a docker container working with logstash in order to get data from the stock market for a project. Unfortunately, primarily due to the fact that I have no experience with … --- ## [Logstash Json filter re-formated the @timestamp field unexpectedly](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027)
**Author:** [@STAR\_DEVX](https://discuss.elastic.co/u/STAR_DEVX)\ **Replies:** 5\ **Last updated:** [June 15, 2020, 12:36pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027 "2020-06-15T12:36:13Z")
Hi, I'm encounter an issue with the logstash json plugin. In my "message", there's an field called "@timestamp", it's in UNIX epoch float format. However, when I try to use json filter to parse the "message" field, it … --- ## [How to generate grok pattern using existing elasticsearch index?](https://discuss.elastic.co/t/how-to-generate-grok-pattern-using-existing-elasticsearch-index/237116)
**Author:** [@dattack13](https://discuss.elastic.co/u/dattack13)\ **Replies:** 0\ **Last updated:** [June 15, 2020, 11:07am UTC](https://discuss.elastic.co/t/how-to-generate-grok-pattern-using-existing-elasticsearch-index/237116 "2020-06-15T11:07:54Z")
Hello All, Let me explain my existing structure, I have 4 servers (Web Server, API Server, Database server, SSIS Severs) and installed filebeat and winlog in all four servers and from there I am getting all logs in my l… --- ## [Calculate new field in logstash config file](https://discuss.elastic.co/t/calculate-new-field-in-logstash-config-file/237097)
**Author:** [@inchirah](https://discuss.elastic.co/u/inchirah)\ **Replies:** 0\ **Last updated:** [June 15, 2020, 9:19am UTC](https://discuss.elastic.co/t/calculate-new-field-in-logstash-config-file/237097 "2020-06-15T09:19:48Z")
Hello everyone, I wonder if I can calculate a new fields using the fields in my csv file in the logstash configuration file . Priciasly I want to add a field attempt complaint (number of observation where ServRvLevdle \>… --- ## [Write data event to azure blob storage from logstash conf file](https://discuss.elastic.co/t/write-data-event-to-azure-blob-storage-from-logstash-conf-file/237074)
**Author:** [@Pankaj\_Jadhav](https://discuss.elastic.co/u/Pankaj_Jadhav)\ **Replies:** 0\ **Last updated:** [June 15, 2020, 6:40am UTC](https://discuss.elastic.co/t/write-data-event-to-azure-blob-storage-from-logstash-conf-file/237074 "2020-06-15T06:40:06Z")
Hi, can someone will help me for writing data in azure blob storage from logstash. When I will get any exception inside tags on condition basis needs to send that data inside azure blob storage --- ## [Updating preinstalled Logstash gems](https://discuss.elastic.co/t/updating-preinstalled-logstash-gems/235541)
**Author:** [@Manuel\_M](https://discuss.elastic.co/u/Manuel_M)\ **Replies:** 2\ **Last updated:** [June 15, 2020, 6:09am UTC](https://discuss.elastic.co/t/updating-preinstalled-logstash-gems/235541 "2020-06-15T06:09:10Z")
Hello, I have already developed multiple logstash-filter plugins for my company, which are located in our private repository. I don't have any problems installing them and using them in our Logstash. I had to make chan… --- ## [Load balancer endpoint for pipeline](https://discuss.elastic.co/t/load-balancer-endpoint-for-pipeline/235944)
**Author:** [@shubhamshd99](https://discuss.elastic.co/u/shubhamshd99)\ **Replies:** 4\ **Last updated:** [June 15, 2020, 6:00am UTC](https://discuss.elastic.co/t/load-balancer-endpoint-for-pipeline/235944 "2020-06-15T06:00:48Z")
Hi everyone, I wanted to use jdbc plugin to ingest data in any database into logstash, and this, in turn, will require setting up a load balancer endpoint. So could anyone give me any tips on how to achieve this??? --- ## [How to search across multiple indices in single query?](https://discuss.elastic.co/t/how-to-search-across-multiple-indices-in-single-query/237059)
**Author:** [@Krishna\_Sai\_Nag\_G](https://discuss.elastic.co/u/Krishna_Sai_Nag_G)\ **Replies:** 1\ **Last updated:** [June 15, 2020, 5:02am UTC](https://discuss.elastic.co/t/how-to-search-across-multiple-indices-in-single-query/237059 "2020-06-15T05:02:46Z")
I want to search across multiple indices in a single query, is it possible to search in elastic search? --- ## [Edit default logstash index template](https://discuss.elastic.co/t/edit-default-logstash-index-template/236682)
**Author:** [@FabioAntunes](https://discuss.elastic.co/u/FabioAntunes)\ **Replies:** 0\ **Last updated:** [June 11, 2020, 10:49am UTC](https://discuss.elastic.co/t/edit-default-logstash-index-template/236682 "2020-06-11T10:49:13Z")
Hey folks, been going back and forth on documentation and discuss topics. Unfortunately, I couldn't found a solution to my problem. I have a default index template for logstash, if I do GET /\_template/elasticsearch-temp… --- ## [Remove spaces from field names after JSON parsing from filebeat](https://discuss.elastic.co/t/remove-spaces-from-field-names-after-json-parsing-from-filebeat/236806)
**Author:** [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Replies:** 2\ **Last updated:** [June 14, 2020, 11:14pm UTC](https://discuss.elastic.co/t/remove-spaces-from-field-names-after-json-parsing-from-filebeat/236806 "2020-06-14T23:14:41Z")
Sample data { "time": "2020-06-11 01:59:16.9760 -05:00", "level": "Info", "threadId": "164", "Message": "GetContactDataForCustomAutoAssignWorker", "Message": "Completed", "Call Tracker Id": "7e6a1602-6a89-4933-9329-d173… --- ## [Unable to parse date in logstash getting \_grokparsefailure in elasticsearch](https://discuss.elastic.co/t/unable-to-parse-date-in-logstash-getting-grokparsefailure-in-elasticsearch/237039)
**Author:** [@Ankit\_Goel1](https://discuss.elastic.co/u/Ankit_Goel1)\ **Replies:** 1\ **Last updated:** [June 14, 2020, 10:33pm UTC](https://discuss.elastic.co/t/unable-to-parse-date-in-logstash-getting-grokparsefailure-in-elasticsearch/237039 "2020-06-14T22:33:12Z")
I am using beats to read logs from my log file and log format is like: 2020-06-07T17:45:35.8964992Z|101|Info|136|ThreadPool|ValidationService.GetValidValueDate.LookupByCurrencyAndOwnersOfClientId I tried many patterns … --- ## [Metricbeat to logstash parsing error: illegal\_state\_exception & Can't get text on a START\_OBJECT](https://discuss.elastic.co/t/metricbeat-to-logstash-parsing-error-illegal-state-exception-cant-get-text-on-a-start-object/237035)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 1\ **Last updated:** [June 14, 2020, 7:40pm UTC](https://discuss.elastic.co/t/metricbeat-to-logstash-parsing-error-illegal-state-exception-cant-get-text-on-a-start-object/237035 "2020-06-14T19:40:39Z")
Hello, I am using the following configuration to get metrics from a Raspebrry Pi (running Raspberry OS) using Metricbeat. It works flawlessly when I send the logs directly to elasticsearch. However, I want to move my ing… --- ## [Logstash giving this error when I am running it in my Command prompt](https://discuss.elastic.co/t/logstash-giving-this-error-when-i-am-running-it-in-my-command-prompt/237036)
**Author:** [@TUSHAR\_Sinha](https://discuss.elastic.co/u/TUSHAR_Sinha)\ **Replies:** 0\ **Last updated:** [June 14, 2020, 7:17pm UTC](https://discuss.elastic.co/t/logstash-giving-this-error-when-i-am-running-it-in-my-command-prompt/237036 "2020-06-14T19:17:01Z")
Error: Could not find or load main class Stack\\logstash-7.7.1\\logstash-7.7.1\\logstash-core\\lib\\jars\\animal-sniffer-annotations-1.14.jar I have installed the logstash and while running it. It is showing me this error. Ho… --- ## [Parse inconsistent XML trace using logstash](https://discuss.elastic.co/t/parse-inconsistent-xml-trace-using-logstash/236985)
**Author:** [@joshua.stanley](https://discuss.elastic.co/u/joshua.stanley)\ **Replies:** 1\ **Last updated:** [June 14, 2020, 5:12pm UTC](https://discuss.elastic.co/t/parse-inconsistent-xml-trace-using-logstash/236985 "2020-06-14T17:12:09Z")
I am very new to ELK , need your guidance in my below usecase development. We have lot of XML files generated from source server(around 55GB per day )..but, from each file, we need only very few lines. There are 2 para… --- ## [Could not send Elasticresearch via logstash](https://discuss.elastic.co/t/could-not-send-elasticresearch-via-logstash/237013)
**Author:** [@sssjin](https://discuss.elastic.co/u/sssjin)\ **Replies:** 1\ **Last updated:** [June 14, 2020, 4:20pm UTC](https://discuss.elastic.co/t/could-not-send-elasticresearch-via-logstash/237013 "2020-06-14T16:20:26Z")
Want to send my original log for CPU usage to elasticsearch via logstash. Here is the log file pasth:/opt/dev/resource/cpu/log file name:result.log 06/14/2020:22:30:02,41 2.logstash input file input { file { pat… --- ## [Where can I find the grok pattern for the COMBINEDAPACHELOG in logstash?](https://discuss.elastic.co/t/where-can-i-find-the-grok-pattern-for-the-combinedapachelog-in-logstash/236990)
**Author:** [@curiousmind](https://discuss.elastic.co/u/curiousmind)\ **Replies:** 4\ **Last updated:** [June 14, 2020, 2:41pm UTC](https://discuss.elastic.co/t/where-can-i-find-the-grok-pattern-for-the-combinedapachelog-in-logstash/236990 "2020-06-14T14:41:53Z")
When I try parsing the apache access logs with the following grok rule, it comes out with 0 grokparsefailures match =\> { "message" =\> \["%{COMBINEDAPACHELOG}"\] } But when I try doing the same with the following grok pa… --- ## [Logstash output condition to filter a string in syslog](https://discuss.elastic.co/t/logstash-output-condition-to-filter-a-string-in-syslog/237024)
**Author:** [@sanjedi](https://discuss.elastic.co/u/sanjedi)\ **Replies:** 1\ **Last updated:** [June 14, 2020, 12:50pm UTC](https://discuss.elastic.co/t/logstash-output-condition-to-filter-a-string-in-syslog/237024 "2020-06-14T12:50:37Z")
Hi, I am new to ELK and trying to send syslogs based on a customer code to different ES indexes using Logstash. I am trying following with no luck. input { udp { type =\> syslogs port =\> 5001 } } output { if \[… --- ## [Logstash email output plugin throwing error](https://discuss.elastic.co/t/logstash-email-output-plugin-throwing-error/237018)
**Author:** [@Mahebub\_Sayyed](https://discuss.elastic.co/u/Mahebub_Sayyed)\ **Replies:** 0\ **Last updated:** [June 14, 2020, 10:09am UTC](https://discuss.elastic.co/t/logstash-email-output-plugin-throwing-error/237018 "2020-06-14T10:09:03Z")
Below is logstash config email { to =\> "to@email" from =\> "noreply@fromemail" subject =\> "Alert" body =\> '%{message}' port =\> 25 debug =\> "true" address =\> "emailserverIP" … --- ## [Can't run logstash. pathname contains null byte](https://discuss.elastic.co/t/cant-run-logstash-pathname-contains-null-byte/237008)
**Author:** [@Mahathir\_Mohammad](https://discuss.elastic.co/u/Mahathir_Mohammad)\ **Replies:** 13\ **Last updated:** [June 14, 2020, 7:25am UTC](https://discuss.elastic.co/t/cant-run-logstash-pathname-contains-null-byte/237008 "2020-06-14T07:25:54Z")
I'm trying to run logstash for elasticsearch but getting this error. I don't know what is the reason. Please help me. \[ERROR\] 2020-06-14 01:17:31.633 \[\[main\]-pipeline-manager\] javapipeline - Pipeline aborted due to … --- ## [Logstash crashes, restarts and then it just duplicates the data over and over](https://discuss.elastic.co/t/logstash-crashes-restarts-and-then-it-just-duplicates-the-data-over-and-over/236997)
**Author:** [@jensolsson.se](https://discuss.elastic.co/u/jensolsson.se)\ **Replies:** 0\ **Last updated:** [June 13, 2020, 9:17pm UTC](https://discuss.elastic.co/t/logstash-crashes-restarts-and-then-it-just-duplicates-the-data-over-and-over/236997 "2020-06-13T21:17:51Z")
I have set up logstash to index data from AWS ELB logs. I download the logs to the machine every minute and then I index the logs using logstash. Now I believe since the log files are in .gz that there is a corrupt gzip… --- ## [Logstash cannot get filebeat's data](https://discuss.elastic.co/t/logstash-cannot-get-filebeats-data/236955)
**Author:** [@azteker](https://discuss.elastic.co/u/azteker)\ **Replies:** 3\ **Last updated:** [June 13, 2020, 10:23am UTC](https://discuss.elastic.co/t/logstash-cannot-get-filebeats-data/236955 "2020-06-13T10:23:48Z")
This is my /etc/filebeat/filebeat.yml filebeat.inputs: - type: log enabled: true paths: - /var2/log/cloud/test.log filebeat.config.modules: # Glob pattern for configuration loading … --- ## [Logstash installation problems Ubuntu 20.04 VM](https://discuss.elastic.co/t/logstash-installation-problems-ubuntu-20-04-vm/235231)
**Author:** [@E\_Anon](https://discuss.elastic.co/u/E_Anon)\ **Replies:** 12\ **Last updated:** [June 13, 2020, 4:09am UTC](https://discuss.elastic.co/t/logstash-installation-problems-ubuntu-20-04-vm/235231 "2020-06-13T04:09:37Z")
Could someone please walk me through the correct Logstash installation procedure for Ubuntu 20.04? Starting with correct JDK installation as required by Logstash installation. Elasticsearch and Kibana install just fine… --- ## [How to set geo\_point type from csv files using logstash?](https://discuss.elastic.co/t/how-to-set-geo-point-type-from-csv-files-using-logstash/236182)
**Author:** [@Jun.8](https://discuss.elastic.co/u/Jun.8)\ **Replies:** 7\ **Last updated:** [June 13, 2020, 1:59am UTC](https://discuss.elastic.co/t/how-to-set-geo-point-type-from-csv-files-using-logstash/236182 "2020-06-13T01:59:55Z")
Hi, I'm trying to set geo\_point type from csv files which have latitude and longitude using logstash. But I really don't know how to convert latitude and longitude to location(geo\_point type). I'm using logstash for this… --- ## [Conditional index in output](https://discuss.elastic.co/t/conditional-index-in-output/236951)
**Author:** [@JustDave](https://discuss.elastic.co/u/JustDave)\ **Replies:** 0\ **Last updated:** [June 13, 2020, 1:10am UTC](https://discuss.elastic.co/t/conditional-index-in-output/236951 "2020-06-13T01:10:19Z")
I was doing research and some testing in my environment. I am using Filebeat to read 2 kinds of logs, that go into 2 different index patterns. I looked up conditionals in output, and found some articles, but one had a so… --- ## [Enabling the sFlow Codec in Logstash Container?](https://discuss.elastic.co/t/enabling-the-sflow-codec-in-logstash-container/236948)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 0\ **Last updated:** [June 12, 2020, 9:46pm UTC](https://discuss.elastic.co/t/enabling-the-sflow-codec-in-logstash-container/236948 "2020-06-12T21:46:24Z")
Hi Logstash Jedi Masters, I recently saw an instance of Logstash 7.7.1 that was accepting sFlow traffic as input. My colleague said he was able to set it up with no problem at all. I would like to do this, but in a Lo… --- ## [How to parse XML using XPATH from SOAP](https://discuss.elastic.co/t/how-to-parse-xml-using-xpath-from-soap/236597)
**Author:** [@vince.mar04](https://discuss.elastic.co/u/vince.mar04)\ **Replies:** 4\ **Last updated:** [June 12, 2020, 4:02pm UTC](https://discuss.elastic.co/t/how-to-parse-xml-using-xpath-from-soap/236597 "2020-06-12T16:02:30Z")
Continuing the discussion from How to parse XML using XPATH from SOAP API in Logstash: in this part we mentioned this question, it was solved with /soapenv:Envelope/soapenv:Body/multiRef/metricValue/text() but in my ca… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=341) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=343)