# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=403 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 404 --- ## [Logstash Two Different Date Format](https://discuss.elastic.co/t/logstash-two-different-date-format/219718)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 3\ **Last updated:** [February 18, 2020, 2:10pm UTC](https://discuss.elastic.co/t/logstash-two-different-date-format/219718 "2020-02-18T14:10:00Z")
I am parsing data from mysql to kibana. I have datefield which has datatype varchar now i have two different date format that is 1/4/2020 2:17:23 PM and 1/1/2020 12:00:00 AM can anyone please tell me what is the correc… --- ## [Multi Pipeline-Single Port | Logstash](https://discuss.elastic.co/t/multi-pipeline-single-port-logstash/219709)
**Author:** [@deeshu](https://discuss.elastic.co/u/deeshu)\ **Replies:** 1\ **Last updated:** [February 18, 2020, 2:04pm UTC](https://discuss.elastic.co/t/multi-pipeline-single-port-logstash/219709 "2020-02-18T14:04:13Z")
Hi techs... can we use single incoming ports for multiple pipeline conf files in logstash? As of now all the syslog coming in one port and pipeline. Idea is to create separate pipeline for each unique device types. TIA… --- ## [Logstash was installed successfully and works normally, but port 9600 is not listening](https://discuss.elastic.co/t/logstash-was-installed-successfully-and-works-normally-but-port-9600-is-not-listening/219783)
**Author:** [@dapeng](https://discuss.elastic.co/u/dapeng)\ **Replies:** 0\ **Last updated:** [February 18, 2020, 12:42pm UTC](https://discuss.elastic.co/t/logstash-was-installed-successfully-and-works-normally-but-port-9600-is-not-listening/219783 "2020-02-18T12:42:36Z")
Logstash was installed successfully and works normally, but port 9600 is not listening. Why is that ? Help me ,thanks. Information as follows: check logstash log as follows: --- ## [Logstash does not parse agent from NGINX access logs as it already has filebeat agent information in agent](https://discuss.elastic.co/t/logstash-does-not-parse-agent-from-nginx-access-logs-as-it-already-has-filebeat-agent-information-in-agent/219776)
**Author:** [@chaitanya\_chandake](https://discuss.elastic.co/u/chaitanya_chandake)\ **Replies:** 0\ **Last updated:** [February 18, 2020, 11:44am UTC](https://discuss.elastic.co/t/logstash-does-not-parse-agent-from-nginx-access-logs-as-it-already-has-filebeat-agent-information-in-agent/219776 "2020-02-18T11:44:12Z")
When tried to use default COMBINEDAPACHELOG for NGINX access logs, all fields are getting parsed correctly except agent field which contains useragent information. (https://www.elastic.co/guide/en/logstash/current/config… --- ## [Ruby filter to extract JSON value](https://discuss.elastic.co/t/ruby-filter-to-extract-json-value/219745)
**Author:** [@karlm](https://discuss.elastic.co/u/karlm)\ **Replies:** 1\ **Last updated:** [February 18, 2020, 10:36am UTC](https://discuss.elastic.co/t/ruby-filter-to-extract-json-value/219745 "2020-02-18T10:36:14Z")
I have an array and need to extract the latest date value, the array below looks like this { "conversations": \[ { "userMessages": \[ { "sbdh": { … --- ## [Integration with Cisco Umbrella](https://discuss.elastic.co/t/integration-with-cisco-umbrella/219741)
**Author:** [@joelle.nader](https://discuss.elastic.co/u/joelle.nader)\ **Replies:** 0\ **Last updated:** [February 18, 2020, 9:22am UTC](https://discuss.elastic.co/t/integration-with-cisco-umbrella/219741 "2020-02-18T09:22:43Z")
Hello, Is it possible to get logs from Cisco Umbrella and integrate them with logstash ? Cisco Umbrella writes the logs in AWS --- ## [Useragent filter not working properly](https://discuss.elastic.co/t/useragent-filter-not-working-properly/219731)
**Author:** [@mark.quilates](https://discuss.elastic.co/u/mark.quilates)\ **Replies:** 0\ **Last updated:** [February 18, 2020, 8:09am UTC](https://discuss.elastic.co/t/useragent-filter-not-working-properly/219731 "2020-02-18T08:09:58Z")
Hi, On my ELK Stack 7.6.0 the useragent logstash filtering not working properly. It doesn't able to get the %{os\_major} and %{os\_minor} when useragent used from windows or linux OS. But if used iOS or Android able to g… --- ## [Set order to elasticsearch fields using logstash](https://discuss.elastic.co/t/set-order-to-elasticsearch-fields-using-logstash/219620)
**Author:** [@Mohan\_vel](https://discuss.elastic.co/u/Mohan_vel)\ **Replies:** 1\ **Last updated:** [February 17, 2020, 3:10pm UTC](https://discuss.elastic.co/t/set-order-to-elasticsearch-fields-using-logstash/219620 "2020-02-17T15:10:56Z")
Hi Below is my config i am trying to index the data from MSSQL to Elasticsearch using Logstash but field order is getting changed after getting indexed into elasticsearch. input { jdbc { jdbc\_driver\_library =\> "… --- ## [Is there a way to ingest logs from Azure Log Analytics workspace to Logstash?](https://discuss.elastic.co/t/is-there-a-way-to-ingest-logs-from-azure-log-analytics-workspace-to-logstash/219698)
**Author:** [@iniyan.g](https://discuss.elastic.co/u/iniyan.g)\ **Replies:** 0\ **Last updated:** [February 18, 2020, 4:06am UTC](https://discuss.elastic.co/t/is-there-a-way-to-ingest-logs-from-azure-log-analytics-workspace-to-logstash/219698 "2020-02-18T04:06:06Z")
Hi Team We are working on to develop a central monitoring solution using ELK stack. AWS and Azure are the common platform where we are planning to ingest event logs from various services to logstash for parsing and visu… --- ## [How many Elasticsearch node can add in one logstash server](https://discuss.elastic.co/t/how-many-elasticsearch-node-can-add-in-one-logstash-server/219685)
**Author:** [@Ekta](https://discuss.elastic.co/u/Ekta)\ **Replies:** 0\ **Last updated:** [February 17, 2020, 11:51pm UTC](https://discuss.elastic.co/t/how-many-elasticsearch-node-can-add-in-one-logstash-server/219685 "2020-02-17T23:51:25Z")
In one logstash server how many elasticsearch nodes can add for only log checking purpose? --- ## [How to map fields from Oracle table to Index fields in Logstash?](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675)
**Author:** [@fefontana](https://discuss.elastic.co/u/fefontana)\ **Replies:** 5\ **Last updated:** [February 17, 2020, 9:52pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675 "2020-02-17T21:52:25Z")
Hello, It is not clear for me how to tell Logstash which field from my Oracle table corresponds to specific field of my predefined Elasticsearch index. For example, I have the following SQL configured in the .conf file … --- ## [Logstash Connection Reset by Peer](https://discuss.elastic.co/t/logstash-connection-reset-by-peer/217690)
**Author:** [@kitedj](https://discuss.elastic.co/u/kitedj)\ **Replies:** 5\ **Last updated:** [February 17, 2020, 6:32pm UTC](https://discuss.elastic.co/t/logstash-connection-reset-by-peer/217690 "2020-02-17T18:32:07Z")
Hello, I'm trying to debug a warning message I'm getting in my Logstash logs for my Winlogbeat filter. I've tried adding the setting client\_inactivity\_timeout =\> 0 to my Logstash input filter like I read in similar issue… --- ## [Logstash Twitter hangs after a while](https://discuss.elastic.co/t/logstash-twitter-hangs-after-a-while/219667)
**Author:** [@ainiml](https://discuss.elastic.co/u/ainiml)\ **Replies:** 0\ **Last updated:** [February 17, 2020, 6:01pm UTC](https://discuss.elastic.co/t/logstash-twitter-hangs-after-a-while/219667 "2020-02-17T18:01:28Z")
After running for a while Twitter logstash just hangs indefinitely, we've tested it and left it for days. Until we have to manually kill the kubernetes pods Then it would restart and continue again Has anyone else exp… --- ## [Parquet Format Output to S3](https://discuss.elastic.co/t/parquet-format-output-to-s3/216273)
**Author:** [@stranjer](https://discuss.elastic.co/u/stranjer)\ **Replies:** 1\ **Last updated:** [February 17, 2020, 5:08pm UTC](https://discuss.elastic.co/t/parquet-format-output-to-s3/216273 "2020-02-17T17:08:30Z")
Is there a way to have Logstash output to an S3 bucket in Parquet format, preferably using Snappy compression? I see there is an Avro codec but not Parquet one, and a Webhdfs output plugin that allows Snappy compression… --- ## [Does multiline codec expects log message into a field called \ field for processing?](https://discuss.elastic.co/t/does-multiline-codec-expects-log-message-into-a-field-called-message-field-for-processing/219636)
**Author:** [@Febin\_Stephen](https://discuss.elastic.co/u/Febin_Stephen)\ **Replies:** 2\ **Last updated:** [February 17, 2020, 3:32pm UTC](https://discuss.elastic.co/t/does-multiline-codec-expects-log-message-into-a-field-called-message-field-for-processing/219636 "2020-02-17T15:32:32Z")
We collect logs from journald default location where docker containers puts logs through stderr/stdout. default field name for log message is "log". If i want to use multi codec plugin inside logstash does it expect log … --- ## [Logstash aggregate filter skipping records](https://discuss.elastic.co/t/logstash-aggregate-filter-skipping-records/219362)
**Author:** [@saifrehman](https://discuss.elastic.co/u/saifrehman)\ **Replies:** 4\ **Last updated:** [February 17, 2020, 3:24pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-skipping-records/219362 "2020-02-17T15:24:26Z")
logstash aggregate filter skipping records. i have set the pipeline.worker to 1 but it's still missing nested records. I follow the instruction at here but noting work. It is not skipping records all the time. but it's … --- ## [ParserError: Illegal character ((CTRL-CHAR, code 27)): only regular white space (\\r, \\n, \\t) is allowed between tokens](https://discuss.elastic.co/t/parsererror-illegal-character-ctrl-char-code-27-only-regular-white-space-r-n-t-is-allowed-between-tokens/219550)
**Author:** [@Valm1980](https://discuss.elastic.co/u/Valm1980)\ **Replies:** 1\ **Last updated:** [February 17, 2020, 3:05pm UTC](https://discuss.elastic.co/t/parsererror-illegal-character-ctrl-char-code-27-only-regular-white-space-r-n-t-is-allowed-between-tokens/219550 "2020-02-17T15:05:08Z")
Hello, I am trying to filter out a log and drop any entries that have certain IP address, however I am getting an error: \[logstash.filters.json \] Error parsing json {:source=\>"message", :raw=\>"\\e\[32minfo\\e\[39m: 127.0… --- ## [How can i accelarete loading csv to logstash](https://discuss.elastic.co/t/how-can-i-accelarete-loading-csv-to-logstash/219643)
**Author:** [@samargh](https://discuss.elastic.co/u/samargh)\ **Replies:** 1\ **Last updated:** [February 17, 2020, 2:52pm UTC](https://discuss.elastic.co/t/how-can-i-accelarete-loading-csv-to-logstash/219643 "2020-02-17T14:52:55Z")
i have a csv file that contains 6 millions rows but it takes too much time to load i tried PUT /twitter/\_settings { "index" : { "number\_of\_replicas" : 0 } } and even PUT /twitter/\_settings { "index" : { "ref… --- ## [Add\_field not getting the field](https://discuss.elastic.co/t/add-field-not-getting-the-field/219445)
**Author:** [@Kassio\_Silva](https://discuss.elastic.co/u/Kassio_Silva)\ **Replies:** 5\ **Last updated:** [February 17, 2020, 2:51pm UTC](https://discuss.elastic.co/t/add-field-not-getting-the-field/219445 "2020-02-17T14:51:37Z")
Hello guys, I am a newbie in logstash filtering and i having issues to get a specific field. i want to get the sum of "Accounting-Output-Octets" that is inside the "service\_data\_container" field, is it possible using m… --- ## [Logstash DNS unexpected error](https://discuss.elastic.co/t/logstash-dns-unexpected-error/215741)
**Author:** [@vmanyushin](https://discuss.elastic.co/u/vmanyushin)\ **Replies:** 5\ **Last updated:** [February 17, 2020, 1:41pm UTC](https://discuss.elastic.co/t/logstash-dns-unexpected-error/215741 "2020-02-17T13:41:56Z")
After upgrade to 7.5.1 a got error messages in log: \[2020-01-20T16:00:39,036\]\[ERROR\]\[logstash.filters.dns \]\[main\] DNS: Unexpected Error. {:field=\>"upstream\_hostname", :value=\>"172.16.21.57", :message=\>"Mutex relocki… --- ## [Grok filter not triggered](https://discuss.elastic.co/t/grok-filter-not-triggered/219526)
**Author:** [@pol4bear](https://discuss.elastic.co/u/pol4bear)\ **Replies:** 2\ **Last updated:** [February 17, 2020, 11:03am UTC](https://discuss.elastic.co/t/grok-filter-not-triggered/219526 "2020-02-17T11:03:02Z")
Hi. I'm trying to transfer data from filebeat and metricbeat. I'm using Elastic Stack 7.6.0 and I saw document on Elastic homepage. Configure below is my logstash.conf but metricbeat works well but filebeat's apache2 and… --- ## [Logstash Json Input](https://discuss.elastic.co/t/logstash-json-input/219593)
**Author:** [@Ramon\_Mateo](https://discuss.elastic.co/u/Ramon_Mateo)\ **Replies:** 4\ **Last updated:** [February 17, 2020, 10:48am UTC](https://discuss.elastic.co/t/logstash-json-input/219593 "2020-02-17T10:48:19Z")
Hi. I have a problem with my input and pipeline. i'm new with pipelines and json filter. I don't know how to get logstash filter my json files and send to elasticsearch. This a exemple of my json file: {"tenantID": … --- ## [Problem using external Ruby script in Logstash](https://discuss.elastic.co/t/problem-using-external-ruby-script-in-logstash/219387)
**Author:** [@LuigiDelavega](https://discuss.elastic.co/u/LuigiDelavega)\ **Replies:** 5\ **Last updated:** [February 17, 2020, 10:23am UTC](https://discuss.elastic.co/t/problem-using-external-ruby-script-in-logstash/219387 "2020-02-17T10:23:53Z")
Hello guys, I'm having problem trying to get my ruby code filter into an external ruby file called test2.rb When i was using a ruby {} inside my configuration, it was working very well, and i am new to ruby so maybe my… --- ## [Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline/219168)
**Author:** [@samargh](https://discuss.elastic.co/u/samargh)\ **Replies:** 7\ **Last updated:** [February 13, 2020, 11:21am UTC](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline/219168 "2020-02-13T11:21:54Z")
i have this code : input { file { path =\> "/home/samar/data/subscriber.csv" start\_position =\> "beginning" sincdb =\> "/dev/null" } } filter { csv { sparator =\> "," columns =\> \["Subscriber ID (MSISDN)","Account … --- ## [metrics.gauge.LazyDelegatingGauge unknown type](https://discuss.elastic.co/t/metrics-gauge-lazydelegatinggauge-unknown-type/219195)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 4\ **Last updated:** [February 17, 2020, 7:38am UTC](https://discuss.elastic.co/t/metrics-gauge-lazydelegatinggauge-unknown-type/219195 "2020-02-17T07:38:33Z")
Updated logstash and are still seeing the warning: constant ::Fixnum upon launching logstash anything to do clear this warning? Downloading Packages: logstash-7.6.0.rpm … --- ## [Logstash file shutdown](https://discuss.elastic.co/t/logstash-file-shutdown/219369)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 12\ **Last updated:** [February 17, 2020, 6:35am UTC](https://discuss.elastic.co/t/logstash-file-shutdown/219369 "2020-02-17T06:35:15Z")
Logstash file is not starting getting this error please help --- ## [Dynamic tables in logstash v6.3 and above jdbc](https://discuss.elastic.co/t/dynamic-tables-in-logstash-v6-3-and-above-jdbc/219568)
**Author:** [@Kartik\_Kolachina](https://discuss.elastic.co/u/Kartik_Kolachina)\ **Replies:** 0\ **Last updated:** [February 17, 2020, 6:23am UTC](https://discuss.elastic.co/t/dynamic-tables-in-logstash-v6-3-and-above-jdbc/219568 "2020-02-17T06:23:59Z")
Is there a way to fetch data using logstash jdbc plugin from dynamically generated tables that follows a pattern? --- ## [Logstash and mongodb connection using JDBC is not working](https://discuss.elastic.co/t/logstash-and-mongodb-connection-using-jdbc-is-not-working/219119)
**Author:** [@SanTiaGo\_KarThiCk](https://discuss.elastic.co/u/SanTiaGo_KarThiCk)\ **Replies:** 5\ **Last updated:** [February 17, 2020, 5:42am UTC](https://discuss.elastic.co/t/logstash-and-mongodb-connection-using-jdbc-is-not-working/219119 "2020-02-17T05:42:01Z")
This is the configuration file gives output as \[unable to connect to database and no suitable driver was found!\] and I am using logstash version 7.5.1 --- ## [If statement incorrectly matches wildcard](https://discuss.elastic.co/t/if-statement-incorrectly-matches-wildcard/219322)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 2\ **Last updated:** [February 17, 2020, 4:04am UTC](https://discuss.elastic.co/t/if-statement-incorrectly-matches-wildcard/219322 "2020-02-17T04:04:53Z")
Hi, I have the following piece of code and noticed that for some reason when the IP is in the 192.168.150.\* range, both dev and test are matched. If the IP is in the 192.168.1.\* range it correctly matches only dev. If … --- ## [Tough config problem: logstash -\> hosted elastic stack, using TLS](https://discuss.elastic.co/t/tough-config-problem-logstash-hosted-elastic-stack-using-tls/219535)
**Author:** [@marc-at-qventus](https://discuss.elastic.co/u/marc-at-qventus)\ **Replies:** 1\ **Last updated:** [February 16, 2020, 7:00pm UTC](https://discuss.elastic.co/t/tough-config-problem-logstash-hosted-elastic-stack-using-tls/219535 "2020-02-16T19:00:19Z")
Hi all -- I'm having a tough configuration problem with I'm hoping the community can help with. First, the scenario: = I have a logstash instance (7.6.0) running in a docker container. = I am using the elastic hosted… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=402) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=404)