# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=308 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 309 --- ## [New way in which java\_execution breaks the aggregate filter](https://discuss.elastic.co/t/new-way-in-which-java-execution-breaks-the-aggregate-filter/239377)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 4\ **Last updated:** [September 2, 2020, 12:47pm UTC](https://discuss.elastic.co/t/new-way-in-which-java-execution-breaks-the-aggregate-filter/239377 "2020-09-02T12:47:45Z")
When java\_execution is enabled, the periodic\_flush does not occur, so you cannot use a timeout to trigger flushing an event from an aggregate filter. I do not think the final flush occurs either. This is in 7.8.0, and I … --- ## [Dashboards through logstash](https://discuss.elastic.co/t/dashboards-through-logstash/247207)
**Author:** [@headtea](https://discuss.elastic.co/u/headtea)\ **Replies:** 0\ **Last updated:** [September 2, 2020, 9:55am UTC](https://discuss.elastic.co/t/dashboards-through-logstash/247207 "2020-09-02T09:55:43Z")
I'm directing my beats (auditbeat & filebeat) output directly to logstash. In both beats' configs I have dashboards enabled. The problem is that the dashboards data doesn't seem to end up in kibana if I output to logsta… --- ## [Logstash dateparsefailure](https://discuss.elastic.co/t/logstash-dateparsefailure/247191)
**Author:** [@Orest\_Gulman](https://discuss.elastic.co/u/Orest_Gulman)\ **Replies:** 1\ **Last updated:** [September 2, 2020, 9:24am UTC](https://discuss.elastic.co/t/logstash-dateparsefailure/247191 "2020-09-02T09:24:17Z")
Hi. My log timestamp has a format Sep 2, 2020 @ 11:14:00.000 When I'm trying to parse it in logstash config I'm getting an error \_dateparsefailure date { target =\> "@timestamp" match =\> \[ "timestamp",… --- ## [How to snyc mongo data into ES](https://discuss.elastic.co/t/how-to-snyc-mongo-data-into-es/247199)
**Author:** [@Milan\_Suthar](https://discuss.elastic.co/u/Milan_Suthar)\ **Replies:** 0\ **Last updated:** [September 2, 2020, 8:52am UTC](https://discuss.elastic.co/t/how-to-snyc-mongo-data-into-es/247199 "2020-09-02T08:52:59Z")
as im new to ES i want to replicate my mongodb collections to ES how to snyc mongo(v3.4 ) data into ES ( version can be anyone lastest ), version is not an issue, --- ## [Enriching MMDB files with your own data using Go](https://discuss.elastic.co/t/enriching-mmdb-files-with-your-own-data-using-go/247158)
**Author:** [@nchelluri](https://discuss.elastic.co/u/nchelluri)\ **Replies:** 1\ **Last updated:** [September 2, 2020, 2:49am UTC](https://discuss.elastic.co/t/enriching-mmdb-files-with-your-own-data-using-go/247158 "2020-09-02T02:49:14Z")
I work for MaxMind and it was pointed out to me that a popular blog post of ours around here was this post from 2015 showing how to build an MMDB using Perl. Well, it's 2020 now and we've got a post showing how to enric… --- ## [Unable to get Logstash Centralised Management working](https://discuss.elastic.co/t/unable-to-get-logstash-centralised-management-working/247091)
**Author:** [@david.preston](https://discuss.elastic.co/u/david.preston)\ **Replies:** 0\ **Last updated:** [September 1, 2020, 12:42pm UTC](https://discuss.elastic.co/t/unable-to-get-logstash-centralised-management-working/247091 "2020-09-01T12:42:58Z")
This morning i tried to move an existing logstash to use Centralised Management. I configured the logstash yml as per documentation. xpack.management.enabled: true xpack.management.pipeline.id: \["{pipeline-id}"\] xpac… --- ## [Ingesting .doc .txt .csv .sql and some others](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861)
**Author:** [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Replies:** 7\ **Last updated:** [September 1, 2020, 8:48pm UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861 "2020-09-01T20:48:07Z")
Hi everyone, I have a question. What would be the fastest way to ingest data? I have a folder that contains a variety of file formats including .doc, .txt, .csv and .sql. I have roughly 30 gigs of files like this that I… --- ## [Time zone in the field?](https://discuss.elastic.co/t/time-zone-in-the-field/247139)
**Author:** [@Alex9](https://discuss.elastic.co/u/Alex9)\ **Replies:** 1\ **Last updated:** [September 1, 2020, 7:52pm UTC](https://discuss.elastic.co/t/time-zone-in-the-field/247139 "2020-09-01T19:52:06Z")
Good day :slightly\_smiling\_face: (I probably have a simple question, but I haven't been able to figure it out on my own) Please tell me how to properly configure the date display in the fields? I have a service that, … --- ## [Sending a all source document into one dest field](https://discuss.elastic.co/t/sending-a-all-source-document-into-one-dest-field/247134)
**Author:** [@bmth](https://discuss.elastic.co/u/bmth)\ **Replies:** 3\ **Last updated:** [September 1, 2020, 7:45pm UTC](https://discuss.elastic.co/t/sending-a-all-source-document-into-one-dest-field/247134 "2020-09-01T19:45:56Z")
I have the following documents in a index: { "demand\_id": 1, "date": "2020-08-20 12:00:00" }, { "action\_id": 1, "demand\_id": 1, "client": "renato", "date": "2020-08-20 12:05:00", "topic": "ready" }, { "action\_id": 1, "d… --- ## [Issues with special characters in docker env](https://discuss.elastic.co/t/issues-with-special-characters-in-docker-env/247021)
**Author:** [@NickyW1995](https://discuss.elastic.co/u/NickyW1995)\ **Replies:** 2\ **Last updated:** [September 1, 2020, 7:45pm UTC](https://discuss.elastic.co/t/issues-with-special-characters-in-docker-env/247021 "2020-09-01T19:45:45Z")
Hi, I've got an issue trying to parse special characters into CONFIG\_STRING in my dockerfile. I'm not sure here is the right place to ask, but figured someone might've come across this before. If I try to parse EN… --- ## [Logstash and InfluxDB](https://discuss.elastic.co/t/logstash-and-influxdb/246965)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 2\ **Last updated:** [September 1, 2020, 7:39pm UTC](https://discuss.elastic.co/t/logstash-and-influxdb/246965 "2020-09-01T19:39:25Z")
Hello everyone! I'm struggling with cooperation of Logstash and InfluxDB. I'm trying to use Logstash for parsing some data and then push them to influxdb. I'm trying it with influxdb output plugin that seems not be real… --- ## [Problem with Logstash Kinesis Input Plugin](https://discuss.elastic.co/t/problem-with-logstash-kinesis-input-plugin/247037)
**Author:** [@Rahul\_Kumar4](https://discuss.elastic.co/u/Rahul_Kumar4)\ **Replies:** 2\ **Last updated:** [September 1, 2020, 5:39pm UTC](https://discuss.elastic.co/t/problem-with-logstash-kinesis-input-plugin/247037 "2020-09-01T17:39:43Z")
Seeing this error in the Kinesis input plugin which is preventing the Kinesis records from getting consumed by the Logstash workers. I switched the log level to DEBUG and managed to see this. Is this a known problem? \[2… --- ## [Logstash parsing ECS](https://discuss.elastic.co/t/logstash-parsing-ecs/247108)
**Author:** [@Abdelhalim](https://discuss.elastic.co/u/Abdelhalim)\ **Replies:** 4\ **Last updated:** [September 1, 2020, 3:27pm UTC](https://discuss.elastic.co/t/logstash-parsing-ecs/247108 "2020-09-01T15:27:33Z")
Hello everybody, I am using logstash to parse my firewall logs, and some logs contain these information : src=x.x.x.x srcport=YYY dst=x.x.x.x modsrc=x.x.x.x origdst=x.x.x.x dst= x.x.x.x srcport=YY modsrcport= YY ori… --- ## [Cidr filter for over 1000 subnets](https://discuss.elastic.co/t/cidr-filter-for-over-1000-subnets/247110)
**Author:** [@ksremo](https://discuss.elastic.co/u/ksremo)\ **Replies:** 0\ **Last updated:** [September 1, 2020, 3:04pm UTC](https://discuss.elastic.co/t/cidr-filter-for-over-1000-subnets/247110 "2020-09-01T15:04:54Z")
I have round about 1500 subnets, with masks from /18 to /29, but mainly /24 networks. I tried to use the cidr logstash filter plugin but the pipeline then come not up. i think because the pipeline is too big and there a… --- ## [Can we use " :: " as a separator in csv filter in logstash](https://discuss.elastic.co/t/can-we-use-as-a-separator-in-csv-filter-in-logstash/246983)
**Author:** [@Shahid\_Mustafa](https://discuss.elastic.co/u/Shahid_Mustafa)\ **Replies:** 3\ **Last updated:** [September 1, 2020, 11:59am UTC](https://discuss.elastic.co/t/can-we-use-as-a-separator-in-csv-filter-in-logstash/246983 "2020-09-01T11:59:13Z")
Can we use following csv filter: I cannot test this directly in our system. csv { columns =\> \[ "LogLevel", "DateTime", "RequestID", "LogMessage" \] s… --- ## [How to extract the uri path and the param from the uri](https://discuss.elastic.co/t/how-to-extract-the-uri-path-and-the-param-from-the-uri/245521)
**Author:** [@Tristan\_D](https://discuss.elastic.co/u/Tristan_D)\ **Replies:** 1\ **Last updated:** [September 1, 2020, 11:43am UTC](https://discuss.elastic.co/t/how-to-extract-the-uri-path-and-the-param-from-the-uri/245521 "2020-09-01T11:43:20Z")
I want to extract the uri path and the param from the uri. There are two forms of the uri, one with params and one without. Here is the examples of the uri /sentence\_func?key=23872323232 /bot\_recognize I use this patt… --- ## [Remove ignore\_above default mapping from logstash instead of changing it to each index from elastic search](https://discuss.elastic.co/t/remove-ignore-above-default-mapping-from-logstash-instead-of-changing-it-to-each-index-from-elastic-search/247077)
**Author:** [@Radha](https://discuss.elastic.co/u/Radha)\ **Replies:** 0\ **Last updated:** [September 1, 2020, 11:12am UTC](https://discuss.elastic.co/t/remove-ignore-above-default-mapping-from-logstash-instead-of-changing-it-to-each-index-from-elastic-search/247077 "2020-09-01T11:12:37Z")
Hello, I am having some multiple indexes and logs are being filtered in logstash and sending to elastic search and kibana, when I see in elastic searching mapping some text/keyword fields but with additional property o… --- ## [Drop grok failures](https://discuss.elastic.co/t/drop-grok-failures/246476)
**Author:** [@headtea](https://discuss.elastic.co/u/headtea)\ **Replies:** 9\ **Last updated:** [September 1, 2020, 9:46am UTC](https://discuss.elastic.co/t/drop-grok-failures/246476 "2020-09-01T09:46:42Z")
I use Logstash to filter data from big files like mysqld.log and mongos.log. It receives thousands of entries from these logs and I use grok to filter data and display it in Kibana. There are many entries that logstash … --- ## [Logstash error, Unable to connect with elasticsearch after enabling ssl](https://discuss.elastic.co/t/logstash-error-unable-to-connect-with-elasticsearch-after-enabling-ssl/247066)
**Author:** [@Vi\_shal](https://discuss.elastic.co/u/Vi_shal)\ **Replies:** 0\ **Last updated:** [September 1, 2020, 9:29am UTC](https://discuss.elastic.co/t/logstash-error-unable-to-connect-with-elasticsearch-after-enabling-ssl/247066 "2020-09-01T09:29:27Z")
I have an elk setup (1 master ES, 3 worker es, 1 logstash, 1 kibana) with filebeat being the log collector/emitter. Post enabling x-pack and TLS, ES and Kibana is working fine. the problem is with logstash. I'm currently… --- ## [Logstash - Configuring Two output plugin](https://discuss.elastic.co/t/logstash-configuring-two-output-plugin/247059)
**Author:** [@tamilarasanbravo](https://discuss.elastic.co/u/tamilarasanbravo)\ **Replies:** 1\ **Last updated:** [September 1, 2020, 8:52am UTC](https://discuss.elastic.co/t/logstash-configuring-two-output-plugin/247059 "2020-09-01T08:52:20Z")
Hi Team, Currently, I have an output plugin configured as below. I am looking for another output method to send it to mongodb. input { http { host =\> "10.199.208.43" port =\> "8080" type =\> "… --- ## [How to make single jdbc connection to sql server in logstash configuration instead of making connection everytime to each table](https://discuss.elastic.co/t/how-to-make-single-jdbc-connection-to-sql-server-in-logstash-configuration-instead-of-making-connection-everytime-to-each-table/247045)
**Author:** [@tharak](https://discuss.elastic.co/u/tharak)\ **Replies:** 0\ **Last updated:** [September 1, 2020, 5:44am UTC](https://discuss.elastic.co/t/how-to-make-single-jdbc-connection-to-sql-server-in-logstash-configuration-instead-of-making-connection-everytime-to-each-table/247045 "2020-09-01T05:44:04Z")
I have 10 sql tables in my database, to index all those tables data i have logstash configuration where i am making jdbc connection to each and every table. Instead of doing that how can i make a connection only once to … --- ## [Date Parse failures](https://discuss.elastic.co/t/date-parse-failures/246821)
**Author:** [@vvwood](https://discuss.elastic.co/u/vvwood)\ **Replies:** 7\ **Last updated:** [August 31, 2020, 9:37pm UTC](https://discuss.elastic.co/t/date-parse-failures/246821 "2020-08-31T21:37:51Z")
I have tried opening an official support case without getting any help on the following topic. Hoping someone here can help.... I have several log sources with a date field like this: Mar 22 2020 21:52:00 In my Index T… --- ## [Output logs to host based on timestamp](https://discuss.elastic.co/t/output-logs-to-host-based-on-timestamp/246357)
**Author:** [@ManjaroBlack](https://discuss.elastic.co/u/ManjaroBlack)\ **Replies:** 7\ **Last updated:** [August 31, 2020, 9:30pm UTC](https://discuss.elastic.co/t/output-logs-to-host-based-on-timestamp/246357 "2020-08-31T21:30:52Z")
I have logs that are a mix of old and new. I'd like to send any logs that are newer than 2 weeks to fast SSD, otherwise send to slower HDD. Can someone point me in the direction to figure this out? --- ## [Logstash will not create an index pattern in elastic search](https://discuss.elastic.co/t/logstash-will-not-create-an-index-pattern-in-elastic-search/246992)
**Author:** [@stor314](https://discuss.elastic.co/u/stor314)\ **Replies:** 3\ **Last updated:** [August 31, 2020, 6:08pm UTC](https://discuss.elastic.co/t/logstash-will-not-create-an-index-pattern-in-elastic-search/246992 "2020-08-31T18:08:23Z")
I am an extreme beginner with the elastic stack, I'm simply trying to create a very simple setup with elasticsearch logstash and kibana on my local machine. All of them are installed and are able to run, using versions 7… --- ## [Enable TLS logstash to \<-\> Elasticsearch](https://discuss.elastic.co/t/enable-tls-logstash-to-elasticsearch/246986)
**Author:** [@Fetakunen](https://discuss.elastic.co/u/Fetakunen)\ **Replies:** 0\ **Last updated:** [August 31, 2020, 4:43pm UTC](https://discuss.elastic.co/t/enable-tls-logstash-to-elasticsearch/246986 "2020-08-31T16:43:59Z")
Hi, i'm trying to secure my stack to enable alerts. Elasticsearch and Kibana is up and running but when i'm trying to enable https to the elasticsearch from logstash i get crash. The output config looks like this: outp… --- ## [Logstash error instability \[logstash.inputs.udp\]](https://discuss.elastic.co/t/logstash-error-instability-logstash-inputs-udp/246944)
**Author:** [@bluelight](https://discuss.elastic.co/u/bluelight)\ **Replies:** 1\ **Last updated:** [August 31, 2020, 3:05pm UTC](https://discuss.elastic.co/t/logstash-error-instability-logstash-inputs-udp/246944 "2020-08-31T15:05:46Z")
Hello Folks, I face many problems really big problems I was receiving logs fine before 2months ago. there is no stability in logstash it's doing self crashing. and I'm no more receive TCP Packets only UDP packets. Here… --- ## [Logstash (jdbc\_static) : Exception when filling lookup db from loader](https://discuss.elastic.co/t/logstash-jdbc-static-exception-when-filling-lookup-db-from-loader/246948)
**Author:** [@Mohammad\_Mousavi](https://discuss.elastic.co/u/Mohammad_Mousavi)\ **Replies:** 0\ **Last updated:** [August 31, 2020, 12:48pm UTC](https://discuss.elastic.co/t/logstash-jdbc-static-exception-when-filling-lookup-db-from-loader/246948 "2020-08-31T12:48:06Z")
Hi, This is my confing: loader\_schedule =\> "\* \*/1 \* \* \*" jdbc\_user =\> "logstash" jdbc\_password =\> "password" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" jdbc\_connection\_string =\> "jdbc:mysql://x.x.x.x.:3306/mydb" stagi… --- ## [Http\_poller input plugin digest authentication configuration](https://discuss.elastic.co/t/http-poller-input-plugin-digest-authentication-configuration/246943)
**Author:** [@Olga\_Pilipets](https://discuss.elastic.co/u/Olga_Pilipets)\ **Replies:** 0\ **Last updated:** [August 31, 2020, 11:28am UTC](https://discuss.elastic.co/t/http-poller-input-plugin-digest-authentication-configuration/246943 "2020-08-31T11:28:58Z")
Is it possible to call arbitrary HTTP endpoint with digest authentication using Http poller input plugin? If yes, how digest authentication type should be configured? --- ## [Logstash is overwriting output elasticsearch](https://discuss.elastic.co/t/logstash-is-overwriting-output-elasticsearch/246937)
**Author:** [@Turotakun98](https://discuss.elastic.co/u/Turotakun98)\ **Replies:** 0\ **Last updated:** [August 31, 2020, 10:44am UTC](https://discuss.elastic.co/t/logstash-is-overwriting-output-elasticsearch/246937 "2020-08-31T10:44:35Z")
Hi, Is it possible to add a field in elasticsearch output only if the document in the index does not contain that field? I'm trying to fetch data from a API (using the input http\_poller) and storing the data in a elast… --- ## [TCP Plugin - Error: event executor terminated](https://discuss.elastic.co/t/tcp-plugin-error-event-executor-terminated/246934)
**Author:** [@anon56147639](https://discuss.elastic.co/u/anon56147639)\ **Replies:** 0\ **Last updated:** [August 31, 2020, 10:27am UTC](https://discuss.elastic.co/t/tcp-plugin-error-event-executor-terminated/246934 "2020-08-31T10:27:31Z")
Hello, I'm facing an issue with the TCP input plugin in Logstash: \[2020-08-31T11:57:15,669\]\[ERROR\]\[logstash.javapipeline \]\[main\]\[ebfb52d241eb37911098921f56a3a2be4b082eb7a6e749166438a3e26fe28ccb\] A plugin had an unrec… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=307) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=309)