# Too many events

**URL:** <https://discuss.elastic.co/t/too-many-events/170911>\
**Category:** Logstash\
**Created:** [March 5, 2019, 1:18pm UTC](https://discuss.elastic.co/t/too-many-events/170911 "2019-03-05T13:18:52Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [March 5, 2019, 1:18pm UTC](https://discuss.elastic.co/t/too-many-events/170911/1 "2019-03-05T13:18:52Z")

</div>

Hi!

I am using jdbc plugin to make a query in an SQL server db and then i import the result of the query in ES.  
In SQL, when i run select all i get 200.000 rows and after i export them into ES i get over 1 million events.  
Does anyone have an idea why is this happening? Do i get duplicates event somehow?

Thanks!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 5, 2019, 1:51pm UTC](https://discuss.elastic.co/t/too-many-events/170911/2 "2019-03-05T13:51:46Z")

</div>

What does the input configuration look like? Do you have an ORDER BY clause?

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [March 5, 2019, 2:05pm UTC](https://discuss.elastic.co/t/too-many-events/170911/3 "2019-03-05T14:05:43Z")

</div>

I have a SELECT \* query and in the filter section i just get a substring and add it in a different field using grok filter.

```
grok {
			match => {
				"logevent" => 'MessageType":"?(?<messagetype>[a-zA-Z]+)'
			}
		}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 5, 2019, 2:10pm UTC](https://discuss.elastic.co/t/too-many-events/170911/4 "2019-03-05T14:10:29Z")

</div>

That did not answer either of my questions...

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [March 5, 2019, 2:23pm UTC](https://discuss.elastic.co/t/too-many-events/170911/5 "2019-03-05T14:23:55Z")

</div>

```
input {
	jdbc {
	    charset => "UTF-8"
		jdbc_driver_library => "D:\sqljdbc_7.0\enu\mssql-jdbc-7.0.0.jre8.jar"
		jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
		jdbc_connection_string => "jdbc:sqlserver:// *****************************"
		jdbc_user => " ********"
		jdbc_password => " **************!"
		statement => "SELECT * FROM [dbo].[__Logs]"
		use_column_value => true
		tracking_column => "id"
	}
}
filter {
	if "MessageType" in [logevent] {
		grok {
			match => {
				"logevent" => 'MessageType":"?(?<messagetype>[a-zA-Z]+)'
			}
		}
	}
}
output {
	stdout {
		codec => json_lines {charset => "UTF-8"}
	}
	elasticsearch {
		hosts => "https:// *****************************"
		user => "elastic"
		password => " **************"
		index => "count_of_events-%{+YYYY.MM.dd}"
		document_type => "logs"
	}
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 5, 2019, 2:28pm UTC](https://discuss.elastic.co/t/too-many-events/170911/6 "2019-03-05T14:28:38Z")

</div>

Does it work any better if you use this?...

```
statement => "SELECT * FROM [dbo].[__Logs] ORDER BY id"
```

---

<div class="post-metadata">

**Author:** ![Costi](https://avatars.discourse-cdn.com/v4/letter/c/4491bb/32.png) [@Costi](https://discuss.elastic.co/u/Costi)\
**Post date:** [March 6, 2019, 7:38am UTC](https://discuss.elastic.co/t/too-many-events/170911/7 "2019-03-06T07:38:02Z")

</div>

I tried but i get again too many events

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 3, 2019, 7:38am UTC](https://discuss.elastic.co/t/too-many-events/170911/8 "2019-04-03T07:38:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.