# Elastic search not ingesting data

**URL:** <https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197>\
**Category:** Logstash\
**Created:** [August 4, 2016, 9:17am UTC](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197 "2016-08-04T09:17:59Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![James\_Kazie](https://avatars.discourse-cdn.com/v4/letter/j/bb73d2/32.png) [@James\_Kazie](https://discuss.elastic.co/u/James_Kazie)\
**Post date:** [August 4, 2016, 9:17am UTC](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197/1 "2016-08-04T09:17:59Z")

</div>

I suspect i am doing something blatantly wrong.  
Logstash appears to be going through my data properly and showing the 5000 data points being sent to elastic search.  
However when i look up the index in elastic search it shows only 5 events.  
the configuration i used is as follows

# The # character at the beginning of a line indicates a comment Use

# comments to describe your configuration

input {  
file {  
path =\> "C:\Users\James Kazie\Desktop\ProV\Copy of Devices List 04-08-16 (72046).csv"  
type =\> "ProV"  
start\_position =\> "beginning"

```
}

```

}

filter {  
csv {

```
	columns => [
		"Agent.MachineName",
		"Version",
		"PulseInfo.IsOnline",
		"Agent.MacAddress",
		"Agent.Inventory.Computer.Model",
		"Agent.Inventory.OperatingSystem.OsName",
		"Agent.DeviceAgentVersion",
		"Agent.IpAddress",
		"Agent.NetworkAddress",
		"Agent.WriteFilter",
		"Agent.IsInPersistance",
		"PulseInfo.LastPulse"
		]
	separator => ","
}

```

geoip{  
source =\> "Agent.IpAddress"  
}  
date {  
match =\> ["PulseInfo.LastPulse","dd/MM/YYYY HH:mm"]  
}

mutate {rename =\> {"Agent.MachineName"=\> "WADid"}}  
mutate {rename =\> {"PulseInfo.IsOnline" =\> "LastOnline" }}  
mutate {rename =\> {"Agent.MacAddress" =\> "MacAddress"}}  
mutate {rename =\> {"Agent.Inventory.OperatingSystem.OsName" =\> "OS"}}  
mutate {rename =\> {"Agent.Inventory.Computer.Model"=\>"ComputerModel"}}  
mutate {rename =\> {"Agent.DeviceAgentVersion" =\> "AgentVersion" }}  
mutate {rename =\> {"Agent.IpAddress" =\> "IP"}}  
mutate {rename =\> {"Agent.NetworkAddress" =\> "NetworkAddress"}}  
mutate {rename =\> {"Agent.WriteFilter" =\> "WWF" }}  
mutate {rename =\> {"Agent.IsInPersistance" =\> "Persistence"}}  
mutate {rename =\> {"PulseInfo.LastPulse" =\> "LastPulse"}}  
}

output {  
elasticsearch {  
hosts=\>"localhost"  
action =\> "index"  
index =\> "devicestatusprov"  
document\_id =\> "WADid"}  
stdout {}  
}

---

<div class="post-metadata">

**Author:** ![James\_Kazie](https://avatars.discourse-cdn.com/v4/letter/j/bb73d2/32.png) [@James\_Kazie](https://discuss.elastic.co/u/James_Kazie)\
**Post date:** [August 4, 2016, 7:29pm UTC](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197/2 "2016-08-04T19:29:02Z")

</div>

I realised my error, i should have used document\_id =\> %{ID} my config was overwriting elastic search 5000 times

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 5, 2016, 12:36am UTC](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197/3 "2016-08-05T00:36:40Z")

</div>

Why are you renaming the columns? Just set them to whatever you want in the CSV filter part and that is the fieldname that will be used.

---

<div class="post-metadata">

**Author:** ![James\_Kazie](https://avatars.discourse-cdn.com/v4/letter/j/bb73d2/32.png) [@James\_Kazie](https://discuss.elastic.co/u/James_Kazie)\
**Post date:** [August 5, 2016, 10:36pm UTC](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197/4 "2016-08-05T22:36:08Z")

</div>

I mistook the use of the statement. As my file already had that column i thought i needed to specify. I am aware it is not needed now

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:44am UTC](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197/5 "2017-07-06T04:44:26Z")

</div>