# Clients sending invalid docs **URL:** \ **Category:** Logstash\ **Created:** [April 13, 2016, 2:23pm UTC](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261 "2016-04-13T14:23:52Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![mhughes](https://avatars.discourse-cdn.com/v4/letter/m/85e7bf/32.png) [@mhughes](https://discuss.elastic.co/u/mhughes)\ **Post date:** [April 13, 2016, 2:23pm UTC](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261/1 "2016-04-13T14:23:52Z")
I have a bunch of clients that are sending documents directly to my ES cluster via bulkIndex. Those documents are valid for 1.x, but are now invalid with 2.x. Specifically, a lot of the documents had periods in field names. Sadly, it won't be trivial or quick to upgrade all those clients. My current plan is to stand up a separate 2.3 cluster. I can use logstash's elasticsearch input/output plugins to copy over all the existing indices. My problem is, new data is being written all the time with the now-invalid documents. Is there anything in logstash that will watch an index and slurp in just the data? All the indicies are daily indicies; at worst I could setup a cron job to copy over the current day's indices every N time period. But I'd rather have it react to real traffic. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 14, 2016, 12:57am UTC](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261/2 "2016-04-14T00:57:12Z")
The ES input can't do streaming like that, it's batch only. ---
**Author:** ![mhughes](https://avatars.discourse-cdn.com/v4/letter/m/85e7bf/32.png) [@mhughes](https://discuss.elastic.co/u/mhughes)\ **Post date:** [April 14, 2016, 1:20pm UTC](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261/3 "2016-04-14T13:20:20Z")
Ok, is there any way that ES Watcher helps? My two other options that I can see are (short of upgrading every client, which again, is out of my control): 1. writing some sort of http proxy that intercepts and mangles the request before it hits ES 2. awake the ES input every N minutes and re-index everything. If I just limit it to the past 24-48 hours, I think that is feasible. I'm not sending that much data. Maybe a 1 gig a day at most. Thoughts on either option or what I could have done to keep me out of this hole? I guess I could have gone directly to logstash for everything but as there was no transformation needed at the time, it seemed wasteful. And to be honest, Elasticsearch performed a lot more reliably IME than Logstash. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 14, 2016, 11:00pm UTC](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261/4 "2016-04-14T23:00:13Z")
> [@mhughes](#): > > Ok, is there any way that ES Watcher helps? It could, might be a bit complicated though. > [@mhughes](#): > > 1. awake the ES input every N minutes and re-index everything. If I just limit it to the past 24-48 hours, I think that is feasible. I'm not sending that much data. Maybe a 1 gig a day at most. I'd do that. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:02am UTC](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261/5 "2017-07-06T05:02:10Z")
Closed on July 6, 2017, 5:02am UTC