# Logstash 404 errors when dealing with UPDATES after INSERTS

**URL:** <https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608>\
**Category:** Logstash\
**Created:** [August 18, 2015, 6:06pm UTC](https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608 "2015-08-18T18:06:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![crickeys](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickeys/32/4254_2.png) [@crickeys](https://discuss.elastic.co/u/crickeys)\
**Post date:** [August 18, 2015, 6:06pm UTC](https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608/1 "2015-08-18T18:06:12Z")

</div>

I have a logstash config like this:

```
input {
  file {
    path => ["/home/csdata/*.data"]
    codec => json {
      }
    start_position => "beginning"
    discover_interval => 5
 }
}
output{
if [_up] == 1 {
 elasticsearch {
        protocol => "http"
        host => "[myelasticsearchip]"
        cluster => "clustername"
        flush_size => 50
        index => "%{_index}"
        action => "update"
        document_id => "%{_id}"
        index_type => "%{_type}"
        }
}
else if [_id] != "" {
  elasticsearch {
        protocol => "http"
        host => "[myelasticsearchip]"
        cluster => "clustername"
        flush_size => 50
        index => "%{_index}"
        document_id => "%{_id}"
        index_type => "%{_type}"
        }
 }
else{
  elasticsearch {
       protocol => "http"
        host => "[myelasticsearchip]"
        cluster => "clustername"
        index => "%{_index}"
         flush_size => 50
        index_type => "%{_type}"
        }
    }
}

```

I have a ton of

```
failed action with response of 404, dropping action:

```

The data should all be coming into the same file and in order, so things should be created before they are being updated. This doesn't happen with ALL items, but with plenty. I would expect to have none of these errors.

Is this because of the different flush\_sizes? Eventhough the items are in order in the original file, meaning an INSERT always comes before an UPDATE.

Any ideas would be greatly appreciated!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 18, 2015, 11:45pm UTC](https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608/2 "2015-08-18T23:45:36Z")

</div>

A 404 usually means it cannot find the document to update, maybe check your ES logs to see if there is something corresponding there as well.

---

<div class="post-metadata">

**Author:** ![crickeys](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickeys/32/4254_2.png) [@crickeys](https://discuss.elastic.co/u/crickeys)\
**Post date:** [August 18, 2015, 11:50pm UTC](https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608/3 "2015-08-18T23:50:56Z")

</div>

Yes, it definitely can't find it. I'll even check directly in elasticsearch and the item isn't there. But eventually it gets there. I'm more confused about the ordering, because I thought if I put everything in one log file in order it should get to elasticsearch in that order. For example, if this is my log file (simplified):

```
Insert { 'id' : 1, 'content': 'whatever' }
Insert { 'id' : 2, 'content': 'whatever' }
Insert { 'id' : 3, 'content': 'whatever' }
Update { 'id' : 2, 'content': 'whatever is changed' }

```

Since, id: 2 came before I thought it would be inserted and guaranteed to be there for the Update, but it isn't always there. That's why I'm wondering if that has to do with having 2 outputs and different flush sizes. Do they then become independent of each other. And if so, what's the correct way to batch the updates and make sure the inserts have occurred.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 19, 2015, 12:36am UTC](https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608/4 "2015-08-19T00:36:12Z")

</div>

It could be flush size, especially if you are working with changes that are closer together than 50 events.

I'm not really sure what can be done here but I'll give it some thought 🙂

---

<div class="post-metadata">

**Author:** ![crickeys](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickeys/32/4254_2.png) [@crickeys](https://discuss.elastic.co/u/crickeys)\
**Post date:** [August 19, 2015, 4:28am UTC](https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608/5 "2015-08-19T04:28:12Z")

</div>

I wonder if being able to use upsert would help. If I just grouped both the inserts and updates into one update but used doc\_as\_upsert from this pull [https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/116](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/116)

I can't figure out how to get that doc\_as\_upsert running in logstash though ☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:31am UTC](https://discuss.elastic.co/t/logstash-404-errors-when-dealing-with-updates-after-inserts/27608/6 "2017-07-06T05:31:35Z")

</div>