Help deleting data

Hi there,

Forgive my ignorance in advance.

I've inherited an old stack with elasticsearch 1.7. Previously, as has been documented in many threads, it was possible to delete logs using a chrome plugin, which has since been disabled.

I'm reluctant to upgrade the stack because I don't know what else I'll need to do and I have no internal resources to draw on. Previously I would delete logstash logs older than a certain date just by changing the date parameter in a query in the plugin console on Chrome. Please could someone give me some help in how I delete logs older than a specific date going forward?

I can't remember the exact syntax but it went something like "DELETE LOGSTASH ...."

Thanks in advance (and apologies once again if this is a dumb question).

This seems more appropriate for the ElasticSeach subforum :slight_smile:

Anyhow, delete-by-query is probably what you're looking for, where you can query for all documents older than a specific date and delete them.

Thanks @paz,

Apologies for mis-posting. I take it that delete-by-query can only be invoked from the command line?

I'd assume you can also run it via the head plugin, though I have never tested so myself.

Thanks @paz ,

Apologies once more for my ignorance. what plugin are you referring to? Is it a Chrome plugin?

No worries. It's this one. Depending on your ElasticSearch version you can either install it as an ElasticSearch plugin or a standalone version.

If with "Chrome plugin" you mean this, it's practically the same thing as above wrapped in a browser extension, so you can use that indeed.

1 Like

Thanks @paz,

Sorted now. Installed the plugin. It took me a bit to get my head round it. I couldn't find any docs for the chrome plugin, so was trying all sorts of complicated structured queries, when I stumbled upon the delete option in the Actions dropdown. D'Oh!

Happy now! :grinning:

Oh wow. Delete by query is a horrible idea for time series data. It's like the difference in SQL between DELETE from TABLE where timestamp < x and DROP TABLE. Millions of atomic operations in one vs. a single operation for the other.

Use Elasticsearch Curator to delete time series indices on a schedule.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.