Help deleting data

Elastic Stack Logstash
1

Hi there,

Forgive my ignorance in advance.

I've inherited an old stack with elasticsearch 1.7. Previously, as has been documented in many threads, it was possible to delete logs using a chrome plugin, which has since been disabled.

I'm reluctant to upgrade the stack because I don't know what else I'll need to do and I have no internal resources to draw on. Previously I would delete logstash logs older than a certain date just by changing the date parameter in a query in the plugin console on Chrome. Please could someone give me some help in how I delete logs older than a specific date going forward?

I can't remember the exact syntax but it went something like "DELETE LOGSTASH ...."

Thanks in advance (and apologies once again if this is a dumb question).
Keith

2

This seems more appropriate for the ElasticSeach subforum :slight_smile:

Anyhow, delete-by-query is probably what you're looking for, where you can query for all documents older than a specific date and delete them.

3

Thanks @paz,

Apologies for mis-posting. I take it that delete-by-query can only be invoked from the command line?

4

I'd assume you can also run it via the head plugin, though I have never tested so myself.

5

Thanks @paz ,

Apologies once more for my ignorance. what plugin are you referring to? Is it a Chrome plugin?

6

No worries. It's this one. Depending on your ElasticSearch version you can either install it as an ElasticSearch plugin or a standalone version.

If with "Chrome plugin" you mean this, it's practically the same thing as above wrapped in a browser extension, so you can use that indeed.

1 Like
7

Thanks @paz,

Sorted now. Installed the plugin. It took me a bit to get my head round it. I couldn't find any docs for the chrome plugin, so was trying all sorts of complicated structured queries, when I stumbled upon the delete option in the Actions dropdown. D'Oh!

Happy now! :grinning:

8

Oh wow. Delete by query is a horrible idea for time series data. It's like the difference in SQL between DELETE from TABLE where timestamp < x and DROP TABLE. Millions of atomic operations in one vs. a single operation for the other.

Use Elasticsearch Curator to delete time series indices on a schedule.

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.