# How to store & get data with different time interval

**URL:** <https://discuss.elastic.co/t/how-to-store-get-data-with-different-time-interval/252775>\
**Category:** Logstash\
**Created:** [October 21, 2020, 6:04am UTC](https://discuss.elastic.co/t/how-to-store-get-data-with-different-time-interval/252775 "2020-10-21T06:04:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![James\_Liu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_liu/32/77509_2.png) [@James\_Liu](https://discuss.elastic.co/u/James_Liu)\
**Post date:** [October 21, 2020, 6:04am UTC](https://discuss.elastic.co/t/how-to-store-get-data-with-different-time-interval/252775/1 "2020-10-21T06:04:30Z")

</div>

Our team are currently focusing on log datas and visualizing them.  
We are building a platform analyzing every audit log by log4j in our java project:

log -\> filebeat -\> logstash -\> elasticsearch -\> our platform

CURRENT TECHNIQUE STACK:

- filebeat - 7.9.1
- logstash -7.9.1
- elasticsearch -7.9.1
- Java - HighLevelRestClient

When in visualizing I had some problems. We have some log datas, and we need to visualize them in different time intervals, like recent 15 min, recent 30 min, recent 1 hour.

```java
searchSourceBuilder
            .fetchSource(new String[]{"data_size_in","data_size_out", "server_id"},null)
            .query(QueryBuilders.rangeQuery("start_time")
                .lte("2020-09-25T02:50:00.000Z")
                .gte("2020-09-30T01:50:00.000Z"))
            .query(QueryBuilders.matchQuery("gateway_code","/memcached"))
            .aggregation(AggregationBuilders.dateHistogram("interval")
                .fixedInterval(DateHistogramInterval.DAY).field("start_time")
                .subAggregation(AggregationBuilders.terms("function")
                    .size(5).field("pub_item_name.keyword"))
            );

```

Currently we are solving by aggregation with intervals, and change the interval based on time range we choose. Is there some more elegant solutions?

Here's my improvement thought:

1. create more index with certain intervals(maybe use elasticsearch pipelines)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 18, 2020, 6:04am UTC](https://discuss.elastic.co/t/how-to-store-get-data-with-different-time-interval/252775/2 "2020-11-18T06:04:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.