# Creating a pipeline which runs with particular interval on same index

**URL:** <https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584>\
**Category:** Kibana\
**Created:** [June 1, 2021, 6:25am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584 "2021-06-01T06:25:14Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![aswanth\_jabba](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\
**Post date:** [June 1, 2021, 6:25am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/1 "2021-06-01T06:25:14Z")

</div>

Can anyone please tell me, if this is possible ?

I am trying to create a pipeline but I want it to run in particular intervals on the same index

For example :

```auto
{
  "description":"Find the Progress",
  "processors":[
    {
      "set":{
        "field":"_source.now",
        "value":"{{_ingest.timestamp}}"
      }
    },

    {
      "script":{
        "lang":"painless",
        "source": """
        ctx['SubmitDate']=ctx['SubmitDate'].substring(0,10)+'T'+ctx['SubmitDate'].substring(11,16)+':00Z';
        if(ctx.containsKey("now") && ctx.containsKey("SubmitDate"))
          {
            ctx['daysleft']=ChronoUnit.DAYS.between(ZonedDateTime.parse(ctx['now']),ZonedDateTime.parse(ctx['SubmitDate']));
        if(ctx['daysleft']>0)
          {
            ctx['Progress']="In Progress";
          }
        if(ctx['daysleft']<=0)
          {
            ctx['Progress']="Completed";
          }
        }
        """
      }
    }
  ]
}

```

In the above example I want to refresh the Progress field for every day using some calculation.

I am not able find about this any where, I am just curious if that's possible using pipeline and also , I don't know if this is the right way to refresh the fields with some calculation.

Please suggest any methods to perform such calculation and refresh.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [June 1, 2021, 8:09am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/2 "2021-06-01T08:09:41Z")

</div>

Hey,

a pipeline is only executed, before a document is indexed, so you would need to run an update by query and reindex all documents. I don't think that is the best approach, so maybe, if you elaborate (without talking about any Elasticsearch/Kibana features), why you need this and what sense this makes for you, maybe we can come up with another solution.

Thanks!

--Alex

---

<div class="post-metadata">

**Author:** ![aswanth\_jabba](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\
**Post date:** [June 1, 2021, 8:35am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/3 "2021-06-01T08:35:54Z")

</div>

Hi Alex,

I have a field called "SubmitDate" and I want to find the difference between the submit date and present date which should be stored in a new variable "daysleft".

Now, I want to see the latest updated value of daysleft whenever I want.

Thanks..

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [June 1, 2021, 8:48am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/4 "2021-06-01T08:48:01Z")

</div>

Hey,

how about a scripted field or a runtime field? See [Retrieve selected fields from a search | Elasticsearch Guide [7.13] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.13/search-fields.html#script-fields) or more recent [Runtime fields | Elasticsearch Guide [7.13] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.13/runtime.html) (which can also be configured in Kibana in the most recent release) and used for searches.

--Alex

---

<div class="post-metadata">

**Author:** ![aswanth\_jabba](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\
**Post date:** [June 1, 2021, 9:03am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/5 "2021-06-01T09:03:22Z")

</div>

Hi Alex,

Thanks for guiding me through this, What should I be using in the case of Index pattern ?

I think Runtime fields would be perfect, does that work with index pattern ?

Thanks

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [June 1, 2021, 9:14am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/6 "2021-06-01T09:14:22Z")

</div>

This should help: [Manage index pattern data fields | Kibana Guide [7.13] | Elastic](https://www.elastic.co/guide/en/kibana/7.13/managing-index-patterns.html)

---

<div class="post-metadata">

**Author:** ![aswanth\_jabba](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\
**Post date:** [June 1, 2021, 9:22am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/7 "2021-06-01T09:22:45Z")

</div>

Hi Alex,

Thank you so much for helping me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 29, 2021, 9:23am UTC](https://discuss.elastic.co/t/creating-a-pipeline-which-runs-with-particular-interval-on-same-index/274584/8 "2021-06-29T09:23:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.