# Pipeline: copy unique value to a new index

**URL:** <https://discuss.elastic.co/t/pipeline-copy-unique-value-to-a-new-index/322946>\
**Category:** Logstash\
**Created:** [January 11, 2023, 3:19pm UTC](https://discuss.elastic.co/t/pipeline-copy-unique-value-to-a-new-index/322946 "2023-01-11T15:19:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![SalvoDM91](https://avatars.discourse-cdn.com/v4/letter/s/e19adc/32.png) [@SalvoDM91](https://discuss.elastic.co/u/SalvoDM91)\
**Post date:** [January 11, 2023, 3:19pm UTC](https://discuss.elastic.co/t/pipeline-copy-unique-value-to-a-new-index/322946/1 "2023-01-11T15:19:27Z")

</div>

I guys, I need an help!  
I have an index called s1v6\_new\_cleaninq2 with more then 1 million of documents. Inside each documents I have a field called "num\_pratica" and this field could be repeated for a couple of documents.  
Here you can find a screenshot of my "Discover" page:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/a/8aeaf4128f95153ae34c3362dde13edfe61541a5.png)

I'm trying to create a pipeline to copy and paste (in a new index called "s1v6\_new\_cleaninq2\_prtuniche") the documents with unique "num\_pratica".  
So I would like to find only one document for each num\_pratica different value.

This is my pipeline:

```auto
input {
  elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "s1v6_new_cleaninq2"
  }
}

filter {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "s1v6_new_cleaninq2_prtuniche"
    query => "num_pratica:%{num_pratica}"
    add_tag => "not_insert"
  }
}

output {
  if "not_insert" not in [tags] {
    elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "s1v6_new_cleaninq2_prtuniche"
    }
  }
}

```

But I can find several hits for same "num\_pratica" as you can see in the screenshot:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/1/a101ff92404912acd4814a481b78386c44de3b9b.png)

Could you please help me?

Thanks!!  
Salvo

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2023, 3:19pm UTC](https://discuss.elastic.co/t/pipeline-copy-unique-value-to-a-new-index/322946/2 "2023-02-08T15:19:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.