# Specifying pipeline as a query string parameter in bulk actions

**URL:** <https://discuss.elastic.co/t/specifying-pipeline-as-a-query-string-parameter-in-bulk-actions/78498>\
**Category:** Elasticsearch\
**Created:** [March 14, 2017, 11:05am UTC](https://discuss.elastic.co/t/specifying-pipeline-as-a-query-string-parameter-in-bulk-actions/78498 "2017-03-14T11:05:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bernt\_Rostad](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@Bernt\_Rostad](https://discuss.elastic.co/u/Bernt_Rostad)\
**Post date:** [March 14, 2017, 11:05am UTC](https://discuss.elastic.co/t/specifying-pipeline-as-a-query-string-parameter-in-bulk-actions/78498/1 "2017-03-14T11:05:34Z")

</div>

I'm struggling to pass documents through an Ingest pipeline during bulk indexing operations and wonder if anyone else have made this work properly using the CPAN Search::Elasticsearch perl framework, which I use for all Elasticsearch operations (from index creation to document indexing and searches).

For bulk indexing I'm using the bulk\_helper() method to get a bulk instance which allows me to call add\_action() for every document I want to index in one bulk operation:

my $es = Search::Elasticsearch-\>new( %host\_conf );  
my $bulk = $es-\>bulk\_helper(  
%bulk\_conf,  
on\_success =\> sub { ... },  
on\_error =\> sub { ... } );

for my $d (@docs) {  
$bulk-\>add\_action( index =\> { index=\>$d-\>{index}, type=\>$d-\>{type}, source=\>$d-\>{body} } );  
}

The perl documentation says "Each action can include the same parameters that you would pass to the equivalent "index() ..." so I looked up the regular index() method docs which lists "Query string parameters: op\_type, parent, pipeline, ...". So it seems I can add a pipeline query string parameter to the bulk operation. But where do I do that? I've already tried

$es-\>bulk\_helper( ..., pipeline=\>$my\_pipe ); # fails silently  
$bulk-\>add\_action( index=\>{ ... }, pipeline=\>$my\_pipe ); # throws "Unrecognised action" exception  
$bulk-\>add\_action( index=\>{ ... , pipeline=\>$my\_pipe } ); # throws "Unknown params " exception

I'm now running out of ideas on how to get pipeline added to the bulk operation.

Refs:  
[http://search.cpan.org/dist/Search-Elasticsearch/lib/Search/Elasticsearch/Client/5\_0/Direct.pm#bulk()](http://search.cpan.org/dist/Search-Elasticsearch/lib/Search/Elasticsearch/Client/5_0/Direct.pm#bulk())  
[http://search.cpan.org/~drtech/Search-Elasticsearch-5.01/lib/Search/Elasticsearch/Client/5\_0/Direct.pm#index()](http://search.cpan.org/~drtech/Search-Elasticsearch-5.01/lib/Search/Elasticsearch/Client/5_0/Direct.pm#index())

---

<div class="post-metadata">

**Author:** ![Bernt\_Rostad](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@Bernt\_Rostad](https://discuss.elastic.co/u/Bernt_Rostad)\
**Post date:** [March 17, 2017, 12:35pm UTC](https://discuss.elastic.co/t/specifying-pipeline-as-a-query-string-parameter-in-bulk-actions/78498/2 "2017-03-17T12:35:39Z")

</div>

After running with the Perl debugger and curl-level trace-logging I must conclude that the bulk\_helper API doesn't support pipelines, instead I'll have to use the regular bulk() method and implement my own flushing strategy.

This is not ideal as I rely heavily on the on\_success() and on\_error() methods but it will have to do for now.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 14, 2017, 12:35pm UTC](https://discuss.elastic.co/t/specifying-pipeline-as-a-query-string-parameter-in-bulk-actions/78498/3 "2017-04-14T12:35:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.