Hello All,
I am using App search based engines. By Default my crawler uses " app_search_crawler" ingestion pipeline. But for one of the specific engines i want to configure it to use a different ingestion pipeline.. how can i achieve it??
Hello All,
I am using App search based engines. By Default my crawler uses " app_search_crawler" ingestion pipeline. But for one of the specific engines i want to configure it to use a different ingestion pipeline.. how can i achieve it??
Hi @maddy30,
What version of elastic are you using?
You may want to check out this related forum post. Here are a few other resources that might be helpful here.
Hope this helps!
Jess
Hi @maddy30 ,
For App Search, you can't configure different pipelines per crawler.
What you could do is leverage the Elastic Web Crawler (which allows you to "copy and customize" pipelines on an index-by-index basis) and then create an App Search Engine from the resulting Elasticsearch Index. This would be my recommendation, as the Elastic Web Crawler is the newer one, and has had a few feature improvements/additions that are not present in the App Search Crawler.
If that option doesn't work for you, check out. these docs, specifically:
Enterprise Search uses an Elasticsearch ingest pipeline to power the web crawler’s binary content extraction. This pipeline, named
app_search_crawler
, is automatically created when Enterprise Search first starts, if Enterprise Search is configured to enable binary content extraction.You can view and update this pipeline in Kibana or with Elasticsearch APIs.
NOTE: If you make changes to the default
app_search_crawler
ingest pipeline, these will not be overwritten when you upgrade Enterprise Search. Therefore, we recommend comparing the defaultapp_search_crawler
pipeline that comes with the newer version — to determine if you need to incorporate the latest changes into your customized pipeline.
Thanks for your response. i see that we can use different pipelines ar the index level in Elasticsearch.
Thanks for your response @Sean_Story . i see that we can use different pipelines at the index level in Elasticsearch. I have configured it.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.