# How to replace multiple new lines with one in Ingest Pipeline gsub **URL:** \ **Category:** Elasticsearch\ **Tags:** ingest-pipeline\ **Created:** [December 29, 2023, 12:39pm UTC](https://discuss.elastic.co/t/how-to-replace-multiple-new-lines-with-one-in-ingest-pipeline-gsub/350127 "2023-12-29T12:39:05Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Bowfish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bowfish/32/130394_2.png) [@Bowfish](https://discuss.elastic.co/u/Bowfish)\ **Post date:** [December 29, 2023, 12:39pm UTC](https://discuss.elastic.co/t/how-to-replace-multiple-new-lines-with-one-in-ingest-pipeline-gsub/350127/1 "2023-12-29T12:39:05Z")
I want to replace multiple new lines (\n\n+) with one single new line (\n) with a gsub processor in the ingest pipeline. This is my gsub processor: ```auto { "gsub": { "field": "attachment.content_processed", "pattern": "\\n\\n", "replacement": "\\n", "ignore_missing": true, "if": "ctx?._replace_bullets == true", "tag": "replace_bullets", "description": "Replace multiple newlines", "on_failure": [ { "append": { "description": "Record error information", "field": "_ingestion_errors", "value": "Processor 'gsub' with tag 'remove_page_numbers' in pipeline '{{ _ingest.on_failure_pipeline }}' failed with message '{{ _ingest.on_failure_message }}'" } } ] } }, ``` Like this it replaces \n\n with n and not \n. What is the correct "replacement" string? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [December 29, 2023, 4:14pm UTC](https://discuss.elastic.co/t/how-to-replace-multiple-new-lines-with-one-in-ingest-pipeline-gsub/350127/2 "2023-12-29T16:14:24Z")
Hi @Bowfish The best way for use to help... It is to provide a JSON document with the actual (anonymized) data / `attachment.content_processed,` string and the desired output. Or at least a representative sample ```auto POST _ingest/pipeline/_simulate { "pipeline": { "processors": [ { "gsub": { "field": "message", "pattern": "\\\n", "replacement": "", "ignore_missing": false, "description": "Replace multiple newlines", "on_failure": [ { "append": { "description": "Record error information", "field": "_ingestion_errors", "value": "Processor 'gsub' with tag 'remove_page_numbers' in pipeline '{{ _ingest.on_failure_pipeline }}' failed with message '{{ _ingest.on_failure_message }}'" } } ] } } ] }, "docs": [ { "_source": { "message": """String with more than 1 newline """ } } ] } # Result { "docs": [ { "doc": { "_index": "_index", "_version": "-3", "_id": "_id", "_source": { "message": "String with more than 1 newline " }, "_ingest": { "timestamp": "2023-12-29T16:27:48.349457039Z" } } } ] } ``` ---
**Author:** ![Bowfish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bowfish/32/130394_2.png) [@Bowfish](https://discuss.elastic.co/u/Bowfish)\ **Post date:** [December 29, 2023, 4:45pm UTC](https://discuss.elastic.co/t/how-to-replace-multiple-new-lines-with-one-in-ingest-pipeline-gsub/350127/3 "2023-12-29T16:45:49Z")
> [@stephenb](#): > > ```auto > { > "docs": [ > { > "doc": { > "_index": "_index", > "_version": "-3", > "_id": "_id", > "_source": { > "message": "String with more than 1 newline " > }, > "_ingest": { > "timestamp": "2023-12-29T16:27:48.349457039Z" > } > } > } > ] > } > > ``` The result of your example should be: ```auto { "docs": [ { "doc": { "_index": "_index", "_version": "-3", "_id": "_id", "_source": { "message": "String with more than 1 newline " }, "_ingest": { "timestamp": "2023-12-29T16:38:18.121740582Z" } } } ] } ``` If there are more than one newlines in the string it should substitute the multiple newlines with one newline. If there is only one newline it should keep it. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [December 29, 2023, 4:57pm UTC](https://discuss.elastic.co/t/how-to-replace-multiple-new-lines-with-one-in-ingest-pipeline-gsub/350127/4 "2023-12-29T16:57:34Z")
> [@Bowfish](#): > > If there are more than one newlines in the string it should substitute the multiple newlines with one newline. If there is only one newline it should keep it. Now it is just a regex exercise I am not a regex expert... but I did a quick google search found [this](https://stackoverflow.com/a/701658) ```auto POST _ingest/pipeline/_simulate { "pipeline": { "processors": [ { "gsub": { "field": "message", "pattern": "\\\n+", "replacement": "\\\n", "ignore_missing": false, "description": "Replace multiple newlines", "on_failure": [ { "append": { "description": "Record error information", "field": "_ingestion_errors", "value": "Processor 'gsub' with tag 'remove_page_numbers' in pipeline '{{ _ingest.on_failure_pipeline }}' failed with message '{{ _ingest.on_failure_message }}'" } } ] } } ] }, "docs": [ { "_source": { "message": """String with 1 newline Then 2 newlines Then 3 Newlines The End """ } } ] } # Result { "docs": [ { "doc": { "_index": "_index", "_version": "-3", "_id": "_id", "_source": { "message": """String with 1 newline Then 2 newlines Then 3 Newlines The End """ }, "_ingest": { "timestamp": "2023-12-29T16:58:37.484845559Z" } } } ] } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 26, 2024, 4:57pm UTC](https://discuss.elastic.co/t/how-to-replace-multiple-new-lines-with-one-in-ingest-pipeline-gsub/350127/5 "2024-01-26T16:57:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.