# Multiline not working properly when used along with exclude\_lines **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [September 3, 2020, 4:24pm UTC](https://discuss.elastic.co/t/multiline-not-working-properly-when-used-along-with-exclude-lines/247422 "2020-09-03T16:24:37Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![MANI\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mani_m/32/73784_2.png) [@MANI\_M](https://discuss.elastic.co/u/MANI_M)\ **Post date:** [September 3, 2020, 4:24pm UTC](https://discuss.elastic.co/t/multiline-not-working-properly-when-used-along-with-exclude-lines/247422/1 "2020-09-03T16:24:37Z")
We were trying to combine all the logs into a single event after excluding lines with a specific pattern. But it seems `filebeat` is appending the excluded lines in multiline and misses out other data leading to misbehavior. Not sure whether the pattern I've given is wrong. There is an [issue related to this in github](https://github.com/elastic/beats/issues/1940) but it is closed without proper explanation. As per [this answer](https://discuss.elastic.co/t/exclude-data-before-multiline/60978/2) by @ruflin, is it because exclude\_lines is a processor and multiline is a reader? We use filebeat-7.5.2 which is running on kubernetes 1.18.8 Here is the kubectl configmap for filebeat.yaml: ```auto apiVersion: v1 data: filebeat.yaml: |- filebeat.inputs: - type: log enabled: true paths: - /logs/*.log exclude_lines: ['^Microsoft\.Data\.Tools\.Diagnostics\.Tracer (Warning|Verbose).*'] multiline: pattern: '.*' negate: false match: after output.console: pretty: true filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: false setup.template.settings: index.number_of_shards: 1 setup.ilm.enabled: auto setup.ilm.rollover_alias: "filebeat" setup.ilm.pattern: "{now/d}-000001" processors: - add_host_metadata: ~ - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~ - add_fields: target: event fields: dataset: "sql-job" kind: ConfigMap metadata: creationTimestamp: "2020-08-28T11:51:28Z" name: sql-job-filebeat namespace: test ``` Example input: ```auto Microsoft.Data.Tools.Diagnostics.Tracer Warning: 0 : 2020-09-03T15:23:37 : SqlStudioException.TraceException(Exception) Exception: System.ArgumentException: The property Microsoft.Data.Tools.Schema.Sql.CommandLine.CommandLineResources specified is not present in the resource AdditionalDeploymentContributorPaths_FriendlyName. Parameter name: resourcePropertyName Publishing to database 'devtestSystem' on server '1.2.3.4'. Microsoft.Data.Tools.Diagnostics.Tracer Information: 0 : 2020-09-03T15:23:37 : Initialized ModelSchema - elements 149 Microsoft.Data.Tools.Diagnostics.Tracer Information: 19 : 2020-09-03T15:23:37 : IsolatedStorageFile probe success Microsoft.Data.Tools.Diagnostics.Tracer Warning: 0 : 2020-09-03T15:23:37 : Logging Ambient Settings... Microsoft.Data.Tools.Diagnostics.Tracer Warning: 0 : 2020-09-03T15:23:37 : DatabaseLockTimeout: 60000 Microsoft.Data.Tools.Diagnostics.Tracer Verbose: 0 : 2020-09-03T15:23:37 : Loading collation id 1078 af-ZA Microsoft.Data.Tools.Diagnostics.Tracer Verbose: 1 : 2020-09-03T15:24:13 : ColumnResolver: #47626729 ResolvedDescriptor for column: ResolvedDescriptor: [tableInsID] Potentials(3): [SqlColumn : P1 - [dbo.tablexyzIns.tableInsID]] [SqlColumn : P1 - [dbo.tablexyzIns.tables]::PtableInsID] [SqlColumn : P1 - [dbo.tableIns.tables]::PtableInsID] Microsoft.Data.Tools.Diagnostics.Tracer Verbose: 1 : 2020-09-03T15:24:13 : ColumnResolver: #32559816 Merging source without existing source, ExposedName = tables Microsoft.Data.Tools.Diagnostics.Tracer Verbose: 1 : 2020-09-03T15:24:13 : ColumnResolver: #32559816 Added source:^M ExposedName = tables^M Target=(ISqlColumnSource)dbo.tablexyzIns^M Affinity = 1^M AmbiguousSchema = False^M IsCTE = False^M IsMirror = False Microsoft.Data.Tools.Diagnostics.Tracer Verbose: 1 : 2020-09-03T15:24:13 : ColumnResolver: #20099428 ResolvedDescriptor for column: ResolvedDescriptor: [tableExecuteModeID] Potentials(7): [SqlColumn : P1 - [dbo.tablexyzIns.tableIns]::PtableExecuteModeID] [SqlColumn : P1 - [dbo.tableIns.tableExecuteModeID]] [SqlColumn : P1 - [dbo.tableIns.tableIns]::PtableExecuteModeID] [SqlColumn : P1 - [dbo.tableDef.tableIns]::PtableExecuteModeID] [SqlColumn : P1 - [TinyEnum.LockedStatus.tableIns]::PtableExecuteModeID] [SqlColumn : P1 - [TinyEnum.VisibilityStatus.tableIns]::PtableExecuteModeID] [SqlColumn : P1 - [TinyEnum.tableExecuteMode.tableIns]::PtableExecuteModeID] Microsoft.Data.Tools.Diagnostics.Tracer Verbose: 0 : 2020-09-03T15:24:50 : Executing Step 16, Not Tracked, Type 'CreateElementStep', Section 'User', Operation '1', Ignorable Errors 'None' (SqlCheckConstraint), Script is as follows: IF @@ERROR <> 0 AND @@TRANCOUNT > 0 BEGIN ROLLBACK; END IF @@TRANCOUNT = 0 BEGIN INSERT INTO #tmpErrors (Error) VALUES (1); BEGIN TRANSACTION; END Microsoft.Data.Tools.Diagnostics.Tracer Verbose: 0 : 2020-09-03T15:24:50 : Executing Step 17, Not Tracked, Type 'SqlEndTransactionStep', Section 'User', Script is as follows: IF EXISTS (SELECT * FROM #tmpErrors) ROLLBACK; GO IF @@TRANCOUNT > 0 BEGIN PRINT N'The transacted portion of the database update succeeded.'; COMMIT TRANSACTION; END ELSE PRINT N'The transacted portion of the database update failed.'; Available Column Sources: ColumnSource tables (affinity = 1) with no known columns ColumnSource tableIns (affinity = 1) with no known columns ColumnSource tableDef (affinity = 1) with no known columns ColumnSource LockedStatus (affinity = 1) with no known columns ColumnSource VisibilityStatus (affinity = 1) with no known columns ColumnSource tableExecuteMode (affinity = 1) with no known columns ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 1, 2020, 6:24pm UTC](https://discuss.elastic.co/t/multiline-not-working-properly-when-used-along-with-exclude-lines/247422/2 "2020-10-01T18:24:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.