# Include\_lines for pubsub input isn't working: no filtering

**URL:** <https://discuss.elastic.co/t/include-lines-for-pubsub-input-isnt-working-no-filtering/381036>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 14, 2025, 9:33am UTC](https://discuss.elastic.co/t/include-lines-for-pubsub-input-isnt-working-no-filtering/381036 "2025-08-14T09:33:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![blankoworld](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/blankoworld/32/144630_2.png) [@blankoworld](https://discuss.elastic.co/u/blankoworld)\
**Post date:** [August 14, 2025, 9:33am UTC](https://discuss.elastic.co/t/include-lines-for-pubsub-input-isnt-working-no-filtering/381036/1 "2025-08-14T09:33:19Z")

</div>

Hi everyone,

Today I’m encounting some problems using Filebeat with the Pubsub input.

I want to fetch data from a PubSub (from GCP) and then push it in Kafka. All is OK for that.  
Nevertheless I need to filter lines to only take those that have a “textPayload” that starts with a date.

I so use this configuration:

```yaml
filebeat.inputs:
  - type: gcp-pubsub
    project_id: blahblah
    topic: mytopic
    subscription.name: mysub
    credentials_json: {somecredentials}
    include_lines: ['something']
    fields_under_root: true
    fields:
      type_log: GCP
  - type: gcp-pubsub
    project_id: blahblah
    topic: mysecondtopic
    subscription.name: mysecondsub
    credentials_json: {somecredentials}
    fields_under_root: true
    fields:
      type_log: NGINX

filebeat.registry.path: /usr/share/filebeat/data/registry
name: SuperName
max_procs: 1
fields_under_root: true
fields:
  namespace: mynamespace
  topic: TOPIC_NAME_FOR_REFERENCE

output.kafka:
  enabled: true
  hosts: ["my.kafka.com:12345"]
  topic: '%{[topic]}'
  max_message_bytes: 5000000
  compression: gzip
  ssl.enabled: true
  ssl.renegotiation: freely
  ssl.certificate_authorities: ["/path/to/file.pem"]
  required_acks: 1
  partition.round_robin:
    reachable_only: false

logging.level: info
logging.to_stderr: true
logging.metrics.enabled: false

```

The problem is: even if I change include\_lines content ALL lines comes into Kafka. ALL Lines 😥 .

Have you any suggestions? Does incluces\_lines really works for gcp-pubsub module? Does this module be broken?

I use filebeat 8.17.

Thanks in advance for any help 🙏

---

<div class="post-metadata">

**Author:** ![Tortoise](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tortoise/32/147587_2.png) [@Tortoise](https://discuss.elastic.co/u/Tortoise)\
**Post date:** [August 14, 2025, 11:17am UTC](https://discuss.elastic.co/t/include-lines-for-pubsub-input-isnt-working-no-filtering/381036/2 "2025-08-14T11:17:24Z")

</div>

Hello @blankoworld

Welcome to the Community!!

As per the documentation :

> **[GCP Pub/Sub input | Filebeat Reference \[8.17\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/8.17/filebeat-input-gcp-pubsub.html)**

It has processor for filtering :

> **[Filter and enhance data with processors | Filebeat Reference \[8.17\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/8.17/filtering-and-enhancing-data.html)**

```auto
processors:
  - drop_event:
      when:
        regexp:
          message: "^DBG:"

```

I am not sure if because of this it is sending all the data & not considering this parameter include\_lines.

Thanks!!