# Datafeed \[datafeed-packetbeat\_dns\_tunneling\] cannot retrieve data because no index matches datafeed's indices \[packetbeat-\*\]

**URL:** <https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356>\
**Category:** Kibana\
**Tags:** elastic-stack-machine-learning\
**Created:** [May 25, 2023, 5:27pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356 "2023-05-25T17:27:29Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [May 25, 2023, 5:27pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/1 "2023-05-25T17:27:30Z")

</div>

Hi folks  
I have been searching high and low regarding this error and I am very new to the ELK stack.

ELK 8.7.1, using Elastic Agents on clients.

When I try to enable the ML job packetbeat\_dns\_tunneling it fails with the above error messages.  
In general every ML job that involves the packetbeat -\* indices fails to be created/run.

I did add the packetbeat integration to a policy, deployed to + 50 agents and the index is there, but I am clueless now as what to do about the above error.

Can someone point me in a direction for further investigation here

Cheers

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [May 29, 2023, 10:59am UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/2 "2023-05-29T10:59:58Z")

</div>

Hi guys

do I have to set the 'index: packetbeat' in the logstash configuration here ?

..... is there a 'fast lane' here for questions, we have an Enterprise License, but I would rather have everybody benefiting from this discussion/issue here.

Cheers

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [June 10, 2023, 2:52pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/3 "2023-06-10T14:52:45Z")

</div>

anyone ?

From what I can read from the documentation, an elastic agent will automatically create all the beats indexes automatically.

packetbeat-\* is not created and hence my ML jobs fails (those involving packetbeat-\*)

so what todo here ?

my 2 logstash servers in front of elasticsearch have the input filter elastic\_agent specified and the elasticsearch output filter have the data\_stream =\> true (+ ssl, apikey etc.)

can someone help me troubleshoot this, where to begin 🤷‍♂️.  
🧐

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 10, 2023, 3:28pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/4 "2023-06-10T15:28:32Z")

</div>

You really should open a support ticket.

This forum is certainly not a fast lane. It might be sometimes but other times not... There are no SLAs here 🙂

> [@A113n](#):
>
> deployed to + 50 agents and the index is there,

Can you show the index? Just because it index is there does not mean the data is in it.

Second, I think you're leaving out some key pieces. It sounds like you're trying to use agent through logstash. Is that correct?

If so, you need to share your agent configuration and logstash configuration?

Find and show where you believe the packetbeat data is ending up.

Worse Comes to worse. You Can just clone that ML job or GET it and edit where it's getting the data from..

It Looks what you're trying to do should work according to the docs So show us where the data is ending up? You might just need that edit the ML job. Maybe it hasn't been updated for agent yet

> Events indexed into Elasticsearch with the Logstash configuration shown here will be similar to events directly indexed by Elastic Agent into Elasticsearch.

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [June 11, 2023, 10:34am UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/5 "2023-06-11T10:34:18Z")

</div>

Hi Stephen  
thank you for your reply here, appreciate it.

> [@stephenb](#):
>
> Can you show the index? Just because it index is there does not mean the data is in it.

> [@A113n](#):
>
> packetbeat-\* is not created and hence my ML jobs fails (those involving packetbeat-\*)

No sorry Stephen, I meant the logs-elastic\_agent.packetbeat-default is there, but the packetbeat-\* is not.

> [@stephenb](#):
>
> Second, I think you're leaving out some key pieces. It sounds like you're trying to use agent through logstash. Is that correct?

That is correct, I did not explicit state this, I am new to all this and figured the L in ELK was enough, sorry this was not clear.  
I have 2 logstash servers in front of Elasticsearch. Elastic Agents deployed with various integrations, including the packetbeat (renamed to Network Capture I believe).  
Nothing have been done on the agent configuration, it runs the default agent config. The same applies to the logstash.yml, I only changed it to support queues (memory-\>persistent), **nothing** else have been changed. Perhaps I need to run a plugin, the docs does indicate I don't have to do anything special here, but 🤷‍♂️

The output filter in Fleet UI have been configured to Logstash and I have added support for loadbalancer and 4 workers. This is reflected on the agent side running an inspect.  
Literally everything else is runnig as expected(ML jobs, security rules), except for the packetbeat index not being created.

> [@stephenb](#):
>
> Maybe it hasn't been updated for agent yet

how can I verify this ? api, some version setting ? I made sure 8.7.1 across the stack.

> [@stephenb](#):
>
> Find and show where you believe the packetbeat data is ending up.

I stopped believing 🙂 I can not show you this as I have no idea where data is ending up, should they actually arrive, but suggestions are welcome on how to diagnose this further.

> [@stephenb](#):
>
> ou Can just clone that ML job or GET it and edit where it's getting the data from..

That is not possible either as this results in an error messages telling me packetbeat-\* is missing 😂

I am looking at my logstash filter, this is configured to handle data\_streams, I was wondering if I have to create one more elasticsearch output specifying the :

`index => "%{[@metadata][beat]}-%{[@metadata][version]}`  
?

> [@stephenb](#):
>
> You really should open a support ticket.

support ticket sure, last resort.

My main beef with this is, 'most' topics in this forum seems to end with a "DM" or "raise a support ticket". All this is great for the individual having a problem, but what about people with similar problems, the are left behind. All I am saying is, it should be accessible to everybody.

Anyway I not here to change anything, be gentle 🙂 I am new to all this.

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [June 11, 2023, 12:23pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/6 "2023-06-11T12:23:49Z")

</div>

Stephen I have an update,

I followed this guide [here](https://www.elastic.co/guide/en/beats/packetbeat/8.7/packetbeat-template.html).

This created the index ✔, but now I see no data is coming in.

This can only mean either Logstash or the agents are configured wrong ? (right 🤦‍♂️)  
so far my bet is on the logstash filter. The jury is still out on the extra output filter suggested in my previous reply.

Do you really want me to post the massive output from elastic-agent inspect 😁, the logstash.yml is the default configuration with the exception of the persistant modification.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 11, 2023, 1:33pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/7 "2023-06-11T13:33:39Z")

</div>

There is some confusion in this.

First, the Elastic Agent will store data on data sreams where the name of the data stream starts with `logs-*`.

Since you are using the Network Packet Capture integration, it will save the logs in a data stream with `network_traffic` in its name, according to the [documentation](https://docs.elastic.co/integrations/network_traffic).

Do you have a similar data stream in your Cluster?

Independent of the name of the data stream, using the Elastic Agent you will not have any data stream named `packetbeat-*`, those would be created only using the stand-alone packetbeat instalation.

Second, when using the Elastic Agent Elastic expects that you will send the data directly to Elasticsearch, when you add Logstash between them things can get a little more complicated, it works, but you need to configure it like indicated in the [documentation](https://www.elastic.co/guide/en/fleet/current/logstash-output.html#logstash-output).

For example, you should use the `elastic_agent` input and your output will basically only have the hosts and `data_stream` set to `true`.

Also, I'm not 100% sure, but if I'm not wrong the `index` option is ignored when you use data\_streams, it doesn't matter what you put there, what matters are the `data_stream_*` settings, which you also should not have when using the elastic agent as this will come from the agent itself.

You should share your logstash configuration, or at least the inputs and outputs you are using.

And Third, from what I saw here I don't think that this issue is at your side, but at Elastic side.

The ML job _packetbeat\_dns\_tunneling_ is a built-in job, and this job is configured to look at the `packetbeat-*` indices/data streams, but those indices/data streams are not created by the Elastic Agent, only by the stand-alone packetbeat, so the job needs to be updated by Elastic or a new one needs to be created.

You should open a support ticket to check on this.

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [June 11, 2023, 2:42pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/8 "2023-06-11T14:42:06Z")

</div>

Hi Leandro, thanks for contributing here.

> [@leandrojmp](#):
>
> There is some confusion in this.

yes absolutely and it begins with me thinking the "old" packetbeat was [integrated](https://github.com/elastic/integrations/issues/1771) with the Network Packet Capture integration. I believe I've read it was orginally called packetbeat then renamed to Network Packet Capture.

> [@leandrojmp](#):
>
> Do you have a similar data stream in your Cluster?

✔

> [@leandrojmp](#):
>
> Independent of the name of the data stream, using the Elastic Agent you will not have any

I kinda get that now 😂

> [@leandrojmp](#):
>
> Second, when using the Elastic Agent Elastic expects that you will send the data directly to Elasticsearch, when you add Logstash between them things can get a little more complicated, it works,

This works fine for me!.

I could not find documentation stating Agents are expected to talk directly to Elasticsearch.  
From a security pov this is naive/bad, having a jwt token and a direct endpoint in your agent config telling you where to abuse Elasticsearch is and should not be considered best practices.

Leandro if you have any other solutions here that could shut up the security guys, please chip in 🙏.

> [@leandrojmp](#):
>
> For example, you should use the `elastic_agent` input and your output will basically only have the hosts and `data_stream` set to `true`.

✔ this has been working from day 1.

> [@leandrojmp](#):
>
> Also, I'm not 100% sure, but if I'm not wrong the `index` option is ignored when you use data\_streams,

idk 🤷‍♂️, which is why I suggested a 2nd output filter in logstash using this behaviour.  
At the moment it doesnt matter as we have established Network Packet Caputure does not create the packetbeat index.

> [@leandrojmp](#):
>
> what matters are the `data_stream_*` settings, which you also should not have when using the elastic agent as this will come from the agent itself

not sure I follow here Leandro, when you're using logstash I have to specify at least the elastic\_agent and the data\_stream\>=true, can't say much about other data\_stream\_\* settings, I did not explict configure this in my logstash ... or I am missing your point.  
see here:

```auto
input {, 
  elastic_agent {
    port => 5044
    ssl => true
    ssl_certificate_authorities => [".."]
    ssl_certificate => ".."
    ssl_key => ".."
    ssl_verify_mode => "force_peer"
  }
}

output {
  elasticsearch {
    hosts => "ip:9200"
    api_key => "..."
    data_stream => true
    ssl => true
    cacert => "path to crt"
  }
}

```

Initially I figured I could just add another elasticsearch output with the index specified, again this is irrelevant as the Network Package Capture" integration does not create the packetbeat index.

> [@leandrojmp](#):
>
> only by the stand-alone packetbeat,

got it, more stupid questions here, can stand-alone packetbeat and an agent with the Network Package Capture run side by side, there seems to be so much overlap here ?.

I really appreciate the input I get here from you guys, thanks.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 11, 2023, 3:14pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/9 "2023-06-11T15:14:00Z")

</div>

> [@A113n](#):
>
> yes absolutely and it begins with me thinking the "old" packetbeat was [integrated](https://github.com/elastic/integrations/issues/1771) with the Network Packet Capture integration. I believe I've read it was orginally called packetbeat then renamed to Network Packet Capture.

The Elastic Agent runs beats behind the scenes, at first it was only filebeat, then other beats where added, like winlogbeat, metricbeat, auditbeat and packetbeat, but this is transparent for the user as you would just add the integrations.

> [@A113n](#):
>
> I could not find documentation stating Agents are expected to talk directly to Elasticsearch.

You won't find, this is not documented, it is just how the Elastic Agent started and works now.

The Elastic Agent relies on ingest pipelines to parse the data, those ingest pipelines are created and executed on Elasticsearch ingest nodes, since the main functionality of Logstash is to parse the data, it becomes redundant to have it as you can do almost everything with Ingest pipelines.

If I'm not wrong the first versions of Elastic Agent didn't even supported Logstash as an output.

If you are not doing any parse or enrich on Logstash, just have an `input` and `outpu`, I see no reason to use it, it will be just another tool to maintain.

> [@A113n](#):
>
> From a security pov this is naive/bad, having a jwt token and a direct endpoint in your agent config telling you where to abuse Elasticsearch is and should not be considered best practices.

Not sure what you mean with that and how this an issue and how this would be different from having the logstash output.

> [@A113n](#):
>
> can't say much about other data\_stream\_\* settings, I did not explict configure this in my logstash ... or I am missing your point.

The elasticsearch output in logstash has some [data stream settings](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data-streams) like `data_stream_namespace` etc, but you do not need to configure it as it is already present on the documents that come from the elastic agent.

The data stream in which logstash will write is derived from the `data_stream` fields present in every event, this happens because [data\_stream\_auto\_routing](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data_stream_auto_routing) is set to `true` by default.

> [@A113n](#):
>
> can stand-alone packetbeat and an agent with the Network Package Capture run side by side, there seems to be so much overlap here ?

Not sure, but I see no reason to do that, you should just run one or another, not both.

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [June 11, 2023, 5:03pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/10 "2023-06-11T17:03:51Z")

</div>

> [@leandrojmp](#):
>
> The Elastic Agent runs beats behind the scenes, at first it was only filebeat, then other beats where added, like winlogbeat, metricbeat, auditbeat and packetbeat,

exactly and I just mistakenly thought PacketBeat would behave like the stand alone package, further more the stand alone packages seems to be decommissioned eventually and the general advices is to embrace the Agent approach.

> [@leandrojmp](#):
>
> If you are not doing any parse or enrich on Logstash, just have an `input` and `outpu`, I see no reason to use it, it will be just another tool to maintain

Right, there will be use cases for us where the need to parse very old custom logfiles must be handled, logstash seems to fit here, but again novice when it comes to elasticsearch.

> [@leandrojmp](#):
>
> Not sure what you mean with that and how this an issue and how this would be different from having the logstash output.

If I know where 9200 is located in the network, I can start enumerating data, dump indexes, steal corporate secrets even manipulate data. Maybe delete some of the security indexes before doing a dcsync 😄, I mean this will not trigger an alert for the blue team now.  
Slowing dowing attackers with authentication is of couse always nice (if not disabled by default 🤪), no matter the authentication type. It's always best practices putting an intermediate in front on the stack.  
Having Logstash in front here seems like a good choice, one thing I like with Logstash in front of Elasticsearch, is the Agent _client side_ support of _loadbalancers_, this always scales better than server side + gotta love them persistent queues. There are other considerations here too, like different subnets, but I am not a certified lumberjack on our current setup 😜.

> [@leandrojmp](#):
>
> you should just run one or another, not both.

👍 noted, thank you for your valuable input.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 11, 2023, 5:45pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/11 "2023-06-11T17:45:12Z")

</div>

> [@A113n](#):
>
> I can start enumerating data, dump indexes, steal corporate secrets even manipulate data. Maybe delete some of the security indexes before doing a dcsync

Everything you mentioned requires the client to be authenticate and have the right permissions, with security enabled every elasticsearch requests needs to be authenticated.

I'm not sure what you mean with the following:

> From a security pov this is naive/bad, having a jwt token and a direct endpoint in your agent config telling you where to abuse Elasticsearch is and should not be considered best practices.

If I'm not wrong when you use fleet, the fleet server will generate an API key for the agents to use and will send this along with the configuration to the agent, and this will be encrypted in the agent server, this is explained [here in the documentation](https://www.elastic.co/guide/en/fleet/current/_elastic_agent_configuration_encryption.html#_elastic_agent_configuration_encryption).

So, using fleet, the endpoint and the API key are not present as plain-text on any files on the agent server.

The endpoint and the API key would be present in plain-text in the configuration file only if you were using the Elastic Agent in [standalone mode](https://www.elastic.co/guide/en/fleet/current/install-standalone-elastic-agent.html#install-standalone-elastic-agent), which is an Advanced use case.

In this scenario you need to make sure to limit access to the Elastic Agent configuration file, and this is no different of having the API Key in Endpoint in your Logstash configuration as you would also need to limit the access to the configuration files/Logstash server.

Elastic strongly recommends using the Elastic Agent with a Fleet Server.

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [June 11, 2023, 7:43pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/12 "2023-06-11T19:43:56Z")

</div>

Hi Leandro

> [@leandrojmp](#):
>
> Everything you mentioned requires the client to be authenticate

Yes, so ? you think that would stop a hacker 😇, as I said authentication is just a way of slowing down a hacker, this has never been the main concern. Do kerberos authentication, rest as sure I will get a TGT else where in the corporate network.

> [@leandrojmp](#):
>
> If I'm not wrong when you use fleet, the fleet server will generate an API key

right, but I'm **not** worried about fleet, take for instance the Endpoint Defend integration.  
Look at the output section 😉 , there you find Elasticsearch with the official endpoint and an ApiKey. I don't know what this ApiKey grants me access to, but I am sure I can enumerate templates, nodes and other useful stuff + I know an admin will f#¤k up the permissions eventually.  
That is my main concern, here is an endpoint and an apikey to go.

> [@leandrojmp](#):
>
> Elastic strongly recommends using the Elastic Agent with a Fleet Server.

sure ? I believe I never questioned the Fleet Server, this is simply an amazing way of pushing tedious configuration to clients.

Leandro, you provided me with the right answer, Elastic Agent does not create the packetbeat-\* index (yet) and some ML jobs (still) depends on the stand alone packetbeat, hence this needs to be install also.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 11, 2023, 8:56pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/13 "2023-06-11T20:56:17Z")

</div>

> [@A113n](#):
>
> ML jobs (still) depends on the stand-alone packetbeat, hence this needs to be install also.

Yes look like the packetbeat ML jobs have not been ported yet to the agent integration yet.

However, there is no great magic on the ML jobs, they just take a datafeed, some fields, and a config. We can most likely just load that ML job through the packetbeat and then clone / edit that job and change the data feed.. perhaps a few field names

Perhaps, If I get a chance I will take a look, it will probably be later in the week, I used to build the DNS Tunnelling / Exfil ML job by hand all the time...

It probably comes down to doing

Run `packetbeat setup -e`

GET on the ML job

Editing a Couple items in the JSON, data feed perhaps a few field names,

the PUT ing it back, and then it will work against the new data stream.

If you are interested let me know.

> [@leandrojmp](#):
>
> > [@A113n](#):
> >
> > can stand-alone packetbeat and an agent with the Network Package Capture run side by side, there seems to be so much overlap here ?
> 
> Not sure, but I see no reason to do that, you should just run one or another, not both.

Agreed, Pick one

- Use packetbeat and you get the ML job for Free.
- Use agent and we (or support 😉 ) can probably get the ML job ported.

Let me know if you want me to look at the ML job...

> [@A113n](#):
>
> My Main beef with this is, 'most' topics in this forum seems to end with a "DM" or "raise a support ticket".

PS. not sure where you got that, that is factually inaccurate / an over-exaggeration at the very least ... seems like an odd way to ask for assistance, especially from a user that is self-admittedly relatively new to the community.

For a little perspective:

- I referred you to support, as most people that _ **pay** _ for support with SLAs want to use it as we are all volunteers here, no SLA no guarantee that any / every topic will even be addressed.
- The vast majority of users that come to this forum use Basic / Free and thus do not have support, AND we do not cover licensing costs, or specific account issues etc. here
- We do, however, appreciate you want to share the issues and results with the community

Let see if we can get this to work for you and the community

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 12, 2023, 3:17am UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/14 "2023-06-12T03:17:20Z")

</div>

Well OK, I just looked into it and looks like the ML jobs are actually there ... you just need to create them with the correct data view... took me 5 mins. There is one hitch there is a slight but important miss-configuration that will need to be corrected i.e. the correct `event.dataset` ... I will show you how.

I understand there is already a PR to fix this, I don't have it handy

EDIT 8.8.0 should already be [fixed](https://github.com/elastic/kibana/blob/main/x-pack/plugins/ml/server/models/data_recognizer/modules/security_packetbeat/ml/datafeed_packetbeat_dns_tunneling.json#L10),  
8.7.1 still has this error

1st I am doing this with Elastic Agent Network Capture -\> Elasticsearch  
(No logstash in the middle although that should work according to the documentation)

Assumes agent is sending data

Go To ML - Jobs - Create Job

 ![Screenshot 2023-06-11 at 7.02.41 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/e/3e408bef744717b6d576cb3562624dcf82f27f2e.png)

Select the Correct Data View `logs-network_traffic`

 ![Screenshot 2023-06-11 at 7.04.25 PM](https://us1.discourse-cdn.com/elastic/original/3X/8/5/8595fb261d63b39dbd17745ae0b16fc151b3db42.png)

When you do that it will recognize it and then Select The Correct Job Group a little confusing because it says packetbeat (that should get cleaned up)

Select it

 ![Screenshot 2023-06-11 at 7.05.15 PM](https://us1.discourse-cdn.com/elastic/original/3X/6/f/6f91129acd49c62514040b44cc42934cfdbfcd0a.png)

And you will get this screen and select Create Jobs

 ![Screenshot 2023-06-11 at 7.05.27 PM](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7ef563b97b7a1715ee2fdc2a70c5f5fedebeedaf.jpeg)

You need to go in and make one edit...

 ![Screenshot 2023-06-11 at 8.10.30 PM](https://us1.discourse-cdn.com/elastic/original/3X/6/5/65eeeed0be87d9e38a23aa66e7bc6c7011cadb80.jpeg)

Edit the Data Feed (the even.dataset is wrong

 ![Screenshot 2023-06-11 at 7.17.44 PM](https://us1.discourse-cdn.com/elastic/original/3X/a/3/a3e835c3178d7b1b8066eb270a094367724b29f3.png)

```auto
{
  "bool": {
    "filter": [
      {
        "term": {
          "event.dataset": "network_traffic.dns" <!---- THIS 
        }
      },

```

Save and then Test the Data Feed ...

Should look something like this..

 ![Screenshot 2023-06-11 at 7.17.59 PM](https://us1.discourse-cdn.com/elastic/original/3X/0/7/0797c3f09061bf84495899f352d57693a6973a34.png)

And Whalluh you have the correct jobs pulling from the correct data view, You can just start it

You can start it when you are ready.... Probably need to do the same with the others the `event.dataset`

 ![Screenshot 2023-06-11 at 8.14.41 PM](https://us1.discourse-cdn.com/elastic/original/3X/2/6/267c3b985b820f569e21d864dc501cfb5ee66e0e.png)  
will be incorrect

---

<div class="post-metadata">

**Author:** ![A113n](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@A113n](https://discuss.elastic.co/u/A113n)\
**Post date:** [June 12, 2023, 10:01am UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/15 "2023-06-12T10:01:09Z")

</div>

Stephen, fantastic job, this works flawless.

Thank you for your support here.

> [@stephenb](#):
>
> Let see if we can get this to work for you and the community

🎯🙏✔  
look how far we got around various topics, thanks to Leandro and you 👍

......  
If you need anything from me, like license info etc. let me know, just so you can verify I'm not freeloading. I will next time absolutely reconsider asking topics here and go strait to support instead, road seems less bumpy as both of you adviced me to do, again thank you for your time and support.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 12, 2023, 2:07pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/16 "2023-06-12T14:07:10Z")

</div>

@A113n your welcome!

Glad you got it working. Please come here and ask questions anytime you like... There is no freeloading. This is a community.

I think we just got off on the wrong foot because often when a user has support that may be a quicker more direct method or may include more sensitive data etc.

Also does depend on the level of support. Sometimes a user may only have break / fix others may have more consultative depending on the volume of your license.

Come on back. Bring us a good question!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 10, 2023, 2:07pm UTC](https://discuss.elastic.co/t/datafeed-datafeed-packetbeat-dns-tunneling-cannot-retrieve-data-because-no-index-matches-datafeeds-indices-packetbeat/334356/17 "2023-07-10T14:07:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
