Permission error for custom log integration on Fleet-managed Agent

[Panagiota_Mitsopoulo (Panagiota Mitsopoulou)]

Hello!

I have a Nodejs app that ingests logs into Elasticsearch Service deployment 7.15.1 using Filebeat. More of less this is the example I've been doing adapted to my needs Ingest logs from a Node.js web application using Filebeat | Elasticsearch Service Documentation | Elastic and everything works fine.

Now I am trying to use Fleet and Elastic agent to send my custom json logs but here's the error I got:

[elastic_agent.filebeat][error] Harvester could not be started on new file: /Users/panagiotamitsopoulou/Documents/HYF/nodejs_ES_logs/logs/log.json, Err: error setting up harvester: Harvester setup failed. Unexpected file opening error: Failed opening /Users/panagiotamitsopoulou/Documents/HYF/nodejs_ES_logs/logs/log.json: open /Users/panagiotamitsopoulou/Documents/HYF/nodejs_ES_logs/logs/log.json: operation not permitted.

I am on Mac and here are the steps I've taken.

• On Kibana > Fleet I verified that my agent with Elastic Cloud agent policy is Healthy
• I enabled Collect agent logs under Elastic Cloud agent policy
• I created a new Fleet-managed agent and installed the agent to my machine with the sudo ./elastic-agent install -f .... I copied from the UI.
• I did chmod -R 777 logs
• I keep getting the permission error I wrote above

How can I solve above issue or better how can I troubleshoot if I am doing something wrong or if it is actually a bug?

A couple more questions since I would like to demonstrate above example to an audience if I manage to make it work and I would like to know how it actually works:

• Enabling Collect agent logs under Elastic Cloud agent policy is a prerequisite mandatory step I need to do to be able to send custom logs?
• Do I need to add anything else in my custom configuration?
  
  
• I found a similar post Adding config for custom log integration on an agent, where these two configuration options are mentioned agent.logging.json and agent.logging.ecs. Do I need to update them? If yes where is the installed path of my agent to do so?

If needed I could share the github link of my code.

Thanks,
Panagiota