QUESTION ABOUT ELK SIEM

[Firas_Bougrine]

how i can fix this error permanently when i fix it it shows for me again

[FALEN (Ali Can Saykal)]

Hi,

it seems your Elasticsearch.yml file is not properly formatted, could you please share it ?

[Firas_Bougrine]

how to share it without capture? to see it good

[FALEN (Ali Can Saykal)]

Just copy paste like below

Example 

[Firas_Bougrine]

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
  path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 0.0.0.0
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
cluster.initial_master_nodes: ["node-1"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 04-04-2022 21:59:27
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["node-1"]

# Allow HTTP API connections from localhost and local networks
# Connections are encrypted and require user authentication
http.host: [_local_, _site_]

# Allow other nodes to join the clus

[warkolm (Mark Walkom)]

Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them

[Firas_Bougrine]

sorry but i copy paste not pictures
can you help me and response my question

[warkolm (Mark Walkom)]

I'm not able to read your first picture sorry.

However you have a few blank spaces in front of path.logs which won't be valid.

[Firas_Bougrine]

this the code of the first picture ( the problem ) :
Elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/Elasticsearch.service; enabled; vendor>
Drop-In: /etc/systemd/system/Elasticsearch.service.d
└─startup-timeout.conf
Active: failed (Result: exit-code) since Tue 2022-04-05 22:28:41 UTC; 18s >
Docs: https://www.elastic.co
Process: 851 ExecStart=/usr/share/Elasticsearch/bin/systemd-entrypoint -p $>
Main PID: 851 (code=exited, status=1/FAILURE)

avril 05 22:28:41 test systemd-entrypoint[1418]: at com.fasterxml.jacks>
avril 05 22:28:41 test systemd-entrypoint[1418]: at com.fasterxml.jacks>
avril 05 22:28:41 test systemd-entrypoint[1418]: at org.Elasticsearch.x>
avril 05 22:28:41 test systemd-entrypoint[1418]: at org.Elasticsearch.c>
avril 05 22:28:41 test systemd-entrypoint[1418]: at org.Elasticsearch.c>
avril 05 22:28:41 test systemd-entrypoint[1418]: at org.Elasticsearch.c>
avril 05 22:28:41 test systemd-entrypoint[1418]: ... 8 more
avril 05 22:28:41 test systemd[1]: Elasticsearch.service: Main process exited, >
avril 05 22:28:41 test systemd[1]: Elasticsearch.service: Failed with result 'e>
avril 05 22:28:41 test systemd[1]: Failed to start Elasticsearch.

[Firas_Bougrine]

and for your note about path.logs i will fix it now and i will see the result

[warkolm (Mark Walkom)]

You'd need to look at the Elasticsearch log to find out more information on what the problem is. I'd start by fixing the file and then seeing if there's any other issues.

[Firas_Bougrine]

what's the cmd to access the Elasticsearch log ?

[warkolm (Mark Walkom)]

The logs are usually stored in /var/log/elasticsearch, then you can use whatever you would like to view them.

[Firas_Bougrine]

sudo /var/log/Elasticsearch
sudo: /var/log/Elasticsearch: command not found
i try this command to see the log , but i don't know how to see the logs give the steps to do i'am a beginner in ELK et ubuntu i don't know some CMD

[warkolm (Mark Walkom)]

Try cat /var/log/elasticsearch/elasticsearch.log. Please note that in linux, capitalisation is important.

[stephenb (Stephen Brown)]

See Here : You need to use journalctl to get the logs.

When systemd logging is enabled, the logging information are available using the journalctl commands:

To tail the journal:

sudo journalctl -f

To list journal entries for the elasticsearch service:

sudo journalctl --unit elasticsearch

To list journal entries for the elasticsearch service starting from a given time:

sudo journalctl --unit elasticsearch --since "2016-10-30 18:17:16"

Check man journalctl or journalctl for more command line options.

[Firas_Bougrine]

test@test:~$ journalctl
-- Logs begin at Mon 2022-03-28 17:09:23 UTC, end at Wed 2022-04-06 17:15:44 UT>
mars 28 17:09:23 test kernel: Linux version 5.4.0-105-generic (buildd@lcy02-amd>
mars 28 17:09:23 test kernel: Command line: BOOT_IMAGE=/vmlinuz-5.4.0-105-gener>
mars 28 17:09:23 test kernel: KERNEL supported cpus:
mars 28 17:09:23 test kernel: Intel GenuineIntel
mars 28 17:09:23 test kernel: AMD AuthenticAMD
mars 28 17:09:23 test kernel: Hygon HygonGenuine
mars 28 17:09:23 test kernel: Centaur CentaurHauls
mars 28 17:09:23 test kernel: zhaoxin Shanghai
mars 28 17:09:23 test kernel: Disabled fast string operations
mars 28 17:09:23 test kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 flo>
mars 28 17:09:23 test kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE reg>
mars 28 17:09:23 test kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX reg>
mars 28 17:09:23 test kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]:>
mars 28 17:09:23 test kernel: x86/fpu: Enabled xstate features 0x7, context siz>
mars 28 17:09:23 test kernel: BIOS-provided physical RAM map:
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009>
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000000009e800-0x000000000009>
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x00000000000dc000-0x00000000000f>
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007fed>
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000007fee0000-0x000000007fef>
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000007feff000-0x000000007fef>
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000007ff00000-0x000000007fff>
lines 1-23...skipping...
-- Logs begin at Mon 2022-03-28 17:09:23 UTC, end at Wed 2022-04-06 17:15:44 UTC. --
mars 28 17:09:23 test kernel: Linux version 5.4.0-105-generic (buildd@lcy02-amd64-066) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04)) #119-Ubuntu SMP Mon Mar 7 18:4>
mars 28 17:09:23 test kernel: Command line: BOOT_IMAGE=/vmlinuz-5.4.0-105-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro maybe-ubiquity
mars 28 17:09:23 test kernel: KERNEL supported cpus:
mars 28 17:09:23 test kernel: Intel GenuineIntel
mars 28 17:09:23 test kernel: AMD AuthenticAMD
mars 28 17:09:23 test kernel: Hygon HygonGenuine
mars 28 17:09:23 test kernel: Centaur CentaurHauls
mars 28 17:09:23 test kernel: zhaoxin Shanghai
mars 28 17:09:23 test kernel: Disabled fast string operations
mars 28 17:09:23 test kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
mars 28 17:09:23 test kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
mars 28 17:09:23 test kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
mars 28 17:09:23 test kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
mars 28 17:09:23 test kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
mars 28 17:09:23 test kernel: BIOS-provided physical RAM map:
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009e7ff] usable
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000000009e800-0x000000000009ffff] reserved
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x00000000000dc000-0x00000000000fffff] reserved
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007fedffff] usable
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000007fee0000-0x000000007fefefff] ACPI data
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000007feff000-0x000000007fefffff] ACPI NVS
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x000000007ff00000-0x000000007fffffff] usable
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x00000000f0000000-0x00000000f7ffffff] reserved
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
mars 28 17:09:23 test kernel: BIOS-e820: [mem 0x00000000fffe0000-0x00000000ffffffff] reserved
mars 28 17:09:23 test kernel: NX (Execute Disable) protection: active
mars 28 17:09:23 test kernel: SMBIOS 2.7 present.
mars 28 17:09:23 test kernel: DMI: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
mars 28 17:09:23 test kernel: vmware: hypercall mode: 0x02
mars 28 17:09:23 test kernel: Hypervisor detected: VMware
mars 28 17:09:23 test kernel: vmware: TSC freq read from hypervisor : 1995.383 MHz
mars 28 17:09:23 test kernel: vmware: Host bus clock speed read from hypervisor : 66000000 Hz
mars 28 17:09:23 test kernel: vmware: using sched offset of 29620633669 ns
mars 28 17:09:23 test kernel: tsc: Detected 1995.383 MHz processor
mars 28 17:09:23 test kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
mars 28 17:09:23 test kernel: e820: remove [mem 0x000a0000-0x000fffff] usable
mars 28 17:09:23 test kernel: last_pfn = 0x80000 max_arch_pfn = 0x400000000
mars 28 17:09:23 test kernel: MTRR default type: uncachable
mars 28 17:09:23 test kernel: MTRR fixed ranges enabled:
mars 28 17:09:23 test kernel: 00000-9FFFF write-back
mars 28 17:09:23

[Firas_Bougrine]

test@test:~$ journalctl -xe
-- Subject: Le démarrage du gestionnaire utilisateur est terminé
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- L'instance du gestionnaire d'utilisateurs pour l'utilisateur 1000 a été démarrée.
-- Tous les services en file d'attente pour démarrer ont été lancés.
-- Notez que des services peuvent être encore en train de démarrer,
-- ou d'autres être lancés à tout moment ultérieur.

-- Le démarrage du gestionnaire a nécéssité 321885 microsecondes.
avril 06 17:14:24 test systemd[1]: Started User Manager for UID 1000.
-- Subject: L'unité (unit) user@1000.service a terminé son démarrage
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- L'unité (unit) user@1000.service a terminé son démarrage, avec le résultat done.
avril 06 17:14:24 test systemd[1]: Started Session 1 of user test.
-- Subject: L'unité (unit) session-1.scope a terminé son démarrage
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- L'unité (unit) session-1.scope a terminé son démarrage, avec le résultat done.
avril 06 17:14:59 test sshd[1481]: Accepted password for test from 192.168.222.1 port 50663 ssh2
avril 06 17:14:59 test sshd[1481]: pam_unix(sshd:session): session opened for user test by (uid=0)
avril 06 17:14:59 test systemd-logind[876]: New session 3 of user test.
-- Subject: Une nouvelle session 3 a été créée pour l'utilisateur test
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu
-- Documentation: multiseat

-- Une nouvelle session a été créée pour l'utilisateur test avec
-- l'identifiant (ID) 3.

-- Le processus maître de la session est 1481.
avril 06 17:14:59 test systemd[1]: Started Session 3 of user test.
-- Subject: L'unité (unit) session-3.scope a terminé son démarrage
-- Defined-By: systemd
-- Support: Enterprise open source support | Ubuntu

-- L'unité (unit) session-3.scope a terminé son démarrage, avec le résultat done.
avril 06 17:15:36 test sudo[1593]: test : TTY=pts/0 ; PWD=/home/test ; USER=root ; COMMAND=/usr/bin/systemctl status Elasticsearch.service
avril 06 17:15:36 test sudo[1593]: pam_unix(sudo:session): session opened for user root by test(uid=0)
avril 06 17:15:44 test sudo[1593]: pam_unix(sudo:session): session closed for user root
l

[Firas_Bougrine]

elastic@elastic:~$ journalctl -u Elasticsearch.service
-- Logs begin at Tue 2022-04-05 21:45:45 UTC, end at Wed 2022-04-06 17:20:24 UTC. --
avril 05 22:57:51 elastic systemd[1]: Starting Elasticsearch...
avril 05 22:59:06 elastic systemd[1]: Elasticsearch.service: start operation timed out. Terminating.
avril 05 22:59:07 elastic systemd[1]: Elasticsearch.service: Failed with result 'timeout'.
avril 05 22:59:07 elastic systemd[1]: Failed to start Elasticsearch.
avril 05 23:12:20 elastic systemd[1]: Starting Elasticsearch...
avril 05 23:13:15 elastic systemd-entrypoint[8055]: uncaught exception in thread [main]
avril 05 23:13:15 elastic systemd-entrypoint[8055]: java.lang.IllegalArgumentException: setting [cluster.initial_master_nodes] is not allowed when [discovery.type] is >
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.cluster.coordination.ClusterBootstrapService.(ClusterBootstrapService.java:87)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.cluster.coordination.Coordinator.(Coordinator.java:246)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.discovery.DiscoveryModule.(DiscoveryModule.java:195)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.node.Node.(Node.java:820)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.node.Node.(Node.java:284)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:234)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.cli.Command.main(Command.java:77)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: at org.Elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)
avril 05 23:13:15 elastic systemd-entrypoint[8055]: For complete error details, refer to the log at /var/log/Elasticsearch/Elasticsearch.log
avril 05 23:13:15 elastic systemd[1]: Elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
avril 05 23:13:15 elastic systemd[1]: Elasticsearch.service: Failed with result 'exit-code'.
avril 05 23:13:15 elastic systemd[1]: Failed to start Elasticsearch.
avril 05 23:15:47 elastic systemd[1]: Starting Elasticsearch...
avril 05 23:16:30 elastic systemd-entrypoint[8690]: uncaught exception in thread [main]
avril 05 23:16:30 elastic systemd-entrypoint[8690]: java.lang.IllegalArgumentException: setting [cluster.initial_master_nodes] is not allowed when [discovery.type] is >
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.cluster.coordination.ClusterBootstrapService.(ClusterBootstrapService.java:87)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.cluster.coordination.Coordinator.(Coordinator.java:246)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.discovery.DiscoveryModule.(DiscoveryModule.java:195)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.node.Node.(Node.java:820)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.node.Node.(Node.java:284)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:234)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
avril 05 23:16:30 elastic systemd-entrypoint[8690]: at org.Elasticsearch.cli.Command.main(Command.java:77)
l

[stephenb (Stephen Brown)]

The error is ;

@Firas_Bougrine

[cluster.initial_master_nodes] is not allowed when [discovery.type] is >

Which means you have conflicting settings...

Please post your entire elasticsearch.yml and please format with the </> it is hard for us to help if you do not take the care to do this... Highlight the text and click the format button shown below.