Why Multiple Elasticsearch Nodes Fail to Form a Cluster: master_not_discovered_exception or NotMasterException and How to Fix It

If you’ve encountered a situation where multiple Elasticsearch nodes fail to automatically form a cluster and the logs show the error: “master_not_discovered_exception”, you are not alone.

The Problem

The root cause is often related to system administrators cloning virtual machines to create multiple Elasticsearch servers. When this happens, every Elasticsearch node ends up with the same node ID, and as a result, the cluster cannot successfully elect a master node.

Verifying the Issue

You can verify this issue by listing all the node IDs with the following command:

GET /_cat/nodes?v&h=id,ip,name&full_id=true

However, since the Elasticsearch cluster hasn’t formed, you need to query each node individually, like this:

curl 192.168.110.111:9200/_cat/nodes?v&h=id,ip,name&full_id=true
curl 192.168.110.112:9200/_cat/nodes?v&h=id,ip,name&full_id=true

The Solution

Elasticsearch requires each node to have a unique node ID. To fix this issue, you need to delete the index data on each node. If Elasticsearch was installed using the RPM package, the index data is usually stored in /var/lib/elasticsearch by default. After deleting the data, restart Elasticsearch, and it will generate a new, unique node ID for each node.

Reference

For further details, check the full article here: https://www.656463.com/wenda/jdbhjrjqNotMasterExceptionqgddxc_359.