If you’ve encountered a situation where multiple Elasticsearch nodes fail to automatically form a cluster and the logs show the error: “master_not_discovered_exception”, you are not alone.
The Problem
The root cause is often related to system administrators cloning virtual machines to create multiple Elasticsearch servers. When this happens, every Elasticsearch node ends up with the same node ID, and as a result, the cluster cannot successfully elect a master node.
Verifying the Issue
You can verify this issue by listing all the node IDs with the following command:
GET /_cat/nodes?v&h=id,ip,name&full_id=true
However, since the Elasticsearch cluster hasn’t formed, you need to query each node individually, like this:
curl 192.168.110.111:9200/_cat/nodes?v&h=id,ip,name&full_id=true
curl 192.168.110.112:9200/_cat/nodes?v&h=id,ip,name&full_id=true
The Solution
Elasticsearch requires each node to have a unique node ID. To fix this issue, you need to delete the index data on each node. If Elasticsearch was installed using the RPM package, the index data is usually stored in /var/lib/elasticsearch
by default. After deleting the data, restart Elasticsearch, and it will generate a new, unique node ID for each node.
Reference
For further details, check the full article here: https://www.656463.com/wenda/jdbhjrjqNotMasterExceptionqgddxc_359.