theme: vue-pro highlight: atelier-sulphurpool-light
1. What is a cluster in Elasticsearch: 7.0
An Elasticsearch 7.0 cluster is a system composed of multiple nodes that work together to perform tasks. A cluster includes master nodes, data nodes, and coordinating nodes, which are responsible for managing cluster changes, storing data and executing data operations, and receiving client requests and coordinating data collection and integration respectively. Through clustering, Elasticsearch can maintain data consistency and deliver high-performance search and data analysis capabilities.
2. Hands-on Elasticsearch: 7.0 Cluster Setup (Linux Version)
2.1 Download the software
elasticsearch-7.8.0-linux: Baidu Netdisk download link
elasticsearch-7.8.0-linux: Official download link
2.2 Prepare three CentOS machines: 129, 131, 132
2.3 Extract the compressed archive
Navigate to the folder containing the archive to extract it
cd /java/elasticsearch/
ls
tar -zxvf elasticsearch-7.8.0-linux-x86_64.tar.gz
2.4 Create a user on CentOS
- To ensure the secure and stable operation of Elasticsearch, we need to create a dedicated non-root user on each node. This effectively isolates and reduces the risks of running Elasticsearch with root privileges, and aligns with system security best practices.
useradd es # Add a new es user
passwd es # Set a password for the es user
Grant permissions to the folder
chown -R es:es /java/elasticsearch/elasticsearch-7.8.0
Edit /etc/security/limits.conf, add this configuration to the end of the file
vi /etc/security/limits.conf
es soft nofile 65536
es hard nofile 65536
Edit /etc/security/limits.d/20-nproc.conf, add this configuration to the end of the file
vi /etc/security/limits.d/20-nproc.conf
es soft nofile 65536
es hard nofile 65536
* hard nproc 4096
Edit /etc/sysctl.conf, add this configuration to the end of the file
vi /etc/sysctl.conf
vm.max_map_count=655360
Reload the configuration
sysctl -p
Perform the above operations on every CentOS machine
2.5 Modify the elasticsearch.yml configuration file
① Modify the configuration file for IP: 129
Navigate to this directory
cd /java/elasticsearch/elasticsearch-7.8.0/config
Double-click to open elasticsearch.yml
Add the following configuration to the end of the configuration file
# Add the following configuration
# Cluster name
cluster.name: cluster-es
# Node name: node names must be unique across all nodes
node.name: node-1
# IP address: IP addresses must be unique across all nodes
network.host: 192.168.118.129
# Whether this node is eligible to be elected as master node
node.master: true
node.data: true
http.port: 9200
# These two configurations are required for the head plugin
http.cors.allow-origin: "*"
http.cors.enabled: true
http.max_content_length: 200mb
# New configuration since es7.x: this configuration is required for master election when initializing a new cluster
cluster.initial_master_nodes: ["node-1"]
# New configuration since es7.x: node discovery
discovery.seed_hosts: ["192.168.118.129:9300","192.168.118.131:9300","192.168.118.132:9300"]
gateway.recover_after_nodes: 2
network.tcp.keep_alive: true
network.tcp.no_delay: true
transport.tcp.compress: true
# Number of concurrent data tasks running in the cluster, default is 2
cluster.routing.allocation.cluster_concurrent_rebalance: 16
# Number of concurrent recovery threads when adding/removing nodes or during load balancing, default is 4
cluster.routing.allocation.node_concurrent_recoveries: 16
# Number of concurrent recovery threads during initial data recovery, default is 4
cluster.routing.allocation.node_initial_primaries_recoveries: 16
② Start Elasticsearch
Switch to the es user and start Elasticsearch
su es
cd /java/elasticsearch/elasticsearch-7.8.0/bin/
./elasticsearch
Enter http://192.168.118.129:9200/_cat/nodes in your browser
③ Modify the configuration file for IP: 131
Navigate to this directory
cd /java/elasticsearch/elasticsearch-7.8.0/config
Double-click to open elasticsearch.yml
Add the following configuration to the end of the configuration file
# Add the following configuration
# Cluster name
cluster.name: cluster-es
# Node name: node names must be unique across all nodes
node.name: node-2
# IP address: IP addresses must be unique across all nodes
network.host: 192.168.118.131
# Whether this node is eligible to be elected as master node
node.master: true
node.data: true
http.port: 9200
# These two configurations are required for the head plugin
http.cors.allow-origin: "*"
http.cors.enabled: true
http.max_content_length: 200mb
# New configuration since es7.x: this configuration is required for master election when initializing a new cluster
cluster.initial_master_nodes: ["node-1"]
# New configuration since es7.x: node discovery
discovery.seed_hosts: ["192.168.118.129:9300","192.168.118.131:9300","192.168.118.132:9300"]
gateway.recover_after_nodes: 2
network.tcp.keep_alive: true
network.tcp.no_delay: true
transport.tcp.compress: true
# Number of concurrent data tasks running in the cluster, default is 2
cluster.routing.allocation.cluster_concurrent_rebalance: 16
# Number of concurrent recovery threads when adding/removing nodes or during load balancing, default is 4
cluster.routing.allocation.node_concurrent_recoveries: 16
# Number of concurrent recovery threads during initial data recovery, default is 4
cluster.routing.allocation.node_initial_primaries_recoveries: 16
④ Start Elasticsearch
Switch to the es user and start Elasticsearch
su es
cd /java/elasticsearch/elasticsearch-7.8.0/bin/
./elasticsearch
Enter http://192.168.118.131:9200/_cat/nodes in your browser
⑤ Modify the configuration file for IP: 132
Navigate to this directory
cd /java/elasticsearch/elasticsearch-7.8.0/config
Double-click to open elasticsearch.yml
Add the following configuration to the end of the configuration file
# Add the following configuration
# Cluster name
cluster.name: cluster-es
# Node name: node names must be unique across all nodes
node.name: node-3
# IP address: IP addresses must be unique across all nodes
network.host: 192.168.118.132
# Whether this node is eligible to be elected as master node
node.master: true
node.data: true
http.port: 9200
# These two configurations are required for the head plugin
http.cors.allow-origin: "*"
http.cors.enabled: true
http.max_content_length: 200mb
# New configuration since es7.x: this configuration is required for master election when initializing a new cluster
cluster.initial_master_nodes: ["node-1"]
# New configuration since es7.x: node discovery
discovery.seed_hosts: ["192.168.118.129:9300","192.168.118.131:9300","192.168.118.132:9300"]
gateway.recover_after_nodes: 2
network.tcp.keep_alive: true
network.tcp.no_delay: true
transport.tcp.compress: true
# Number of concurrent data tasks running in the cluster, default is 2
cluster.routing.allocation.cluster_concurrent_rebalance: 16
# Number of concurrent recovery threads when adding/removing nodes or during load balancing, default is 4
cluster.routing.allocation.node_concurrent_recoveries: 16
# Number of concurrent recovery threads during initial data recovery, default is 4
cluster.routing.allocation.node_initial_primaries_recoveries: 16
⑥ Start Elasticsearch
Switch to the es user and start Elasticsearch
su es
cd /java/elasticsearch/elasticsearch-7.8.0/bin/
./elasticsearch
Enter http://192.168.118.132:9200/_cat/nodes in your browser
3. Summary
This article details the concept of an Elasticsearch 7.0 cluster, and demonstrates how to build a three-node Elasticsearch cluster in a Linux environment through step-by-step hands-on instructions. It first introduces the composition and role of a cluster, then elaborates on the entire process from software download, node preparation, user creation, configuration file modification, to starting the cluster. Each step includes specific command-line operations and configuration content, providing a clear guide for beginners. Through this article, readers will understand the setup process and key precautions for Elasticsearch clusters, laying a solid foundation for subsequent search and data analysis work.
This is a discussion topic separated from the original thread at https://juejin.cn/post/7368276508136669247