首页 > 代码库 > 突破单机多实例Elasticsearch
突破单机多实例Elasticsearch
默认大家都是单机单实例es,在实验环境下想尽可能模拟各种场景。单机多实例就出来了。。。
实验拓扑图
01、es安装这里就不说了
详情:http://www.cnblogs.com/xiaochina/p/6852677.html
02、讲elasticsearch.yml配置
要做到单机上开多个实例,需要修改ES的默认配置:
node.max_local_storage_nodes: 2 #单机开启2个实例
配置限制了单节点上可以开启的ES存储实例的个数
http.port: 9200
配置是elasticsearch对外提供服务的http端口配置,默认情况下ES会取用9200~9299之间的端口,如果9200被占用就会自动使用9201
在单机多实例的配置中这个配置实际是不需要修改的,但是为了更好地进行配置管理,建议根据需求修改9201 9202
transport.tcp.port: 9300
配置指定了elasticsearch集群内数据通讯使用的端口,默认情况下为9300,与上面的http.port配置类似,ES也会自动为已占用的端口选择下一个端口号。我们可以将第一个实例的tcp传输端口配置为9300,第二实例配置为9301。
discovery.zen.ping.unicast.hosts 由于到了2.x版本之后,ES取消了默认的广播模式来发现master节点,需要使用该配置来指定发现master节点。这个配置在单机双实例的配置中需要特别注意下,因为习惯上我们配置时并未指定master节点的tcp端口,如果实例的transport.tcp.port配置为9301,那么实例启动后会认为discovery.zen.ping.unicast.hosts中指定的主机tcp端口也是9301,可能导致这些节点无法找到master节点。
该配置中需要指定master节点提供服务的tcp端口。
配置示例:
discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]
node.name
同一主机上的两个实例需要使用不同的node.name
path.data
同一主机上两个实例需要对应不同的数据目录
path.logs
由于默认情况下日志用集群名称来命名,因此同一主机两个实例对应的日志目录需要分开
#禁止HTTP
http.enabled: false
配置样例
node1-es-data
[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.# Before you set out to tweak and tune the configuration, make sure you# understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-1node.master: falsenode.data: true #数据存储node.max_local_storage_nodes: 2 #本机最大两个estransport.tcp.port: 9301 #节点通信端口http.enabled: false #关闭http接口## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.25## Set a custom port for HTTP:##http.port: 9201## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"] #候选主节点地址## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true
node2-es-data
[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.# Before you set out to tweak and tune the configuration, make sure you# understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-2node.master: falsenode.data: truenode.max_local_storage_nodes: 2transport.tcp.port: 9302http.enabled: false## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.25## Set a custom port for HTTP:##http.port: 9202## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true
node3-es-master
[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.# Before you set out to tweak and tune the configuration, make sure you# understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-3node.master: truenode.data: falsenode.max_local_storage_nodes: 2transport.tcp.port: 9301http.enabled: true## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.26## Set a custom port for HTTP:#http.port: 9201## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true
node4-es-data
[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.# Before you set out to tweak and tune the configuration, make sure you# understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-4node.master: falsenode.data: false
node.ingest: false
node.max_local_storage_nodes: 2transport.tcp.port: 9302http.enabled: true## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.26## Set a custom port for HTTP:#http.port: 9202## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true
效果图
注意:复制的elasticsearch文件夹下包含了data文件中示例一的节点数据,需要把示例二data文件下的文件清空
Kibana的elasticsearch.url:配置load-balancedi地址 http://172.24.0.26:9202
学习参考:http://www.tuicool.com/articles/VBVFzyi
https://www.elastic.co/guide/en/kibana/current/production.html#load-balancing #搜索请求节点负载
突破单机多实例Elasticsearch