首页 > 代码库 > 突破单机多实例Elasticsearch

突破单机多实例Elasticsearch

    默认大家都是单机单实例es,在实验环境下想尽可能模拟各种场景。单机多实例就出来了。。。

实验拓扑图

 技术分享

01、es安装这里就不说了

详情:http://www.cnblogs.com/xiaochina/p/6852677.html

02、讲elasticsearch.yml配置

要做到单机上开多个实例,需要修改ES的默认配置:

node.max_local_storage_nodes: 2   #单机开启2个实例

配置限制了单节点上可以开启的ES存储实例的个数

http.port: 9200

配置是elasticsearch对外提供服务的http端口配置,默认情况下ES会取用9200~9299之间的端口,如果9200被占用就会自动使用9201

在单机多实例的配置中这个配置实际是不需要修改的,但是为了更好地进行配置管理,建议根据需求修改9201 9202

transport.tcp.port: 9300

配置指定了elasticsearch集群内数据通讯使用的端口,默认情况下为9300,与上面的http.port配置类似,ES也会自动为已占用的端口选择下一个端口号。我们可以将第一个实例的tcp传输端口配置为9300,第二实例配置为9301。

discovery.zen.ping.unicast.hosts  由于到了2.x版本之后,ES取消了默认的广播模式来发现master节点,需要使用该配置来指定发现master节点。这个配置在单机双实例的配置中需要特别注意下,因为习惯上我们配置时并未指定master节点的tcp端口,如果实例的transport.tcp.port配置为9301,那么实例启动后会认为discovery.zen.ping.unicast.hosts中指定的主机tcp端口也是9301,可能导致这些节点无法找到master节点。

该配置中需要指定master节点提供服务的tcp端口。

配置示例:

discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]
  • node.name

    同一主机上的两个实例需要使用不同的node.name

  • path.data

    同一主机上两个实例需要对应不同的数据目录

path.logs

由于默认情况下日志用集群名称来命名,因此同一主机两个实例对应的日志目录需要分开

 #禁止HTTP
http.enabled: false

配置样例

node1-es-data

[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.#       Before you set out to tweak and tune the configuration, make sure you#       understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-1node.master: falsenode.data: true                 #数据存储node.max_local_storage_nodes: 2 #本机最大两个estransport.tcp.port: 9301        #节点通信端口http.enabled: false             #关闭http接口## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.25## Set a custom port for HTTP:##http.port: 9201## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]  #候选主节点地址## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true

node2-es-data

[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.#       Before you set out to tweak and tune the configuration, make sure you#       understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-2node.master: falsenode.data: truenode.max_local_storage_nodes: 2transport.tcp.port: 9302http.enabled: false## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.25## Set a custom port for HTTP:##http.port: 9202## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true

node3-es-master

[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.#       Before you set out to tweak and tune the configuration, make sure you#       understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-3node.master: truenode.data: falsenode.max_local_storage_nodes: 2transport.tcp.port: 9301http.enabled: true## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.26## Set a custom port for HTTP:#http.port: 9201## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true

node4-es-data

[elk@25 config]$ cat elasticsearch.yml# ======================== Elasticsearch Configuration =========================## NOTE: Elasticsearch comes with reasonable defaults for most settings.#       Before you set out to tweak and tune the configuration, make sure you#       understand what are you trying to accomplish and the consequences.## The primary way of configuring a node is via this file. This template lists# the most important settings you may want to configure for a production cluster.## Please consult the documentation for further information on configuration options:# https://www.elastic.co/guide/en/elasticsearch/reference/index.html## ---------------------------------- Cluster -----------------------------------## Use a descriptive name for your cluster:#cluster.name: mvpbang## ------------------------------------ Node ------------------------------------## Use a descriptive name for the node:#node.name: node-4node.master: falsenode.data: false
node.ingest: false
node.max_local_storage_nodes: 2transport.tcp.port: 9302http.enabled: true## Add custom attributes to the node:##node.attr.rack: r1## ----------------------------------- Paths ------------------------------------## Path to directory where to store the data (separate multiple locations by comma):##path.data: /path/to/data## Path to log files:##path.logs: /path/to/logs## ----------------------------------- Memory -----------------------------------## Lock the memory on startup:#bootstrap.memory_lock: falsebootstrap.system_call_filter: false## Make sure that the heap size is set to about half the memory available# on the system and that the owner of the process is allowed to use this# limit.## Elasticsearch performs poorly when the system is swapping the memory.## ---------------------------------- Network -----------------------------------## Set the bind address to a specific IP (IPv4 or IPv6):#network.host: 172.24.0.26## Set a custom port for HTTP:#http.port: 9202## For more information, consult the network module documentation.## --------------------------------- Discovery ----------------------------------## Pass an initial list of hosts to perform discovery when new node is started:# The default list of hosts is ["127.0.0.1", "[::1]"]#discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]## Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):#discovery.zen.minimum_master_nodes: 1## For more information, consult the zen discovery module documentation.## ---------------------------------- Gateway -----------------------------------## Block initial recovery after a full cluster restart until N nodes are started:##gateway.recover_after_nodes: 3## For more information, consult the gateway module documentation.## ---------------------------------- Various -----------------------------------## Require explicit names when deleting indices:##action.destructive_requires_name: true

 效果图

技术分享

 

 注意:复制的elasticsearch文件夹下包含了data文件中示例一的节点数据,需要把示例二data文件下的文件清空

Kibana的elasticsearch.url:配置load-balancedi地址   http://172.24.0.26:9202

学习参考:http://www.tuicool.com/articles/VBVFzyi

https://www.elastic.co/guide/en/kibana/current/production.html#load-balancing  #搜索请求节点负载

突破单机多实例Elasticsearch