首页 > 代码库 > 最新的ES 5.0路由算法底层实现

最新的ES 5.0路由算法底层实现

2024-08-21 19:14:30   218人阅读

http://www.cnblogs.com/bonelee/p/6078947.html 里分析了ES bulk实现，其中路由代码：

ShardId shardId = clusterService.operationRouting().indexShards(clusterState, concreteIndex, request.id(), request.routing()).shardId();

其实现： https://github.com/elastic/elasticsearch/blob/master/core/src/main/java/org/elasticsearch/cluster/routing/OperationRouting.java

    public ShardIterator indexShards(ClusterState clusterState, String index, String id, @Nullable String routing) {
        return shards(clusterState, index, id, routing).shardsIt();
    }

    protected IndexShardRoutingTable shards(ClusterState clusterState, String index, String id, String routing) {
        int shardId = generateShardId(indexMetaData(clusterState, index), id, routing);
        return clusterState.getRoutingTable().shardRoutingTable(index, shardId);
    }

    static int generateShardId(IndexMetaData indexMetaData, String id, @Nullable String routing) {
        final int hash;
        if (routing == null) {
            hash = Murmur3HashFunction.hash(id);
        } else {
            hash = Murmur3HashFunction.hash(routing);
        }
        // we don‘t use IMD#getNumberOfShards since the index might have been shrunk such that we need to use the size
        // of original index to hash documents
        return Math.floorMod(hash, indexMetaData.getRoutingNumShards()) / indexMetaData.getRoutingFactor();
    }

可以看到最新的Es代码实现路由是：

Math.floorMod(hash, indexMetaData.getRoutingNumShards()) / indexMetaData.getRoutingFactor();

在https://github.com/elastic/elasticsearch/blob/master/core/src/main/java/org/elasticsearch/cluster/metadata/IndexMetaData.java 里可以看到getRoutingFactor实现：

    /**
     * Returns the routing factor for this index. The default is <tt>1</tt>.
     *
     * @see #getRoutingFactor(IndexMetaData, int) for details
     */
    public int getRoutingFactor() {
        return routingFactor;
    }

构造函数里有：

        assert numberOfShards * routingFactor == routingNumShards :  routingNumShards + " must be a multiple of " + numberOfShards;

反正默认是1，也就是所有的shard节点都会负责路由！

当心，ES2.4版本的路由实现：https://github.com/elastic/elasticsearch/blob/2.4/core/src/main/java/org/elasticsearch/cluster/routing/

    @SuppressForbidden(reason = "Math#abs is trappy")
    private int generateShardId(ClusterState clusterState, String index, String type, String id, @Nullable String routing) {
        IndexMetaData indexMetaData = clusterState.metaData().index(index);
        if (indexMetaData =http://www.mamicode.com/= null) {
            throw new IndexNotFoundException(index);
        }
        final Version createdVersion = indexMetaData.getCreationVersion();
        final HashFunction hashFunction = indexMetaData.getRoutingHashFunction();
        final boolean useType = indexMetaData.getRoutingUseType();

        final int hash;
        if (routing == null) {
            if (!useType) {
                hash = hash(hashFunction, id);
            } else {
                hash = hash(hashFunction, type, id);
            }
        } else {
            hash = hash(hashFunction, routing);
        }
        if (createdVersion.onOrAfter(Version.V_2_0_0_beta1)) {
            return MathUtils.mod(hash, indexMetaData.getNumberOfShards());
        } else {
            return Math.abs(hash % indexMetaData.getNumberOfShards());
        }
    }
    @Deprecated
    protected int hash(HashFunction hashFunction, String type, String id) {
        if (type == null || "_all".equals(type)) {
            throw new IllegalArgumentException("Can‘t route an operation with no type and having type part of the routing (for backward comp)");
        }
        return hashFunction.hash(type, id);
    }

而该hash function实现由：

DjbHashFunction.java

SimpleHashFunction.java

Murmur3HashFunction.java

三种。

Murmur3HashFunction.java

最新的ES 5.0路由算法底层实现


联系
我们