聚合根分布
介绍
Phoenix 支持自定义哈希函数，类似于 Kafka 的默认分区策略使用哈希函数来决定将消息/记录分配给主题内的指定分区，Phoenix 也使用哈希函数决定聚合根
分配到指定节点。
注意
需要注意的是，除了哈希函数，重新平衡的策略也会影响聚合根的分配，因此通过哈希函数决定的分配，不一定 100% 保证某一个聚合根一定在某一个节点上分配。
目前哈希函数的作用范围如下：
- Kafka 的分区策略
- EventPublish 工作节点的分配策略
- EventStore 中聚合根的分片策略
- 聚合根的分配策略
- 事务聚合根的分配策略
使用说明
| 配置项 | 描述 | 类型 | 默认值 |
|---|---|---|---|
quantex.phoenix.core.hash-function | 内置哈希函数 | enum | murmur2 可选[murmur2, dek, djb] |
quantex.phoenix.core.custom-hash-function | 自定义哈希函数, 优先级高于内置哈希函数 | string | 无 |
完整配置说明
quantex:
phoenix:
core:
# 默认哈希函数
hash-function: murmur2（可选：murmur2、dek、djb）
# 自定义哈希函数（Spring Bean 方式）
custom-hash-function: springHash
# 自定义哈希函数（反射方式）
custom-hash-function: com.iquantex.phoenix.samples.account.bean.FQCNHash
# 错误的自定义哈希函数，会降级成默认哈希函数
custom-hash-function: asqwwfqwf
使用案例 1: spring bean
import com.iquantex.phoenix.core.hash.HashFunction;
import org.springframework.stereotype.Component;
@Component("springHash") // 需要配置 quantex.phoenix.core.custom-hash-function = springHash
public class SpringHash implements HashFunction {
@Override
public int hash(String str) {
return 0;
}
}
使用案例 2: 反射方式
package com.iquantex.phoenix.samples.account.bean;
import com.iquantex.phoenix.core.hash.HashFunction;
// 需要配置 quantex.phoenix.core.custom-hash-function = com.iquantex.phoenix.samples.account.bean.FQCNHash
public class FQCNHash implements HashFunction {
@Override
public int hash(String str) {
return 1;
}
}
使用案例 3: 配置错误的自定义哈希类名，强制回退到默认
quantex:
phoenix:
core:
# 自定义了反射方式的哈希函数
custom-hash-function: com.iquantex.phoenix.samples.account.bean.FQCNHash
但是该哈希函数出现问题，希望通过环境变量强制回退到框架默认的方式，则：quantex.phoenix.core.custom-hash-function= asdqwjkfjkjfkqjfkqfnlknczxc 强制覆盖即可
内置函数选择
目前 Phoenix 内置了三种哈希函数:
- Murmur2: MurmurHash2 由 Austin Appleby 于 2008 年创建，用于 libmemcached、Maatkit 和 Apache Hadoop, Apache Kafka
- DEK Hash: DEK 是一种基于 Donald E. Knuth 提议的早期乘法哈希，是仍在使用的最古老的哈希之一。
- DJB Hash(DJBX33A): 由 Daniel J. Bernstein 教授提出的算法, Times 33 with Addition
目前 Phoenix 默认使用 Murmur2 函数，原因是其随机性、通用性比其他两个更好，下面是所有函数在不同聚合根数量和节点数量下的分配方差(表格值为: 总体分配方差、10种聚合根类型，每个类型的方差，然后不同类型再方差)：
可以看到 DEK 和 DJB 分别在 4 倍数和 3 倍数节点下出现有规律的异常分配.
Murmur Hash2
| 节点数量/实体数量 | 100 | 200 | 500 | 1000 | 2000 | 10000 | 20000 |
|---|---|---|---|---|---|---|---|
| 2 | {25.0, 532.81} | {16.0, 6757.84} | {16.0, 10838.64} | {961.0, 123353.85} | {225.0, 247142.01} | {88804.0, 1.654488424E7} | {24649.0, 7.751722461E7} |
| 3 | {2.89, 319.4} | {216.22, 1066.56} | {390.22, 6523.34} | {489.56, 28688.54} | {729.56, 143812.0} | {14588.22, 936560.33} | {8177.56, 8158095.84} |
| 4 | {141.5, 321.96} | {117.0, 355.37} | {228.5, 1319.42} | {2085.5, 19380.55} | {5948.5, 91736.65} | {39787.5, 1247781.57} | {21970.5, 6644807.9} |
| 5 | {16.4, 76.91} | {244.0, 204.3} | {461.2, 1027.4} | {747.6, 16272.5} | {967.2, 5779.24} | {17460.0, 781947.47} | {18821.2, 1904470.73} |
| 6 | {130.89, 54.69} | {341.56, 422.81} | {875.56, 3169.43} | {814.89, 5538.34} | {794.56, 6956.75} | {23680.22, 827509.94} | {15242.22, 1812137.51} |
| 7 | {124.98, 69.83} | {271.06, 297.57} | {415.63, 955.41} | {1282.24, 2635.75} | {2068.98, 17878.73} | {14428.78, 873799.2} | {20176.53, 6284840.77} |
| 8 | {62.0, 42.75} | {95.0, 37.84} | {163.0, 382.81} | {1981.75, 2257.24} | {4581.0, 10922.43} | {15557.75, 428093.69} | {19062.75, 693936.47} |
| 9 | {27.88, 7.35} | {141.51, 33.0} | {361.8, 304.36} | {465.43, 2163.33} | {1927.51, 3749.01} | {13171.88, 169621.44} | {20173.28, 1561073.76} |
| 10 | {14.2, 7.03} | {65.8, 12.7} | {274.2, 218.47} | {649.4, 1079.69} | {1178.8, 2947.95} | {13815.0, 123472.9} | {15478.2, 477566.0} |
| 11 | {17.36, 21.58} | {111.79, 38.47} | {438.79, 604.05} | {614.26, 1882.38} | {1178.15, 4266.87} | {7078.63, 102354.77} | {13536.51, 409299.51} |
| 12 | {64.06, 11.17} | {139.56, 16.91} | {319.72, 336.84} | {514.06, 600.37} | {1289.06, 2530.87} | {14490.39, 103865.47} | {7459.72, 170477.43} |
| 13 | {52.07, 11.54} | {105.67, 17.92} | {184.54, 142.66} | {928.33, 840.85} | {1992.4, 1581.25} | {11979.29, 51452.95} | {12861.16, 291491.13} |
| 14 | {49.82, 3.04} | {129.55, 21.04} | {459.12, 128.58} | {766.06, 404.1} | {1547.39, 2428.53} | {6276.69, 209628.84} | {9127.35, 453756.43} |
| 15 | {37.69, 2.86} | {86.36, 15.97} | {92.36, 85.28} | {271.96, 906.39} | {627.02, 984.12} | {3539.02, 27252.07} | {9319.82, 119280.72} |
| 16 | {46.25, 6.58} | {91.75, 10.81} | {232.0, 136.09} | {707.5, 639.71} | {1466.75, 2816.13} | {6379.75, 71598.11} | {7787.25, 29036.2} |
| 17 | {44.26, 6.62} | {47.4, 18.99} | {363.63, 42.5} | {876.42, 105.74} | {909.9, 1756.38} | {7972.58, 40163.47} | {12066.56, 64037.14} |
| 18 | {45.58, 0.98} | {114.99, 5.75} | {215.73, 137.74} | {311.58, 138.49} | {1091.88, 437.85} | {6393.58, 31662.7} | {8769.32, 171970.16} |
| 19 | {56.65, 4.27} | {82.93, 2.98} | {210.13, 62.14} | {491.27, 221.69} | {855.71, 1395.91} | {4398.13, 44273.59} | {12281.9, 105269.82} |
| 20 | {30.3, 2.02} | {42.1, 3.87} | {162.1, 41.29} | {417.0, 293.13} | {1024.4, 715.4} | {5471.9, 11129.1} | {7422.2, 75808.53} |
| 21 | {42.9, 2.78} | {96.09, 16.22} | {280.18, 106.42} | {512.34, 308.85} | {901.85, 457.51} | {4570.28, 15778.06} | {4906.82, 86680.45} |
| 22 | {53.61, 2.5} | {58.99, 1.53} | {148.74, 34.82} | {202.98, 144.8} | {434.99, 541.86} | {4777.79, 7799.64} | {6697.08, 93257.79} |
| 23 | {57.64, 1.3} | {96.91, 3.7} | {268.33, 50.47} | {499.65, 118.94} | {880.42, 368.69} | {1839.19, 10021.06} | {3673.18, 34544.55} |
DEK
| 节点数量/实体数量 | 100 | 200 | 500 | 1000 | 2000 | 10000 | 20000 |
|---|---|---|---|---|---|---|---|
| 2 | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} |
| 3 | {13.56, 1.31} | {2.89, 1.31} | {16.89, 1.31} | {33.56, 2.46} | {42.89, 2.46} | {89.56, 26.51} | {116.22, 26.51} |
| 4 | {400.0, 0.0} | {1600.0, 0.0} | {10000.0, 0.0} | {40000.0, 0.0} | {160000.0, 0.0} | {4000000.0, 0.0} | {1.6E7, 0.0} |
| 5 | {9.2, 0.32} | {8.8, 0.81} | {6.0, 0.84} | {26.0, 2.6} | {6.0, 0.84} | {24.0, 5.76} | {36.0, 31.36} |
| 6 | {13.56, 1.31} | {2.89, 1.31} | {16.89, 1.31} | {33.56, 2.46} | {42.89, 2.46} | {89.56, 26.51} | {116.22, 26.51} |
| 7 | {4.41, 0.18} | {2.2, 0.16} | {9.35, 1.19} | {3.96, 0.35} | {5.27, 1.86} | {27.92, 3.77} | {13.96, 0.44} |
| 8 | {225.0, 0.56} | {900.0, 0.0} | {5625.0, 0.56} | {22500.0, 0.0} | {90000.0, 0.0} | {2250000.0, 0.0} | {9000000.0, 0.0} |
| 9 | {7.65, 0.39} | {7.06, 0.28} | {13.8, 1.11} | {14.32, 0.45} | {29.51, 1.6} | {44.1, 4.87} | {17.95, 1.85} |
| 10 | {4.8, 0.35} | {5.2, 0.32} | {14.0, 1.76} | {10.0, 0.84} | {8.0, 1.76} | {16.0, 2.56} | {14.0, 5.76} |
| 11 | {14.63, 4.89} | {64.69, 62.36} | {275.34, 221.06} | {62.63, 69.57} | {100.69, 822.21} | {742.08, 16375.71} | {2381.06, 202939.91} |
| 12 | {48.56, 0.44} | {182.22, 0.66} | {1118.22, 0.83} | {4464.89, 0.71} | {17800.89, 0.98} | {444510.89, 8.63} | {1777837.89, 8.54} |
| 13 | {8.07, 0.86} | {12.44, 0.41} | {37.47, 6.19} | {37.41, 1.61} | {9.94, 2.5} | {20.06, 1.66} | {34.7, 5.2} |
| 14 | {3.1, 0.21} | {1.55, 0.39} | {6.69, 0.59} | {9.35, 1.06} | {4.53, 0.35} | {10.98, 0.63} | {12.2, 1.02} |
| 15 | {17.82, 0.39} | {31.69, 0.21} | {7.82, 0.5} | {13.69, 1.13} | {19.56, 0.43} | {17.82, 0.64} | {26.09, 4.16} |
| 16 | {318.75, 0.19} | {1275.0, 0.56} | {7968.75, 0.88} | {31875.0, 0.56} | {127500.0, 0.0} | {3187500.0, 0.0} | {1.275E7, 0.0} |
| 17 | {11.79, 0.31} | {15.4, 0.53} | {11.52, 0.12} | {14.18, 1.67} | {6.13, 0.22} | {13.64, 0.45} | {33.62, 7.33} |
| 18 | {6.69, 0.24} | {12.1, 0.86} | {19.95, 3.52} | {11.36, 1.33} | {12.1, 0.45} | {24.69, 1.6} | {19.21, 6.74} |
| 19 | {3.92, 0.4} | {0.93, 0.7} | {20.13, 1.68} | {13.48, 8.52} | {22.23, 135.6} | {189.61, 1110.36} | {781.9, 412.88} |
| 20 | {18.4, 0.16} | {65.8, 0.25} | {406.8, 0.47} | {1607.0, 0.61} | {6403.0, 0.32} | {160008.0, 1.56} | {640027.0, 5.64} |
| 21 | {4.71, 1.36} | {9.61, 0.96} | {46.18, 14.29} | {39.11, 96.71} | {11.76, 0.92} | {135.61, 35.68} | {70.34, 83.13} |
| 22 | {6.25, 0.53} | {18.08, 4.89} | {78.02, 17.03} | {21.7, 6.33} | {31.9, 69.57} | {211.52, 1357.7} | {672.99, 16375.71} |
| 23 | {2.95, 0.32} | {5.52, 0.61} | {19.72, 4.67} | {41.13, 29.45} | {27.72, 5.31} | {37.45, 19.66} | {73.27, 35.2} |
DJB
| 节点数量/实体数量 | 100 | 200 | 500 | 1000 | 2000 | 10000 | 20000 | |
|---|---|---|---|---|---|---|---|---|
| 2 | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | |
| 3 | {22.22, 0.05} | {88.89, 0.79} | {555.56, 0.31} | {2222.22, 0.05} | {8888.89, 0.79} | {222222.22, 0.05} | {888888.89, 0.79} | |
| 4 | {2.0, 0.25} | {4.0, 0.0} | {2.0, 0.25} | {4.0, 0.0} | {8.0, 0.0} | {8.0, 0.0} | {16.0, 0.0} | |
| 5 | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | |
| 6 | {5.56, 0.31} | {22.22, 0.05} | {138.89, 0.79} | {555.56, 0.31} | {2222.22, 0.05} | {55555.56, 0.31} | {222222.22, 0.05} | |
| 7 | {2.12, 0.04} | {2.2, 0.06} | {2.2, 0.06} | {1.96, 0.01} | {3.84, 0.04} | {3.92, 0.06} | {5.96, 0.17} | |
| 8 | {2.0, 0.25} | {6.0, 0.25} | {9.0, 0.06} | {6.0, 0.25} | {19.0, 0.56} | {19.0, 0.56} | {62.0, 0.25} | |
| 9 | {2.77, 0.29} | {11.06, 0.03} | {69.14, 0.34} | {276.54, 0.1} | {1106.17, 0.08} | {27654.32, 0.01} | {110617.28, 0.16} | |
| 10 | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | {0.0, 0.0} | |
| 11 | {26.45, 0.07} | {105.79, 0.0} | {661.16, 0.37} | {2644.63, 0.2} | {10578.51, 0.62} | {264462.81, 0.4} | {1057851.24, 0.26} | |
| 12 | {1.89, 0.31} | {6.56, 0.79} | {35.22, 0.79} | {139.89, 0.05} | {557.56, 0.05} | {13890.89, 0.31} | {55559.56, 0.79} | |
| 13 | {3.3, 0.27} | {0.44, 0.06} | {2.54, 0.16} | {3.1, 0.28} | {6.4, 0.81} | {4.52, 0.41} | {2.54, 0.16} | |
| 14 | {6.39, 0.07} | {13.27, 0.24} | {2.12, 0.04} | {4.35, 0.06} | {12.96, 0.3} | {30.98, 0.0} | {26.49, 0.01} | |
| 15 | {0.89, 0.79} | {3.56, 0.31} | {22.22, 0.05} | {88.89, 0.79} | {355.56, 0.31} | {8888.89, 0.79} | {35555.56, 0.31} | |
| 16 | {52.5, 0.0} | {201.5, 0.25} | {946.25, 0.88} | {1167.0, 0.25} | {4489.25, 0.06} | {26142.25, 0.14} | {100586.25, 0.56} | |
| 17 | {1.44, 0.08} | {4.35, 0.09} | {16.81, 0.06} | {24.42, 0.25} | {92.72, 0.34} | {646.7, 0.28} | {2497.15, 0.0} | |
| 18 | {8.47, 0.06} | {2.77, 0.29} | {25.06, 0.26} | {69.14, 0.34} | {276.54, 0.1} | {6913.58, 0.0} | {27654.32, 0.01} | |
| 19 | {1.71, 0.16} | {2.51, 0.32} | {3.29, 0.41} | {3.16, 0.19} | {4.76, 0.37} | {3.61, 0.41} | {4.53, 0.94} | |
| 20 | {22.8, 0.09} | {14.4, 0.0} | {14.8, 0.09} | {29.6, 0.36} | {27.2, 0.0} | {72.0, 0.04} | {208.0, 0.16} | |
| 21 | {1.95, 0.44} | {5.71, 0.12} | {15.23, 0.76} | {50.73, 0.0} | {189.76, 0.22} | {4537.23, 0.15} | {18141.77, 0.5} | |
| 22 | {6.61, 0.0} | {26.45, 0.07} | {165.29, 0.42} | {661.16, 0.37} | {2644.63, 0.2} | {66115.7, 0.03} | {264462.81, 0.4} | |
| 23 | {0.68, 0.05} | {0.74, 0.04} | {0.85, 0.04} | {1.13, 0.06} | {0.25, 0.0} | {0.84, 0.03} | {1.01, 0.06} |