Getting Started
Build
Install the compress library
yum install snappy-devel (for CentOS) apt-get install libsnappy1 libsnappy-dev (for Debian/Ubuntu) brew install snappy (for Mac)
Build the rocksdb with jemalloc
git clone https://github.com/absolute8511/rocksdb.git cd rocksdb git checkout v6.4.6-patched PORTABLE=1 USE_SSE=1 USE_PCLMUL=1 WITH_JEMALLOC_FLAG=1 JEMALLOC=1 make static_lib
Install the dependency (for old go version only, if using go1.13+, it will be done in go modules):
CGO_CFLAGS="-I/path/to/rocksdb/include" CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lsnappy -lrt -ljemalloc" go get github.com/youzan/gorocksdbCGO_CFLAGS="-I/path/to/rocksdb/include" CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lsnappy -ljemalloc" go get github.com/youzan/gorocksdb (for MacOS)
use the dep ensure
to install other dependencies, or use go modules for go1.13+
Build zankv and placedriver from the source (only support go version 1.10.8+, gcc 4.9+ or xcode-command-line-tools on Mac):
ROCKSDB=/path/to/rocksdb make
If you want package the binary release run the scripts
./pre-dist.sh ROCKSDB=/path/to/rocksdb ./dist.sh
Deploy
- Deploy the rsync daemon which is needed on all server node to transfer the snapshot data for raft
Example config for rsync as below, and start rsync as daemon using sudo rsync --daemon
pid file = /var/run/rsyncd.pid
port = 873
log file = /var/log/rsync.log
list = yes
hosts allow= *
max connections = 1024
log format = %t %a %m %f %b
syslog facility = local3
timeout = 300
# module name must be exactly same as the data_rsync_module in zankv
[zankv]
## this path must be exactly same as the data_dir in zankv node
path = /data/zankv/
read only = yes
list=yes
log file = /data/logs/rsyncd/zankv.log
exclude = /data/zankv/myid myid /zankv/myid
- Deploy etcd cluster which is needed for the meta data for the namespaces
Please refer the etcd documents.
- Deploy the placedriver which is used for data placement:
placedriver -config=/path/to/config
Example config as below:
## <addr>:<port> to listen on for HTTP clients
http_address = "0.0.0.0:13801"
## the network interface for broadcast, the ip will be detected automatically.
broadcast_interface = "eth0"
cluster_id = "test-default"
## the etcd cluster ip list
cluster_leadership_addresses = "http://127.0.0.1:2379,http://127.0.0.2:2379"
## the detail of the log, larger number means more details
log_level = 2
## if empty, use the default flag value in glog
log_dir = "/data/logs/zankv"
## the time period (in hour) that the balance is allowed.
balance_interval = ["1", "23"]
auto_balance_and_migrate = true
- Deploy the zankv for data storage server
zankv -config=/path/to/config
Example config for zankv as below:
{
"server_conf": {
"broadcast_interface":"eth0",
"cluster_id":"test-default",
"etcd_cluster_addresses":"http://127.0.0.1:2379,http://127.0.0.2:2379",
"data_dir":"/data/zankv",
"data_rsync_module":"zankv",
"redis_api_port": 13381,
"http_api_port": 13380,
"grpc_api_port": 13382,
"election_tick": 30,
"tick_ms": 200,
"local_raft_addr": "http://0.0.0.0:13379",
"rocksdb_opts": {
"block_cache":0,
"use_shared_cache":true,
"use_shared_rate_limiter":true,
"rate_bytes_per_sec":50000000,
"cache_index_and_filter_blocks": false
}
}
}
You should at least 3 zankv nodes if you want make replicator=3.
- Init a namespace by sending the create the namespace API to placedriver
curl -X POST '127.0.0.1:13801/cluster/namespace/create?namespace=test_p4&partition_num=4&replicator=3'
API
placedriver has several HTTP APIs to manager the namespace
- list the namespace:
GET /namespaces
- list the data nodes:
GET /datanodes
- list the placedriver nodes:
GET /listpd
- query the namespace meta info:
GET /query/namespace_name
- create the namespace (handle only by leader) :
POST /cluster/namespace/create?namespace=test_p16&partition_num=16&replicator=3
- create the namespace with the new dynamic ttl support (handle only by leader) :
POST /cluster/namespace/create?namespace=test_p16&partition_num=16&replicator=3&data_version=value_header_v1&expiration_policy=wait_compact
- delete the namespace (handle only by leader):
DELETE /cluster/namespace/delete?namespace=test_p16&partition=**
storage server HTTP APIs for stats:
- namespace stats :
GET /stats
- namespace raft internal stats :
GET /raft/stats
- optimize the data storage :
POST /kv/optimize
- get the raft leader of the namespace partition:
GET /cluster/leader/namespace-partition
storage server also support the redis apis for read/write :
- KV:
- Hash Set:
- List:
- Sorted Set:
- ZSet:
Client
Golang client SDK : client-sdk , a redis proxy can be deployed based on this golang sdk if you want use the redis client in other language.