본문 바로가기
리눅스

레디스 클러스터를 설정하는 방법(redis cluster setup)

변군Dev 2022. 10. 27. 19:09
728x90

레디스 클러스터를 설정하는 방법(redis cluster setup)

테스트 환경

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

레디스 클러스터 구성도

레디스 클러스터 서버 현황

호스트명 IP node#1 node#2 비고
pnode1 192.168.0.51 6381 6382  
pnode2 192.168.0.74 6381 6382  
pnode3 192.168.0.75 6381 6382  

 

Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli

 

커널 파라미터 확인

vim /etc/sysctl.conf
net.core.somaxconn=8192
net.ipv4.tcp_max_syn_backlog=8192
net.core.netdev_max_backlog=32768
vm.overcommit_memory=1

ulimit 설정 확인

cat <<EOF >> /etc/security/limits.conf

*       soft    nproc   unlimited
*       hard    nproc   unlimited
*       soft    nofile  65536
*       hard    nofile  65536
EOF

Config, Log, Data 디렉터리 생성

mkdir -pv /var/lib/redis/{6379,6380}
mkdir -pv /var/log/redis
mkdir -pv /etc/redis
cp /usr/local/src/redis-6.2.7redis.conf /etc/redis/redis-6381.conf
cp /usr/local/src/redis-6.2.7redis.conf /etc/redis/redis-6382.conf

redis-6381.conf 편집

vim /etc/redis/redis-6381.conf
bind 0.0.0.0
port 6381
daemonize yes

tcp-backlog 512
timeout 0
tcp-keepalive 300

dir /var/lib/redis/6381

cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 3000

pidfile /var/run/redis_6381.pid

loglevel notice
logfile /var/log/redis-6381.log

appendonly yes
appendfilename appendonly.aof
appendfsync everysec

redis-6382.conf 편집

vim /etc/redis/redis-6382.conf
bind 0.0.0.0
port 6382
daemonize yes

tcp-backlog 512
timeout 0
tcp-keepalive 300

dir /var/lib/redis/6382

cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 3000

pidfile /var/run/redis_6382.pid

loglevel notice
logfile /var/log/redis-6382.log

appendonly yes
appendfilename appendonly.aof
appendfsync everysec
더보기

---

vim cluster-setup.sh
#!/bin/bash

mkdir cluster
cd cluster

# 포트를 다르게 설정하여 6개의 Redis 인스턴스를 생성합니다.
for port in {7000..7005}; do
    mkdir ${port}
    cat > ${port}/redis.conf <<EOF
port ${port}
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
EOF
done
bash +x cluster-setup.sh
vim cluster-instance-start.sh
#!/bin/bash

# 각 디렉토리로 이동하여 Redis 인스턴스를 실행합니다.
for port in {7000..7005}; do
    src/redis-server ${port}/redis.conf
done
bash -x cluster-instance-start.sh

---

redis server start

/usr/local/bin/redis-server /etc/redis/redis-6381.conf
/usr/local/bin/redis-server /etc/redis/redis-6382.conf

redis 로그 확인

$ cat /var/log/redis-6381.log
215410:C 26 Oct 2022 23:10:47.997 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
215410:C 26 Oct 2022 23:10:47.997 # Redis version=6.2.7, bits=64, commit=00000000, modified=0, pid=215410, just started
215410:C 26 Oct 2022 23:10:47.997 # Configuration loaded
215410:M 26 Oct 2022 23:10:47.999 * Increased maximum number of open files to 10032 (it was originally set to 1024).
215410:M 26 Oct 2022 23:10:47.999 * monotonic clock: POSIX clock_gettime
215410:M 26 Oct 2022 23:10:48.004 * No cluster configuration found, I'm f3ee1bbf6718735d1944846f449bbac8063029de
215410:M 26 Oct 2022 23:10:48.013 # A key '__redis__compare_helper' was added to Lua globals which is not on the globals allow list nor listed on the deny list.
215410:M 26 Oct 2022 23:10:48.013 * Running mode=cluster, port=6381.
215410:M 26 Oct 2022 23:10:48.013 # Server initialized
215410:M 26 Oct 2022 23:10:48.014 * Ready to accept connections

프로세스 확인

ps -ef | grep -v grep | egrep redis-server
$ ps -ef | grep -v grep | egrep redis-server
root      215410       1  0 23:10 ?        00:00:00 /usr/local/bin/redis-server 0.0.0.0:6381 [cluster]
root      221396       1  0 23:13 ?        00:00:00 /usr/local/bin/redis-server 0.0.0.0:6382 [cluster]

포트 확인

netstat -nltp | egrep redis
$ netstat -nltp | egrep redis
tcp        0      0 0.0.0.0:16382           0.0.0.0:*               LISTEN      221396/redis-server
tcp        0      0 0.0.0.0:16381           0.0.0.0:*               LISTEN      215410/redis-server
tcp        0      0 0.0.0.0:6382            0.0.0.0:*               LISTEN      221396/redis-server
tcp        0      0 0.0.0.0:6381            0.0.0.0:*               LISTEN      215410/redis-server
728x90

master 노드 추가

redis-cli --cluster create [Redis_Server_IP_#1]:6381 [Redis_Server_IP_#2]:6381 [Redis_Server_IP_#3]:6381

pnode1(192.168.0.51)

redis-cli --cluster create 192.168.0.51:6381 192.168.0.74:6381 192.168.0.75:6381
$ redis-cli --cluster create 192.168.0.51:6381 192.168.0.74:6381 192.168.0.75:6381
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
M: f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381
   slots:[0-5460] (5461 slots) master
M: 041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381
   slots:[5461-10922] (5462 slots) master
M: 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381
   slots:[10923-16383] (5461 slots) master
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.0.51:6381)
M: f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381
   slots:[0-5460] (5461 slots) master
M: 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381
   slots:[10923-16383] (5461 slots) master
M: 041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

slave 노드 추가

redis-cli --cluster add-node [Redis_Server_IP_#3]:6382 [Redis_Server_IP_#1]:6381 --cluster-slave

redis-cli --cluster add-node [Redis_Server_IP_#1]:6382 [Redis_Server_IP_#2]:6381 --cluster-slave

redis-cli --cluster add-node [Redis_Server_IP_#2]:6382 [Redis_Server_IP_#3]:6381 --cluster-slave

pnode1(192.168.0.51)

redis-cli --cluster add-node 192.168.0.74:6382 192.168.0.51:6381 --cluster-slave --cluster-master-id $(redis-cli -p 6381 cluster nodes | grep 192.168.0.51 | grep master | awk '{print $1}')
$ redis-cli --cluster add-node 192.168.0.74:6382 192.168.0.51:6381 --cluster-slave --cluster-master-id $(redis-cli -p 6381 cluster nodes | grep 192.168.0.51 | grep master | awk '{print $1}')
>>> Adding node 192.168.0.74:6382 to cluster 192.168.0.51:6381
>>> Performing Cluster Check (using node 192.168.0.51:6381)
M: f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381
   slots:[0-5460] (5461 slots) master
M: 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381
   slots:[10923-16383] (5461 slots) master
M: 041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.74:6382 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 192.168.0.51:6381.
[OK] New node added correctly.

pnode2(192.168.0.74)

redis-cli --cluster add-node 192.168.0.75:6382 192.168.0.74:6381 --cluster-slave --cluster-master-id $(redis-cli -p 6381 cluster nodes | grep 192.168.0.74 | grep master | awk '{print $1}')

pnode3(192.168.0.75)

redis-cli --cluster add-node 192.168.0.51:6382 192.168.0.75:6381 --cluster-slave --cluster-master-id $(redis-cli -p 6381 cluster nodes | grep 192.168.0.75 | grep master | awk '{print $1}')

클러스터 노드 확인

redis-cli -p 6381 cluster nodes
$ redis-cli -p 6381 cluster nodes
f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381@16381 myself,master - 0 1666794298000 1 connected 0-5460
f6b6efbb92768db1925682b48f45a80e02d5217c 192.168.0.51:6382@16382 slave 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 0 1666794299000 3 connected
0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381@16381 master - 0 1666794300474 3 connected 10923-16383
c591902cc55d2fd74429745c3a1ac44352bbc679 192.168.0.74:6382@16382 slave f3ee1bbf6718735d1944846f449bbac8063029de 0 1666794299809 1 connected
041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381@16381 master - 0 1666794299305 2 connected 5461-10922
e8cfa56223ed8670d9a4757c876019c8b115b20f 192.168.0.75:6382@16382 slave 041c618f6e2e38d795ed47a38424650bfe170ae7 0 1666794299106 2 connected

클러스터 정보 확인

redis-cli -p 6381 cluster info
$ redis-cli -p 6381 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:3
cluster_my_epoch:1
cluster_stats_messages_ping_sent:1182
cluster_stats_messages_pong_sent:1155
cluster_stats_messages_sent:2337
cluster_stats_messages_ping_received:1152
cluster_stats_messages_pong_received:1182
cluster_stats_messages_meet_received:3
cluster_stats_messages_received:2337

슬롯(slot) 확인

redis-cli -h 192.168.0.51 -p 6381 cluster slots
$ redis-cli -h 192.168.0.51 -p 6381 cluster slots
1) 1) (integer) 0
   2) (integer) 5460
   3) 1) "192.168.0.51"
      2) (integer) 6381
      3) "f3ee1bbf6718735d1944846f449bbac8063029de"
   4) 1) "192.168.0.74"
      2) (integer) 6382
      3) "c591902cc55d2fd74429745c3a1ac44352bbc679"
2) 1) (integer) 5461
   2) (integer) 10922
   3) 1) "192.168.0.74"
      2) (integer) 6381
      3) "041c618f6e2e38d795ed47a38424650bfe170ae7"
   4) 1) "192.168.0.75"
      2) (integer) 6382
      3) "e8cfa56223ed8670d9a4757c876019c8b115b20f"
3) 1) (integer) 10923
   2) (integer) 16383
   3) 1) "192.168.0.75"
      2) (integer) 6381
      3) "0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325"
   4) 1) "192.168.0.51"
      2) (integer) 6382
      3) "f6b6efbb92768db1925682b48f45a80e02d5217c"

핑(ping) 테스트

redis-cli -h 192.168.0.51 -p 6381 ping
$ redis-cli -h 192.168.0.51 -p 6381 ping
PONG

레디스 클러스터 테스트

SET

redis-cli -h 192.168.0.51 -p 6381 -c set foo bar
$ redis-cli -h 192.168.0.51 -p 6381 -c set foo bar
OK
redis-cli -h 192.168.0.51 -p 6381 -c set hello world
$ redis-cli -h 192.168.0.51 -p 6381 -c set hello world
OK

GET

redis-cli -h 192.168.0.51 -p 6381 -c get foo
$ redis-cli -h 192.168.0.51 -p 6381 -c get foo
"bar"
redis-cli -h 192.168.0.51 -p 6381 -c get hello
$ redis-cli -h 192.168.0.51 -p 6381 -c get hello
"world"

KEYS

redis-cli -h 192.168.0.51 -p 6381 -c
$ redis-cli -h 192.168.0.51 -p 6381 -c
192.168.0.51:6381> keys *
1) "hello"
192.168.0.51:6381> get foo
-> Redirected to slot [12182] located at 192.168.0.75:6381
"bar"
192.168.0.75:6381> keys *
1) "foo"
192.168.0.75:6381> save
OK
192.168.0.75:6381

pnode3(192.168.0.75) 서버 강제 종료

- 192.168.0.75:6381 노드가 192.168.0.51:6382 노드로 승격됨(슬레이브 -> 마스터)

$ redis-cli -p 6381 cluster nodes
f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381@16381 myself,master - 0 1666797240000 1 connected 0-5460
f6b6efbb92768db1925682b48f45a80e02d5217c 192.168.0.51:6382@16382 master - 0 1666797240000 4 connected 10923-16383
0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381@16381 master,fail - 1666797191581 1666797190000 3 disconnected
c591902cc55d2fd74429745c3a1ac44352bbc679 192.168.0.74:6382@16382 slave f3ee1bbf6718735d1944846f449bbac8063029de 0 1666797240760 1 connected
041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381@16381 master - 0 1666797240254 2 connected 5461-10922
e8cfa56223ed8670d9a4757c876019c8b115b20f 192.168.0.75:6382@16382 slave,fail 041c618f6e2e38d795ed47a38424650bfe170ae7 1666797191581 1666797190000 2 disconnected
$ redis-cli -p 6381 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:4
cluster_my_epoch:1
cluster_stats_messages_ping_sent:9916
cluster_stats_messages_pong_sent:9804
cluster_stats_messages_fail_sent:3
cluster_stats_messages_auth-ack_sent:1
cluster_stats_messages_sent:19724
cluster_stats_messages_ping_received:9801
cluster_stats_messages_pong_received:9915
cluster_stats_messages_meet_received:3
cluster_stats_messages_fail_received:2
cluster_stats_messages_auth-req_received:1
cluster_stats_messages_received:19722

 

참고URL

- [리눅스] 레디스 서버 설치(소스 컴파일) : https://scbyun.com/1368

 

728x90
저작자표시 비영리 동일조건

'리눅스' 카테고리의 다른 글

Ansible 인벤토리를 YAML 파일로 설정하는 방법  (0) 2022.10.31
Ansible 구성 설정(Configuration Settings) 파일의 우선 순위에 대한 설명  (0) 2022.10.31
우분투에서 Redis 서버를 소스 코드로 컴파일하여 설치하는 방법(소스 컴파일)  (0) 2022.10.27
[draft] 우분투에서 SquashFS를 사용하는 방법  (0) 2022.10.24
리눅스 소스 컴파일 중 "c++: fatal error: Killed signal terminated program cc1plus" 에러  (0) 2022.10.24

'리눅스' Related Articles

티스토리툴바