Cluster Administration Commands
This document explains usage of the riak-admin cluster
interface, which enables you to perform a wide
variety of cluster-level actions.
How Cluster Administration Works
Riak provides a multi-phased approach to cluster administration that enables you to stage and review cluster-level changes prior to committing them. This allows you to group multiple changes together, such as adding multiple nodes at once, adding some nodes and removing others, etc.
Enacting cluster-level changes typically follows this set of steps:
- Choose an action or set of actions, such as adding a node, removing multiple nodes, etc. These actions will be staged rather than executed immediately.
- Plan the changes using the
cluster plan
command. This will return a list of staged commands that you can review. - Commit the changes using the
cluster commit
command. This will execute the changes that have been staged and reviewed.
Note on command names
Many of the commands available through the
riak-admin cluster
interface are also available as self-standing commands. Theriak-admin member-status
command is now theriak-admin cluster status
command,riak-admin join
is nowriak-admin cluster join
, etc.We recommend using the
riak-admin cluster
interface over the older, deprecated commands. You will receive a deprecation warning if you use the older commands.
status
Displays a variety of information about the cluster.
riak-admin cluster status
This will return output like the following in a 3-node cluster:
---- Cluster Status ----
Ring ready: true
+--------------------+------+-------+-----+-------+
| node |status| avail |ring |pending|
+--------------------+------+-------+-----+-------+
| (C) dev1@127.0.0.1 |valid | up | 34.4| -- |
| dev2@127.0.0.1 |valid | up | 32.8| -- |
| dev3@127.0.0.1 |valid | up | 32.8| -- |
+--------------------+------+-------+-----+-------+
In the above output, Ring ready
denotes whether or not the cluster
agrees on the ring, i.e. whether the cluster is
ready to begin taking requests.
The following information is then displayed for each node, by nodename
(in this case dev1@127.0.0.1
, etc.):
status
— There are five possible values for status:valid
— The node has begun participating in cluster operationsleaving
— The node is is currently unloading ownership of its data partitions to other nodesexiting
— The node’s ownership transfers are complete and it is currently shutting downjoining
— The node is in the process of joining the cluster but but has not yet completed the join processdown
— The node is not currently responding
avail
— There are two possible values:up
if the node is available and taking requests anddown!
if the node is unavailablering
— What percentage of the Riak ring the node is responsible forpending
— The number of pending transfers to or from the node
In addition, the cluster’s claimant node node will have a (C)
next
to it.
join
Joins the current node to another node in the cluster.
riak-admin cluster join <node>
You must specify a node to join to by nodename. You can join to any
node in the cluster. The following would join the current node to
riak1@127.0.0.1
:
riak-admin cluster join riak1@127.0.0.1
Once a node joins, all of the operations necessary to establish communication with all other nodes proceeds automatically.
Note: As with all cluster-level actions, the changes made when you run the
cluster join
command will take effect only after you have both planned the changes by runningriak-admin cluster plan
and committed the changes by runningriak-admin cluster commit
. You can stage multiple joins before planning/committing.
leave
Instructs the current node to hand off its data partitions, leave the cluster, and shut down.
riak-admin cluster leave
You can also instruct another node (by nodename) to leave the cluster:
riak-admin cluster leave <node>
Note: As with all cluster-level actions, the changes made when you run the
cluster leave
command will take effect only after you have both planned the changes by runningriak-admin cluster plan
and committed the changes by runningriak-admin cluster commit
. You can stage multiple leave command before planning/committing.
force-remove
Removes another node from the cluster (by nodename) without first handing off its data partitions. This command is designed for crashed, unrecoverable nodes and should be used with caution.
riak-admin cluster force-remove <node>
Note: As with all cluster-level actions, the changes made when you run the
cluster force-remove
command will take effect only after you have both planned the changes by runningriak-admin cluster plan
and committed the changes by runningriak-admin cluster commit
. You can stage multiple force-remove actions before planning/committing.
replace
Instructs a node to transfer all of its data partitions to another node and then to leave the cluster and shut down.
riak-admin cluster replace <node1> <node2>
Note: As with all cluster-level actions, the changes made when you run the
cluster replace
command will take effect only after you have both planned the changes by runningriak-admin cluster plan
and committed the changes by runningriak-admin cluster commit
. You can stage multiple replace actions before planning/committing.
force-replace
Reassigns all data partitions owned by one node to another node without first handing off data.
riak-admin cluster force-replace <node_being_replaced> <replacement_node>
Once the data partitions have been reassigned, the node that is being replaced will be removed from the cluster.
Note: As with all cluster-level actions, the changes made when you run the
cluster force-replace
command will take effect only after you have both planned the changes by runningriak-admin cluster plan
and committed the changes by runningriak-admin cluster commit
. You can stage multiple force-replace actions before planning/committing.
plan
Displays the currently staged cluster changes.
riak-admin cluster plan
riak-admin cluster plan
is complex, depending on the staged changes.
- If a
leave
operation has been staged,riak-admin cluster plan
will undo the staged change and no node will be stopped. - If a
join
operation has been staged, the joining node will be shut down after its ring has been cleared. When this node restarts, it will behave like a fresh unjoined node and can be joined again. - If a
cluster clear
operation is staged on a node that remains in the cluster, runningriak-admin cluster plan
will leave the node unaffected.
If there is no current cluster plan, the output will be There are no
staged changes
.
If there is a staged change (or changes), however, you will see a detailed listing of what will take place upon commit, what the cluster will look like afterward, etc.
For example, if a cluster leave
operation is staged in a 3-node cluster the output will look something like this:
=============================== Staged Changes ================================
Action Details(s)
-------------------------------------------------------------------------------
leave 'dev2@127.0.0.1'
-------------------------------------------------------------------------------
NOTE: Applying these changes will result in 2 cluster transitions
###############################################################################
After cluster transition 1/2
###############################################################################
================================= Membership ==================================
Status Ring Pending Node
-------------------------------------------------------------------------------
leaving 32.8% 0.0% 'dev2@127.0.0.1'
valid 34.4% 50.0% 'dev1@127.0.0.1'
valid 32.8% 50.0% 'dev3@127.0.0.1'
-------------------------------------------------------------------------------
Valid:2 / Leaving:1 / Exiting:0 / Joining:0 / Down:0
WARNING: Not all replicas will be on distinct nodes
Transfers resulting from cluster changes: 38
6 transfers from 'dev1@127.0.0.1' to 'dev3@127.0.0.1'
11 transfers from 'dev3@127.0.0.1' to 'dev1@127.0.0.1'
5 transfers from 'dev2@127.0.0.1' to 'dev1@127.0.0.1'
16 transfers from 'dev2@127.0.0.1' to 'dev3@127.0.0.1'
###############################################################################
After cluster transition 2/2
###############################################################################
================================= Membership ==================================
Status Ring Pending Node
-------------------------------------------------------------------------------
valid 50.0% -- 'dev1@127.0.0.1'
valid 50.0% -- 'dev3@127.0.0.1'
-------------------------------------------------------------------------------
Valid:2 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
WARNING: Not all replicas will be on distinct nodes
Notice that there are distinct sections of the output for each of the transitions that the cluster will undergo, including warnings, planned data transfers, etc.
commit
Commits the currently staged cluster changes. Staged cluster changes
must be reviewed using riak-admin cluster plan
prior to being committed.
riak-admin cluster commit
clear
Clears the currently staged cluster changes.
riak-admin cluster clear
partitions
Prints primary, secondary, and stopped partition indices and IDs either for the current node or for another, specified node. The following prints that information for the current node:
riak-admin cluster partitions
This would print the partition information for a different node in the cluster:
riak-admin cluster partitions --node=<node>
Partition information is contained in a table like this:
Partitions owned by 'dev1@127.0.0.1':
+---------+-------------------------------------------------+--+
| type | index |id|
+---------+-------------------------------------------------+--+
| primary | 0 |0 |
| primary | 91343852333181432387730302044767688728495783936 |4 |
| primary |182687704666362864775460604089535377456991567872 |8 |
| ... | .... |..|
| primary |1438665674247607560106752257205091097473808596992|63|
|secondary| -- |--|
| stopped | -- |--|
+---------+-------------------------------------------------+--+
partition-count
Displays the current partition count either for the whole cluster or for a particular node. This would display the partition count for the cluster:
riak-admin cluster partition-count
This would display the count for a node:
riak-admin cluster partition-count --node=<node>
When retrieving the partition count for a node, you’ll see a table like this:
+--------------+----------+-----+
| node |partitions| pct |
+--------------+----------+-----+
|dev1@127.0.0.1| 22 | 34.4|
+--------------+----------+-----+
The partitions
column displays the number of partitions claimed by the
node, while the pct
column displays the percentage of the ring claimed.
partition
The cluster partition
command enables you to convert partition IDs to
indexes and vice versa using the partition id
and partition index
commands, respectively. Let’s say that you run the riak-admin cluster
partitions
command and see that you have a variety of partitions, one
of which has an index of
1004782375664995756265033322492444576013453623296
. You can convert
that index to an ID like this:
riak-admin cluster partition index=1004782375664995756265033322492444576013453623296
Conversely, if you have a partition with an ID of 20, you can retrieve the corresponding index:
riak-admin cluster partition id=20