Creating a cluster¶
Starting the first node¶
First, extract the ZIP package and copy the licence provided into the license folder.
To create a single node cluster, first configure the name of the node. This is used when performing cluster operations using the API:
./bin/valo config set valo.node.name firstNode
In order to add other nodes, we must also configure the hostname of the node. This can be a host or IP address that will be reachable from the other nodes. It does not have to be reachable from API clients:
./bin/valo config set valo.node.host valo1.valo.io
Finally, simply run the binary:
You can tell when it is ready to use based on the log messages:
16:13:38.268 INFO - All front ends have started
For a quick introduction to creating streams and running queries using the API, see API Quickstart.
Starting an additional node¶
To add a new node into the cluster, first configure and start it in the same manner as the first node.
Valo is designed to scale out to use the resources available so we recommend deploying each node on a separate physical machine.
At a minimum, the new node must have a different name to the first node:
./bin/valo config set valo.node.name secondNode
with the hostname of the machine it is running on:
./bin/valo config set valo.node.host valo2.valo.io
Now we run the node as before:
Note: This will start a new one node cluster just as we did in the first step. Next we need to add the new node to the cluster.
Adding it to the cluster¶
To join the new node to the cluster we need to use the Cluster API. The cluster command provides access to this from the shell. Assuming the first node in step 1 was started on the default ports, we simply run:
./bin/valo cluster add -p 8888
This should join the node to the cluster. We have now increased the size of the cluster!
The new node is now
Joining. This means that existing nodes are transferring data to it to ensure an optimal usage of all storage resources in the cluster. The amount of data to transfer is in proportion to the amount of data stored and may take some time.
You can check on the progress of these transfers, again using the Cluster API. Once all transfers are complete, the node will move to
Joined and can receive API requests.
Multiple nodes can be added in a single request. In general this will be faster than adding each node individually, as the number of data transfers between nodes can be minimised. See Adding nodes.
Verify the cluster¶
At any stage, you can check the state of the cluster membership using the cluster command:
./bin/valo cluster state
You can also verify the cluster using the test command:
Once joined into a cluster, a node will remember the state of the cluster on disk. If it is restarted, it will join back in.