Distributed Erlang

Named nodes

A node is one running VM. Without a name it is nonode@nohost and cannot talk to anything: node() answers the atom, nodes() answers the empty list, and a ping to another node answers pang. The name comes from the command line, and -name needs a host with a dot while -sname takes the short host name.

Two nodes on one machine must differ in their names, and both must share the same cookie. This page uses the loopback address as the host, so the names stay the same on any machine.

erl -name alice@127.0.0.1 -setcookie coursecookie

Connect two nodes

Start a second node in another terminal with a different name and the same cookie. Nothing connects until one side asks: net_adm:ping connects and answers pong, or answers pang when the other node is not there.

erl -name bob@127.0.0.1 -setcookie coursecookie

On the first node, the connection shows up in nodes() and in the node monitor. The transcript below is a real session of two nodes on one machine.

(alice@127.0.0.1)1> node().
'alice@127.0.0.1'
(alice@127.0.0.1)2> nodes().
[]
(alice@127.0.0.1)3> net_kernel:monitor_nodes(true).
ok
(alice@127.0.0.1)4> net_adm:ping('bob@127.0.0.1').
pong
(alice@127.0.0.1)5> receive {nodeup, Node} -> Node after 1000 -> no_nodeup end.
'bob@127.0.0.1'
(alice@127.0.0.1)6> nodes().
['bob@127.0.0.1']

nodes() lists only the visible nodes. A connection is symmetric: once alice pinged bob, bob sees alice too, and both keep the link until one side stops or the network drops it.

Remote calls

A call to another node needs only the node name in front of the module: spawn, spawn_link, spawn_monitor and rpc:call all take a node as their first argument. rpc:call answers the value of the function, or {badrpc, Reason} when the call failed.

(alice@127.0.0.1)7> rpc:call('bob@127.0.0.1', lists, seq, [1, 3]).
[1,2,3]
(alice@127.0.0.1)8> {_Pid, Ref} = spawn_monitor('bob@127.0.0.1', fun() -> io:format("running on ~s~n", [node()]) end), receive {'DOWN', Ref, process, _Pid, Reason} -> Reason end.
running on bob@127.0.0.1
normal

The function ran on the other node, printed there, and the monitor reported a normal exit back on the caller. That is the whole idea of distributed Erlang: the same send and receive, the same links and monitors, only the node name is new. Notice that the code itself has to exist on both nodes, which is what a release and the code path of lesson 44 take care of.

A single node without a name still runs the same module: the calls simply find no peer. Save this module as cluster.erl and run it in a plain shell.

-module(cluster). -export([nodes_now/0, ping/1, remote_node/1]). nodes_now() -> nodes(). ping(Node) -> net_adm:ping(Node). remote_node(Node) -> rpc:call(Node, erlang, node, []).

erlc cluster.erl
erl

1> node().
nonode@nohost
2> nodes().
[]
3> cluster:ping('bob@127.0.0.1').
pang
4> cluster:remote_node('bob@127.0.0.1').
{badrpc,nodedown}

The pang and the badrpc are the normal answers of a node with no name and no peer, not exceptions. Start the same shell with -name and the same two calls answer pong and the name of the other node.

Cookies and hidden nodes

A node only accepts a connection from a node with the same cookie. The cookie is a shared secret read from ~/.erlang.cookie or from the -setcookie flag, and it is not a password: anyone who has the file is inside, so treat a cluster as one trust domain. Since OTP 25 distribution can also run over TLS, which authenticates and encrypts the link.

erlang:monitor_nodes/2 can ask for hidden nodes as well, and a node started with -hidden connects without appearing in the visible set. Hidden nodes are the way to attach a monitoring or management tool without joining the cluster the application sees.

Next: Capstone: todo CLI

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does node() answer on a VM started without a name?

    Answer

    nonode@nohost

    An unnamed node is nonode@nohost and cannot connect to other nodes.

  2. Which flag gives a node a long name?

    Answer

    -name

    -name takes a name with a host that contains a dot; -sname takes the short form.

  3. What does net_adm:ping/1 answer when the peer is not running?

    Answer

    pang

    pang is the normal answer for a node that cannot be reached.

  4. What does nodes() list?

    Answer

    The visible nodes this node is connected to

    It lists the visible connected nodes; hidden ones are asked for separately.

  5. What must two nodes share before they can connect?

    Answer

    The same cookie

    The cookie authenticates the connection, and a mismatch is refused.

  6. Which call runs a function on another node and returns its value?

    Answer

    rpc:call/4

    rpc:call(Node, Module, Function, Args) returns the value or {badrpc, Reason}.

  7. What does rpc:call/4 answer when the node is down?

    Answer

    {badrpc, nodedown}

    The failed call is reported as {badrpc, Reason} with nodedown as the reason.

  8. What does spawn_monitor(Node, Fun) do?

    Answer

    Runs the fun on the remote node and monitors it

    The fun runs on the given node, and its exit arrives as a DOWN message.

  9. What is the cookie in a cluster?

    Answer

    A shared secret read from ~/.erlang.cookie

    The cookie file holds the secret, and every node of the cluster must have the same one.

  10. What is a hidden node for?

    Answer

    Attaching without appearing in the visible node set

    A hidden node connects but stays out of nodes(), which suits tools and monitors.

The examples on this page were compiled and run with Erlang/OTP 27 (erts-15.2.7.14) on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev