Skip to main content

Instructions

Steps to operate a relay and join the Marlin network.

For clusters​

Step 1: Preliminaries​

  • Refer to the architecture diagram here and understand the interactions
  • Preferably, set up marlinctl2 (available here) to make the setup process smoother
  • If you prefer a manual setup or marlinctl2 doesn't cover your use case, clone and build OpenWeaver
  • In case you are using marlinctl2, install supervisord for your OS.

Step 2: Set up beacon​

  • Run a beacon node
  • Ensure the beacon is reachable from outside
  • Pass "34.82.79.68:8003" as the beacon address while starting the beacon to register it with the wider network
# Using marlinctl
$ sudo marlinctl beacon keystore create
$ sudo marlinctl beacon create --discovery-addr "0.0.0.0:8002" --heartbeat-addr "0.0.0.0:8003" --bootstrap-addr "34.82.79.68:8003"

# Manually
$ ./beacon/beacon --discovery-addr "0.0.0.0:8002" --heartbeat-addr "0.0.0.0:8003" --beacon-addr "34.82.79.68:8003"

Step 3: Set up relays​

  • Run a relay node
  • Set the discovery-addrs parameter to point to the discovery-addr of your beacon set up above
  • Set the heartbeat-addrs parameter to point to the heartbeat-addr of your beacon set up above
# Using marlinctl
$ sudo marlinctl relay eth create --discovery-addrs "beacon_public_ip:8002" --heartbeat-addrs "beacon_public_ip:8003"

# Manually
$ ./relay/eth_relay "beacon_public_ip:8002" "beacon_public_ip:8003" "/path/to/datadir/"

Step 4: Set up firewalls​

  • Expose for public access

    • Discovery port of beacon
    • Discovery port of relay
    • Pubsub port of relay

  • Expose only for relay access

    • Heartbeat port of beacon

Private network setup​

Make sure the setup above works fine before attempting the private network setup. Also ensure that your nodes can communicate with each other through private IPs.

Replace step 2:​

  • Run two beacons, one public facing and one private facing
# Using marlinctl
$ sudo marlinctl beacon keystore create
# Public beacon with bootstrap address
$ sudo marlinctl beacon create --discovery-addr "0.0.0.0:8002" --heartbeat-addr "0.0.0.0:8003" --bootstrap-addr "34.82.79.68:8003"
# Private beacon without bootstrap address
$ sudo marlinctl beacon create --discovery-addr "0.0.0.0:9002" --heartbeat-addr "0.0.0.0:9003"

# Manually
$ ./beacon/beacon --discovery-addr "0.0.0.0:8002" --heartbeat-addr "0.0.0.0:8003" --beacon-addr "34.82.79.68:8003"
$ ./beacon/beacon --discovery-addr "0.0.0.0:9002" --heartbeat-addr "0.0.0.0:9003"

Replace step 3:​

  • Set the discovery-addrs parameter to point to the discovery-addr of your private beacon set up above
  • Set the heartbeat-addrs parameter to point to the heartbeat-addr of both your public and private beacons set up above
# Using marlinctl
$ sudo marlinctl relay eth create --discovery-addrs "privateip:9002" --heartbeat-addrs "privateip:9003,publicip:8003"

# Manually
$ ./relay/eth_relay "privateip:9002" "privateip:9003,publicip:8003" ""

At the end of this, you should see your private beacon getting heartbeats from the private IPs of the relays and the public beacon getting heartbeats from the public IPs of the relays.

Tuning​

The linux UDP stack needs to be tuned for optimal performance. Here's a good reference for the parameters - https://gist.github.com/voluntas/bc54c60aaa7ad6856e6f6a928b79ab6c.

Note: Try to understand what you're changing and why before you change it, since it depends on your system characteristics if any of the commands there would improve or worsen performance (and stability, be wary of memory exhaustion).

The important bits seem to be:

  • Run watch netstat -us - Check if send or receive errors are increasing with time, indicates buffer overflows
  • If yes, increase buffer size (replace the numbers below as you please)
# Per-socket read buffers
net.core.rmem_default = 8192000
net.core.rmem_max = 8192000

# Per-socket write buffers
net.core.wmem_default = 8192000
net.core.wmem_max = 8192000

# Option memory
net.core.optmem_max = 8192000

# Global tuning (multiplied by 4KB)
net.ipv4.udp_mem = 64000    64000    64000
  • Restart the relays and check again after some time.