Leafnode with JWT Auth (CLI)

Leafnode with JWT Auth in Topologies

This example demonstrates how to bootstrap the decentralized JWT authentication mode using the nsc tool, configuring a server to accept leaf node connections, configuring a server acting as the leaf node itself, and how a client making a request to the leaf node will get handled by a service connected to the main node.

CLI Go Python JavaScript Rust C# Java Ruby Elixir Crystal C

Jump to the output or the recording

$ nbe run topologies/leafnode-jwt/cli

View the source code or learn how to run this example yourself

Code

#!/bin/sh


set -euo pipefail

For this example, we are going to have a service connected to the main server and then another client send a request via a connection to the leaf node.

NATS_MAIN_URL="nats://0.0.0.0:4222"
NATS_LEAF_URL="nats://0.0.0.0:4223"

Create the operator, generate a signing key (which is a best practice), and initialize the default SYS account and sys user.

nsc add operator --generate-signing-key --sys --name local

A follow-up edit of the operator enforces signing keys are used for accounts as well. Setting the server URL is a convenience so that it does not need to be specified with call nsc push.

nsc edit operator --require-signing-keys \
  --account-jwt-server-url "$NATS_MAIN_URL"

Next we need to create an account intended for application usage. The SYS account should be used for operational purposes. These commands create the APP account, generates a signing key, and then creates a user named user.

nsc add account APP
nsc edit account APP --sk generate
nsc add user --account APP user

Check out the current settings of nsc.

nsc env

The nats CLI provides a way to manage different contexts by name. Here we define the server and the credentials (via nsc integration) (notice the operator/account/user hierarchy). We save two one for the main server and one for the leaf node. Note how didn’t provide credentials for the leaf node..

nats context save main-user \
  --server "$NATS_MAIN_URL" \
  --nsc nsc://local/APP/user 


nats context save main-sys \
  --server "$NATS_MAIN_URL" \
  --nsc nsc://local/SYS/sys


nats context save leaf-user \
  --server "$NATS_LEAF_URL"

This command generates the bit of configuration to be used by the server to setup the embedded JWT resolver.

nsc generate config --nats-resolver --sys-account SYS > resolver.conf

Create the most basic server config which enables leaf node connections and include the JWT resolver config.

echo 'Creating the main server conf...'
cat <<- EOF > main.conf
port: 4222
leafnodes: {
  port: 7422
}


include resolver.conf
EOF

The second config is for the leaf node itself. This needs to define the leaf node remotes which is the main server it will be connecting to.

echo 'Creating the leaf node conf...'
cat <<- EOF > leaf.conf
port: 4223
leafnodes: {
  remotes: [
    {
      url: "nats-leaf://0.0.0.0:7422",
      credentials: "/root/.local/share/nats/nsc/keys/creds/local/APP/user.creds"
    }
  ]
}
EOF

Start the main server first.

nats-server -c main.conf 2> /dev/null &
MAIN_PID=$!


sleep 1

We need to put up the APP account JWT we created previously to the main server so that the user credentials file used both for the client providing the service and the client making the request is trusted.

echo 'Pushing the account JWT...'
nsc push -a APP

Now we can start the leaf node which uses the credentials for the remote connection.

nats-server -c leaf.conf 2> /dev/null &
LEAF_PID=$!


sleep 1

Connecting directly to the main server with the user creds, we can create a simple service that will reply to any request published to greet with the text hello. This is put in the background since this will block while serving.

nats --context main-user reply 'greet' 'hello' &
SERVICE_PID=$!

Tiny sleep to ensure the service is connected.

sleep 1

For this CLI invocation, we connect to the leaf server and send a request to greet. Notice two things, one is that no credentials file is specified since the leaf server does not have authentication setup. Instead the leafnodes.remotes section of the config defines the main server and provides the credentials so that the leaf node is authenticated for forwarding messaging. This request will be transparently fullfilled by the service connected to the main server.

nats --context leaf-user request 'greet' ''

Finally stop the service and servers.

kill $SERVICE_PID
kill $LEAF_PID
kill $MAIN_PID

Output

[ OK ] generated and stored operator key "OA3OXA7A5QCHVLLSWEFMEQKBHF45COWCG5ASRY4CCI5HOEKXMHZVPGKD"
[ OK ] added operator "local"
[ OK ] When running your own nats-server, make sure they run at least version 2.2.0
[ OK ] created operator signing key: OCMY7K6YT5C6ANETDZ2ZGNGOYML4SK4DJ2GEQTFEDWE5R7UICVG2NYE5
[ OK ] created system_account: name:SYS id:ACLB2KKRNVAS3U455DSZVAAPSEY6H2QVTML3DAYBGFLATH5M6N2EUV5J
[ OK ] created system account user: name:sys id:UDWNUHBXQWDL6Q5IZODWSI3PRG3GKJTGAITP2CPPXJJJWWBWGRHV5FMD
[ OK ] system account user creds file stored in `~/.local/share/nats/nsc/keys/creds/local/SYS/sys.creds`
[ OK ] strict signing key usage set to: true
[ OK ] set account jwt server url to "nats://0.0.0.0:4222"
[ OK ] edited operator "local"
[ OK ] generated and stored account key "ADTBS7PIIH3UUU62N4FVWHY7LO44ZNK5WVZXUBFHMMDNCPIYW7TDYRNN"
[ OK ] added account "APP"
[ OK ] added signing key "ADTV2Q4YIYR5PI3WYYGUBSDMIE3BNXJRYKRQ5FWBSDATTHFOF5Z4S5BL"
[ OK ] edited account "APP"
[ OK ] generated and stored user key "UAPYRVBFK7EBRNGFMVOY5LZWJOT6AMT2TEJGSA4MKYNNLYIKEOKEQBKI"
[ OK ] generated user creds file `~/.local/share/nats/nsc/keys/creds/local/APP/user.creds`
[ OK ] added user "user" to account "APP"
+----------------------------------------------------------------------------------------------------------+
|                                             NSC Environment                                              |
+--------------------+-----+-------------------------------------------------------------------------------+
| Setting            | Set | Effective Value                                                               |
+--------------------+-----+-------------------------------------------------------------------------------+
| $NSC_CWD_ONLY      | No  | If set, default operator/account from cwd only                                |
| $NSC_NO_GIT_IGNORE | No  | If set, no .gitignore files written                                           |
| $NKEYS_PATH        | No  | ~/.local/share/nats/nsc/keys                                                  |
| $NSC_HOME          | No  | ~/.config/nats/nsc                                                            |
| $NATS_CA           | No  | If set, root CAs in the referenced file will be used for nats connections     |
|                    |     | If not set, will default to the system trust store                            |
| $NATS_KEY          | No  | If set, the tls key in the referenced file will be used for nats connections  |
| $NATS_CERT         | No  | If set, the tls cert in the referenced file will be used for nats connections |
+--------------------+-----+-------------------------------------------------------------------------------+
| From CWD           |     | No                                                                            |
| Default Stores Dir |     | ~/.local/share/nats/nsc/stores                                                |
| Current Store Dir  |     | ~/.local/share/nats/nsc/stores                                                |
| Current Operator   |     | local                                                                         |
| Current Account    |     | APP                                                                           |
| Root CAs to trust  |     | Default: System Trust Store                                                   |
+--------------------+-----+-------------------------------------------------------------------------------+

NATS Configuration Context "main-user"

      Server URLs: nats://0.0.0.0:4222
      Credentials: /root/.local/share/nats/nsc/keys/creds/local/APP/user.creds (OK)
       NSC Lookup: nsc://local/APP/user
             Path: /root/.config/nats/context/main-user.json

NATS Configuration Context "main-sys"

      Server URLs: nats://0.0.0.0:4222
      Credentials: /root/.local/share/nats/nsc/keys/creds/local/SYS/sys.creds (OK)
       NSC Lookup: nsc://local/SYS/sys
             Path: /root/.config/nats/context/main-sys.json

NATS Configuration Context "leaf-user"

      Server URLs: nats://0.0.0.0:4223
             Path: /root/.config/nats/context/leaf-user.json

Creating the main server conf...
Creating the leaf node conf...
Pushing the account JWT...
[ OK ] push to nats-server "nats://0.0.0.0:4222" using system account "SYS":
       [ OK ] push APP to nats-server with nats account resolver:
              [ OK ] pushed "APP" to nats-server NDMOTRWUQLZEECONNOBQMSYJCSJLSDWHY2UYHAZ22NROKLP4EVY74SPH: jwt updated
              [ OK ] pushed to a total of 1 nats-server
13:39:46 Listening on "greet" in group "NATS-RPLY-22"
13:39:47 Sending request on "greet"
13:39:47 [#0] Received on subject "greet":


hello

13:39:47 Received with rtt 332.349µs

Recording

Note, playback is half speed to make it a bit easier to follow.