A cluster of servers

A cluster is two or more OpenBI Servers serving the same domains behind a load balancer. Any node answers any request, and when one fails the others serve everyone, still signed in. The nodes share one database, which holds the records, the sign-in’s keys and what is published.

How it works

• One database. The nodes keep the server’s records, and OpenBI Accounts’, in one database they share. Its own high availability, such as SQL Server Always On or PostgreSQL replication, is what keeps the cluster’s records safe.
• What is published is in the database too: each version’s file and each refresh’s data, by its SHA-256, in 4 MB chunks, each file once. Every node keeps a copy, fetches what it lacks when it is asked for, and looks for new versions and data every 10 seconds. A node that joins fills itself, and losing a node loses nothing.
• Sign-in survives a failover. The keys behind people’s sessions, the report and draft passes, invitations and password links are in the shared database, sealed with the cluster’s key, which every node holds. Any node reads any node’s cookie: nobody signs in again.
• Jobs run once. Refresh schedules, lake maintenance and backups and the live models’ alerts run on the node that holds the jobs’ lease, renewed every 5 seconds for 30. When that node dies, another takes the jobs within about 35 seconds. A refresh holds a lease of its own, so a second one anywhere is refused. A refresh whose node died is settled by what it did (from 0.1.46): a lake load whose version is in its lake counts as done, and is never written twice. Anything else of it is failed and run again once, as a new activity, when it was scheduled or a lake load of it had written already.
• A licence per node. Each node is a machine with a licence activated for each domain. A seat licence is the same key on every node, up to five, and its seats are the cluster’s: the nodes count people from the database they share, take a new seat one at a time under the cluster’s lease, and the node that runs the jobs tells the seller the count (see Seats). A licence from before seats is one per node: a two-node cluster for one domain holds two of them.
• Gateways work with any node: their work is in the shared database, so nothing is pinned to a node, and removing a gateway member takes effect across the cluster within 30 seconds.

The shared database

• Lakes: DuckLake keeps no catalog in SQL Server, so a cluster whose records are there keeps its workspace lakes’ catalogs in a PostgreSQL of their own (see Each workspace’s own lake).
• Size: what is published counts in the database’s size. Mind SQL Server Express’s 10 GB.
• Connections: each node’s server keeps at most 30 connections open to the database, and its OpenBI Accounts 10. The database must take every node’s: PostgreSQL allows 100 by default.

Make a cluster

On each server, the commands run in a Command Prompt as administrator (sudo openbi-server … on a Mac). cluster create and cluster join are dry runs until --apply: they list their steps, done or still to do.

1. The first node

Move the server’s records and OpenBI Accounts’ to the shared database. database move without --apply checks the connection and says what it would copy; with it, it copies every record, checks each table and switches. Stop the services first. Without --connection, each asks for the connection, with its password, so that it stays out of your command history (sqlserver, postgres or mysql):

OpenBI.Server.exe database move --provider sqlserver
OpenBI.Server.exe service stop
OpenBI.Server.exe accounts service stop
OpenBI.Server.exe database move --provider sqlserver --apply
OpenBI.Server.exe accounts database move --provider sqlserver --apply

Then make the cluster, with this server as its first node, and start the services:

OpenBI.Server.exe cluster create --name <cluster>
OpenBI.Server.exe cluster create --name <cluster> --apply
OpenBI.Server.exe service start
OpenBI.Server.exe accounts service start

--apply makes the cluster’s key and shows it once, takes the sign-in’s keys and the data sources’ credentials into the shared database, sealed with it, and stores what is published there. Run again, it finishes what a create stopped halfway did, with the same key.

2. Each other node

Install OpenBI Server on it, with the licence keys for its own domains at hand, then join the cluster by a node’s address (the cluster’s public name, or a node’s own):

OpenBI.Server.exe cluster join https://bi.<domain> --node <name> --apply
OpenBI.Server.exe service restart
OpenBI.Server.exe accounts service restart

It asks for the cluster’s key (typed hidden, piped in, or --cluster-key-file <file>; never an argument), proves to that node that it holds it, and takes the cluster’s settings: the database, OpenBI Accounts and the sign-in. For each domain it has no licence for yet, it asks for a key (or --license-key): a seat licence’s own key, the same as on the first node, or a licence of its own from before seats. --node names the node; the machine’s name by default. The node is ready for people once it has copied what is published.

cluster join doesn’t set up the node’s front door, which stays its own (see What stays each node’s own): on Windows, set up IIS on it for the cluster’s names as on the first node (iis setup --host <name> --apply, see Put IIS in front); a Mac node is its own front door (On a Mac).

3. The load balancer

Any load balancer or DNS failover that checks a health address works, such as two IIS ARR machines, HAProxy, F5, Azure Front Door or Cloudflare. Point its health check at each node’s /api/health/ready every few seconds:

• 200: the node is licensed, reaches the database, has its content current, and isn’t draining.
• 503, with why: send it no one. OpenBI Accounts answers the same at its own /api/health/ready.

Take a node out at its first failed check, and send a request a node refused to connect to another node (HAProxy’s option redispatch). People editing reports in the browser must stay on one node: keep the requests to /w/…/edit, /w/…/d/… and the drafts’ API on the node they began on, with cookie affinity. The command line does not set up load balancers yet.

Look after it

OpenBI.Server.exe cluster status
OpenBI.Server.exe doctor

cluster status shows each node (ready, draining or down, and its content), who runs the jobs and what is refreshing. doctor checks every node too: its state, its version, its clock and the jobs. In the portal, server administrators see the nodes under Admin portal, Cluster: each one’s machine, version and state, which one Holds the jobs, and its last heartbeat. It is read only: the commands below drain and resume a node. See The portal.

• Patching a node: cluster drain <node>. Within a heartbeat its readiness answers 503 and it lets the jobs go; the load balancer sends it no one new, and what it is answering finishes. No one is dropped or signs in again. Then cluster resume <node>.
• A node gone for good: stop its services, then cluster forget <node> takes it out of cluster status.
• Clocks: keep every node’s clock by NTP. Leases take the nodes’ clocks, and doctor says when one is off.
• Sign-in or e-mail changed on one node, such as signin use-accounts, which registers a new secret: run cluster join <that node’s address> --apply again on each other node and restart its services, or people can’t sign in there. The command says so.

Settings

In appsettings.json beside the program, settings.json in the data folder, or the environment’s Cluster__…:

What stays each node’s own

• Its licences, its settings.json (engine limits: machines differ), its front door (IIS and its certificates, or a Mac’s), its logs, its engines and their work folders, and its copy of what is published.
• Drafts of report pages edited in the browser, kept on the node they began on: a node lost loses its unsaved drafts.
• Excel’s failed sign-ins are counted per node, and signing out ends a person’s draft passes on the node they signed out on.

On a Mac

A Mac node is its own front door, on ports 80 and 443. The load balancer sends it the address people asked for as the Host itself, over plain http for now: the node doesn’t trust X-Forwarded-* headers from another machine yet. macOS’s firewall must let OpenBI Server take connections from the network: allow downloaded signed software, or list the server as allowed (System Settings, Network, Firewall).

Upgrade a cluster

Upgrade the nodes one after another, in one window, at an hour no scheduled refresh is due. See Upgrade and rollback.

Frequently asked questions

What if the shared database goes down?

The nodes answer 503 at /api/health/ready while they can’t reach it, so the cluster stops serving. Give the database high availability of its own.

Does each node need its own licence?

Each node activates a licence for each domain it serves. A seat licence is the same key on every node, up to five, with its seats shared; a licence from before seats is one per node. A domain changed with Change domain on one node is finished on each other node with billing domain <new domain> --from <old domain> --apply. A cluster that serves several domains may have a node serve only one of them: cluster join … --domain <domain>, with a licence for that domain alone. The load balancer must then send each domain’s people only to the nodes that serve it.

Last updated 6 November 2026 · OpenBI Server 0.1.49