OpenBI Docs Buy

Notebooks, the kernel and the catalog

The types the studio and the agent's tools use to read, write and run notebooks: the notebook in Jupyter's format, its flow, the kinds of notebooks, the kernel's requests and results, the analytics catalog and the examples. A step seldom needs them.

NotebookDocument

public sealed class NotebookDocument

A notebook in Jupyter's format (.ipynb, nbformat 4.5), as VS Code's Polyglot Notebooks and Jupyter read it. The project keeps it without outputs (so git shows code changes only); the outputs of the last run are kept beside it, out of git.

Constructors

Its constructors: NotebookDocument().

Properties

Its properties: string? Name { get; set; }, List<NotebookCell> Cells { get; }.

Methods

void Save(string path, string? outputsPath = null)
Saves the notebook's code to path (for git), and with its outputs to outputsPath when given.
static NotebookDocument Open(string path, string? outputsPath)
The notebook at path, with the outputs kept at outputsPath for steps whose code has not changed since.

Also: JsonObject ToJson(bool outputs), string ToText(bool outputs), static NotebookDocument Parse(string json), static NotebookDocument Load(string path).

NotebookCell

public sealed class NotebookCell

A step of a notebook: text (markdown) or code in C#, OAX or SQL, with what it showed when it last ran.

Constructors

Its constructors: NotebookCell().

Properties

string Language { get; set; }
csharp, dax, sql or markdown.
string? Name { get; set; }
The variable an OAX or SQL step makes.
JsonObject? Parameters { get; set; }
An OAX step's parameters and their values (@asOf: "2025-06-30").
string? EditedBy { get; set; }
Who last changed it: "agent" or a person's name.
double? Milliseconds { get; set; }
How long its last run took, and when it ran (kept with the outputs).
bool Failed { get; set; }
Its last run failed.
List<JsonObject> Outputs { get; }
Jupyter outputs: display_data, stream, error.

Also: string Id { get; set; }, string Source { get; set; }, int? ExecutionCount { get; set; }, DateTimeOffset? RanAt { get; set; }, bool IsCode { get; }.

Methods

Its methods: static string NewId(), JsonObject ToJson(bool outputs), static NotebookCell FromJson(JsonObject o).

NotebookSteps

public static class NotebookSteps

A notebook's steps named as an agent or a person names them: by id (what notebook_step_add and notebook_read give), or by name (the variable an OAX step makes, a C# step's title in the flow) - "backtest", "subgroups". One that names no step, or several, is an error that lists the steps, never a silent guess: a step added "after" a name nobody has went first in the notebook, and a run "from" or "to" such a name ran nothing and said nothing (the customer log's B-07 and B-08).

Methods

static NotebookCell Find(NotebookDocument doc, string notebook, string step)
The step named by id, else by its name (case aside), else by its title (the first line of a step without a name).
static List<NotebookCell> Range(NotebookDocument doc, string notebook, string? from, string? to)
The code steps from one to another, in order (both named by id or name; from the first, or to the last, when not given). A range that holds no code (from after to, or markdown only) is an error saying so.
static int InsertAt(NotebookDocument doc, string notebook, string? after)
Where a step added "after" another goes: its index (the end when not given).
static string Title(NotebookCell c)
A step's title: its name, else its first line (40 characters).
static string List(NotebookDocument doc)
The steps as "1. id (name, language)", for an error to list.

NotebookTemplates

public static class NotebookTemplates

The first steps of a new notebook.

Methods

static NotebookDocument Blank(string name)
A clean slate: no steps at all.
static NotebookDocument Blank(string name, WorkKind kind)
A clean slate of a kind: a prediction's has no steps; the others say, in one line, how their work is saved.
static NotebookDocument Network(string name, string table, string from, string to, string about)

A network's start: the question, a step measuring the network from a table of edges (shown as a graph), one listing who matters most, and how to keep it as a version.

  • name: The network's name (the notebook's, and its versions').
  • table: The table of edges (OAX: 'Sales', or FILTER('Sales', …)).
  • from: Each row's first end (OAX on the table's rows).
  • to: Each row's other end.
  • about: What an edge is, in words.
static NotebookDocument Optimization(string name)
An operations research model's start: the question, an OAX step for the rows to decide for (with the numbers the model needs), and a step that builds the model from them and solves it.
static NotebookDocument Quantitative(string name)
A quantitative notebook's start: the question, OAX steps for the rates and the positions, the market frozen as of a date, the valuation, its risk, and how to keep it as a version.
static NotebookDocument Start(string name)
A short start: the question, an OAX step with a date parameter, and C# steps to learn from it and save a version.

FlowGraph

public sealed class FlowGraph

A notebook as a flow: its steps are nodes, and an edge goes from the step that makes a variable to each later step that uses it (read from the code, not run). A step that saves a model version (Models.Save("name", …), SaveForecast, Import) leads to a node for that model.

Properties

Its properties: IReadOnlyList<FlowNode> Nodes { get; }, IReadOnlyList<FlowEdge> Edges { get; }.

Methods

IReadOnlyList<string> Downstream(string id)
The steps that use what this one makes, directly or through others.
IReadOnlyList<string> Upstream(string id)
The steps this one needs, directly or through others.
Dictionary<string, int> Layers()
Each node's layer: 0 for those that use nothing, else one below the deepest it uses.

Also: FlowNode? Node(string id), static FlowGraph Analyze(IReadOnlyList<NotebookCell> cells).

FlowNode

public sealed record FlowNode(
    string Id,
    string Title,
    string Kind,
    int Order,
    IReadOnlyList<string> Defines,
    IReadOnlyList<string> Uses)

A node of a notebook's flow: a step (or a model version a step saves), what it makes and what it uses.

Properties

Its properties: string Id { get; init; }, string Title { get; init; }, string Kind { get; init; }, int Order { get; init; }, IReadOnlyList<string> Defines { get; init; }, IReadOnlyList<string> Uses { get; init; }.

FlowEdge

public sealed record FlowEdge(
    string From,
    string To,
    IReadOnlyList<string> Variables)

An edge: the variables one step makes and another uses.

Properties

Its properties: string From { get; init; }, string To { get; init; }, IReadOnlyList<string> Variables { get; init; }.

Workbench

public static class Workbench

The four kinds of notebooks and where each one's work is in a project.

Fields

Its fields: static readonly WorkKind Prediction, static readonly WorkKind OperationsResearch, static readonly WorkKind Quantitative, static readonly WorkKind Network.

Properties

Its properties: static IReadOnlyList<WorkKind> All { get; }.

Methods

static WorkKind Kind(string id)
A kind by its id, its folder or its title (operations research is also "optimization", "or").
static WorkKind? KindOf(string project, string name)
The kind whose folder holds this name's work in the project, or null (none yet).
static string FolderOf(string project, string name)
Where a name's work is in the project: the kind's folder that has it; a name not used yet is a prediction's (models were saved under predictions/ before the other kinds).
static IReadOnlyList<string> Names(string project, WorkKind kind)
The names of a kind's work in the project (its folders holding a notebook), in order.

WorkKind

public sealed record WorkKind(
    string Id,
    string Folder,
    string Title,
    string One,
    string Engine)

A kind of work a notebook does, in its own folder of the project: a prediction (a model that scores), operations research (a plan: what to do given what limits it), a valuation (positions valued, risk) or a network (its nodes measured).

Properties

string Id { get; init; }
prediction, operations-research, quantitative or network.
string Folder { get; init; }
Its folder in the project (predictions, operations-research, quantitative, networks).
string Title { get; init; }
The studio's tab.
string One { get; init; }
One of them, in words ("an operations research model").
string Engine { get; init; }
What does the work, in words.
bool SavesModels { get; }
A prediction saves a model (Models.Save); the others save what they found (Results.Save).

AnalyticsCatalog

public static class AnalyticsCatalog

OpenBI's analytics catalog: by goal, what to use of LightGBM, OR-Tools, Open Graph and OpenBI's own statistics, with a recipe that runs. The agent's analytics_catalog tool reads it; notebook_create(recipe: id) starts a notebook from one.

Fields

static readonly IReadOnlyList<CatalogEntry> Entries
Every entry, by topic.

Also: static readonly string[] Topics.

Methods

static CatalogEntry Get(string id)
An entry by its id.
static IReadOnlyList<CatalogEntry> Find(string goal, string? topic = null, int top = 3)
The entries that answer a goal best (words of the goal against each entry's goal, its other wordings, API and id), best first.
static string Reference(string? api = null)
The notebook library's public API for a type or member ("LightGbm", "Uplift.TwoModel", "Decisions"): each member's signature and what its documentation says, each parameter's too (a parameter's default and what it means - Models.Save's scoreAt, TODAY() unless given - are where an author reads them, not only in the agent's instructions). Without one, the types by namespace with a line each.

CatalogEntry

public sealed record CatalogEntry(
    string Id,
    string Topic,
    string Goal,
    string[] Also,
    string[] Api,
    string[] Dax,
    string[] Caveats,
    CatalogStep[] Steps,
    string? Needs = null)

What OpenBI has for one goal of serious analytics work: the notebook API and the OAX to use, a recipe of notebook steps that runs (on made-up data with a known truth unless it says what model it needs), and what to watch out for.

Properties

string Id { get; init; }
Its name for analytics_catalog(id) and notebook_create(recipe).
string Topic { get; init; }
prediction, calibration and monitoring, experiments and causal, optimization, networks, survival, statistics.
string Goal { get; init; }
The goal in a sentence, as one would ask for it.
string[] Also { get; init; }
Other ways of saying it (what the search matches too).
string[] Api { get; init; }
The notebook calls (OpenBI.Notebooks, OR-Tools' own classes).
string[] Dax { get; init; }
The OAX functions for the same in the model and on report pages.
string[] Caveats { get; init; }
What makes the result wrong when ignored.
CatalogStep[] Steps { get; init; }
The recipe.
string? Needs { get; init; }
What it needs open (a model's tables), when it does not run on its own.
bool SelfContained { get; }
Runs as it is: on made-up data, without a model.

Methods

NotebookDocument ToNotebook(string name)
The recipe as a notebook: a heading with the goal and caveats, then its steps.

Also: JsonObject ToJson(bool full = true).

CatalogStep

public sealed record CatalogStep(
    string Language,
    string Name,
    string Code)

A step of a recipe: its language (csharp, dax, markdown), its name and its code.

Properties

Its properties: string Language { get; init; }, string Name { get; init; }, string Code { get; init; }.

Examples

public static class Examples

The examples of the analytics tabs, on the example project's tables (examples/Example Brewery): a production plan and delivery routes (Operations Research), a bond portfolio (Quantitative), who buys what (Networks). Each is a question, the tables it reads, a brief an agent builds the notebook from, and a notebook that answers it, step by step.

Fields

Its fields: static readonly IReadOnlyList<Example> All.

Methods

Its methods: static Example Get(string id).

Example

public sealed record Example(
    string Id,
    WorkKind Kind,
    string Name,
    string Question,
    string[] Tables,
    string Brief,
    Func<NotebookDocument> Notebook)

An example for newcomers: its question, the model's tables it needs, a brief for the agent, its notebook.

Properties

Its properties: string Id { get; init; }, WorkKind Kind { get; init; }, string Name { get; init; }, string Question { get; init; }, string[] Tables { get; init; }, string Brief { get; init; }, Func<NotebookDocument> Notebook { get; init; }.

ModelConnection

public sealed class ModelConnection : IDisposable

The open model, through the engine OpenBI Studio runs for the project: OAX queries, and DuckDB SQL over the project's sources and the model. The studio gives the kernel the engine's address and key (OPENBI_ENGINE_URL, OPENBI_ENGINE_KEY).

Constructors

Its constructors: ModelConnection(Uri engine, string key).

Properties

Its properties: Uri Address { get; }.

Methods

static ModelConnection? FromEnvironment()
The engine the studio gave this process, or null outside a studio's kernel.
Table Dax(string query, CancellationToken ct = default)
Runs an OAX query (EVALUATE …) on the model and returns its first table. A notebook is where a developer tests, so PREDICT("name", n) on a version nobody approved yet scores here (report pages and OpenBI Server refuse it), and the step says the rows come from a draft.
Table Sql(string sql, CancellationToken ct = default)
Runs DuckDB SQL over the project's sources and the model; every row.
(byte[] Pdf, byte[]? Png) TikZ(string tex, CancellationToken ct = default)
A LaTeX document (a TikZ picture) compiled offline by the engine's Tectonic: its PDF, and its picture (PNG, 192 ppi).

Also: void Dispose().

KernelSession

public sealed class KernelSession : INotebookHost, IDisposable

Runs a notebook's steps in order, keeping their variables: C# scripting, OAX and SQL on the open model (bound to a variable of the step's name). The kernel process holds one session; a step that crashes the process takes only the kernel down, never the studio.

Constructors

KernelSession(Action<JsonObject> send, ModelConnection? model = null, string? projectFolder = null, string? notebookName = null, string? user = null)
  • send: Events while a step runs (display, status, stdout), as JSON.
  • model: The open model (OAX and SQL steps), or null.
  • projectFolder: The project's folder (model versions are saved there), or null.
  • notebookName: The notebook's name.
  • user: Who runs it (a version's author).

Properties

bool ByAgent { get; }
The step running now was sent by the agent.

Also: string? ProjectFolder { get; }, string? NotebookName { get; }, string User { get; }, CancellationToken Interrupted { get; }, bool Running { get; }.

Methods

static Microsoft.CodeAnalysis.Scripting.ScriptOptions Options()
How steps compile: the OpenBI library and .NET's, the latest C#, with their lines in errors. The namespaces a .NET project has without a using (System.IO, System.Net.Http, System.Threading, System.Threading.Tasks) and a few that data steps use (Stopwatch, Regex, JSON) need none either.
static IReadOnlyList<string> CompileErrors(NotebookDocument doc)
Compiles a notebook's C# steps without running them (OAX and SQL steps' variables declared as they would be): the errors, as "step title, line n: message". For notebooks written for someone.
void Interrupt()
Stops the running step (its loops and OpenBI's calls check; a step stuck elsewhere is stopped by restarting the kernel).
Task<StepResult> RunAsync(StepRequest step)
Runs one step.
IReadOnlyList<VariableInfo> Variables()
The variables steps have made, newest definition of each name.
JsonObject RunRecord()
The steps run so far, in order, with what they showed: the notebook as it ran (Jupyter's format).

Also: void Display(DisplayData data), void Status(string text), void Dispose().

KernelServer

public static class KernelServer

The kernel process (OpenBI.Studio.exe --kernel): requests come as JSON lines on standard input ({"op": "run", "id", "language", "code", "name", "parameters"}, {"op": "interrupt"}, {"op": "variables"}, {"op": "record"}, {"op": "shutdown"}); events and results go out as JSON lines on standard output. Steps run one at a time, in the order they come; an interrupt is read while one runs.

Methods

Its methods: static int Run(), static int Run(TextReader input, Stream output).

KernelClient

public sealed class KernelClient : IAsyncDisposable

The studio's side of a notebook's kernel: starts it (a process of its own), sends it steps, and raises what they show as they run. An interrupt asks the step to stop; a step that does not stop within the grace time is ended by restarting the kernel (its variables are then lost, and KernelClient.Restarted says so).

Constructors

Its constructors: KernelClient(Func<KernelConnection> connect).

Properties

Its properties: bool IsStarted { get; }, string? RunningStep { get; }.

Events

event Action<JsonObject>? Event
What running steps show (display, status, stdout events, each with the step's id).
event Action<string>? Restarted
The kernel was restarted (why): its variables are gone.

Methods

static KernelClient ForProcess(string exe, IEnumerable<string> args, IReadOnlyDictionary<string, string> environment)
A kernel in a process: the studio itself with --kernel, given the engine, project and notebook.
Task<StepResult> RunAsync(StepRequest step, CancellationToken ct = default)
Runs a step (one at a time: a second waits for the first).
Task<bool> InterruptAsync(TimeSpan grace)
Asks the running step to stop. True when it stopped within grace; false when the kernel had to be restarted (its variables are gone).
Task RestartAsync(string why = "The kernel was restarted.")
Ends the kernel and starts a new one: every variable is gone.
Task<JsonObject> RecordAsync(CancellationToken ct = default)
The steps run so far with their outputs (the notebook as it ran).

Also: Task StartAsync(CancellationToken ct = default), Task<IReadOnlyList<VariableInfo>> VariablesAsync(CancellationToken ct = default), ValueTask DisposeAsync().

KernelConnection

public sealed record KernelConnection(
    Stream Requests,
    Stream Events,
    Action Kill,
    Func<string> Errors)

A running kernel's two streams (we write requests, we read events) and how to stop it.

Properties

Its properties: Stream Requests { get; init; }, Stream Events { get; init; }, Action Kill { get; init; }, Func<string> Errors { get; init; }.

KernelStoppedException

public sealed class KernelStoppedException : Exception

The kernel stopped (crashed, or was restarted) while a request waited.

Constructors

KernelStoppedException(string message)
The kernel stopped (crashed, or was restarted) while a request waited.

StepRequest

public sealed record StepRequest(
    string Id,
    string Language,
    string Code,
    string? Name = null,
    JsonObject? Parameters = null,
    bool ByAgent = false)

A step to run: its language, code, and for OAX and SQL the variable it makes and its parameters.

Properties

Its properties: string Id { get; init; }, string Language { get; init; }, string Code { get; init; }, string? Name { get; init; }, JsonObject? Parameters { get; init; }, bool ByAgent { get; init; }.

StepResult

public sealed record StepResult(
    string Id,
    bool Ok,
    IReadOnlyList<JsonObject> Outputs,
    StepError? Error,
    double Milliseconds,
    int ExecutionCount,
    IReadOnlyList<VariableInfo> Variables)

What a step did: its outputs, its error (if any), how long it took, and the variables after it.

Properties

Its properties: string Id { get; init; }, bool Ok { get; init; }, IReadOnlyList<JsonObject> Outputs { get; init; }, StepError? Error { get; init; }, double Milliseconds { get; init; }, int ExecutionCount { get; init; }, IReadOnlyList<VariableInfo> Variables { get; init; }.

StepError

public sealed record StepError(
    string Message,
    int? Line = null,
    int? Column = null,
    string? Detail = null)

A step's error: the message, and where in the step (1-based) when known.

Properties

Its properties: string Message { get; init; }, int? Line { get; init; }, int? Column { get; init; }, string? Detail { get; init; }.

VariableInfo

public sealed record VariableInfo(
    string Name,
    string Type,
    string Summary)

A variable the kernel holds after a step: its name, type and a line about its value.

Properties

Its properties: string Name { get; init; }, string Type { get; init; }, string Summary { get; init; }.

Last updated · OpenBI Studio 0.3.78