SURVIVAL.COX.CV OpenBI
A penalized Cox model's cross-validation: the deviance at each penalty, and the one chosen.
Group: Survival analysis. OpenBI’s own: DAX has no function of this name.
Syntax
SURVIVAL.COX.CV ( <table>, <duration>, <event> [, <options>] [, <entry>], <name>, <covariate> [, <name>, <covariate>] … )
| Parameter | Type | Description |
|---|---|---|
table | Table | The rows to follow, one a subject such as a customer, a policy or a machine: a table of the model or a table expression, as the filters leave it. |
duration | Number evaluated for each row | How long each row was followed: until its event, or until we stopped seeing it. In any unit (days, months, years), the same for every row. |
event | Number evaluated for each row | 1 or TRUE when the row's duration ended in the event (the customer left); 0 or FALSE when it did not, so all we know is that it lasted at least that long (censored). |
options | Text optional | Settings in one text, name=value with commas between: alpha (1 when left out), folds (10), seed (1), nlambda (100) and ratio. |
entry | Number optional, evaluated for each row | When each row came under observation: it is at risk from just after its entry to the end of its duration. An expression, not a text, given right before the first name. |
name | Text repeatable | A covariate's name, in quotes: the result calls it so. |
covariate | Expression repeatable, evaluated for each row | The covariate's value on each row: a number, or a text for a factor whose levels are each compared with the first. |
Returns
Table. A row a lambda, the largest first. The columns: Lambda and Log lambda; Deviance, its Std error, Lower and Upper; Nonzero; Deviance ratio; and Chosen.
Remarks
- The fit is the elastic net for Cox's model, as R's glmnet computes it (Efron's handling of ties): alpha=1 is the lasso, which can drop terms altogether; alpha=0 is ridge, which only shrinks them.
- The terms are put on a common scale before the penalty, as glmnet does, and the coefficients come back on the covariates' own scale.
- The path runs over nlambda values of lambda (100 by default, from 5 to 500), from the smallest that keeps every term at 0 down to ratio times it (0.0001, or 0.01 when there are fewer rows than terms).
- Lambda is chosen by cross-validation: the rows are split into folds (10 by default, from 3 to 50; at least twice as many rows as folds), shuffled with the seed (1 by default; seed=0 keeps the rows' order), and each fold is scored by its deviance. Another seed can choose another lambda.
- Deviance is the cross-validated deviance per row at that lambda (lower is better), Lower and Upper one standard error either side: the curve to choose lambda on.
- Nonzero counts the terms still in the model. Deviance ratio is the share of the deviance the fit on all the rows explains.
- Chosen marks the lambda with the least deviance (min) and the largest within one standard error of it (1se), the two
SURVIVAL.COX.PENALIZEDcan take.
Compared with DAX
An OpenBI function: DAX has none like it, so a measure or query that uses it works in OpenBI only, not in Power BI or Analysis Services.
Examples
Ten lambdas, four folds
EVALUATE
VAR Customers =
DATATABLE (
"Plan", STRING, "Months", INTEGER, "Left", INTEGER, "Spend", INTEGER, "Tickets", INTEGER,
{
{ "Basic", 3, 1, 20, 4 }, { "Basic", 5, 1, 25, 3 }, { "Basic", 6, 0, 40, 1 }, { "Basic", 8, 1, 30, 5 },
{ "Plus", 9, 1, 35, 2 }, { "Basic", 10, 0, 45, 0 }, { "Basic", 12, 1, 28, 3 }, { "Plus", 14, 0, 60, 1 },
{ "Plus", 15, 1, 38, 4 }, { "Basic", 16, 0, 50, 1 }, { "Plus", 18, 1, 42, 2 }, { "Plus", 20, 0, 55, 0 },
{ "Basic", 22, 1, 33, 2 }, { "Plus", 24, 0, 65, 1 }, { "Plus", 25, 1, 48, 3 }, { "Basic", 27, 0, 36, 0 },
{ "Plus", 30, 1, 52, 1 }, { "Plus", 32, 0, 70, 0 }, { "Basic", 34, 0, 44, 2 }, { "Plus", 36, 0, 58, 1 },
{ "Basic", 4, 1, 22, 2 }, { "Plus", 7, 1, 30, 3 }, { "Basic", 13, 1, 26, 1 }, { "Plus", 28, 0, 62, 2 }
}
)
RETURN
SELECTCOLUMNS (
SURVIVAL.COX.CV ( Customers, [Months], [Left], "folds=4, nlambda=10", "Plan", [Plan], "Spend", [Spend], "Tickets", [Tickets] ),
"Lambda", [Lambda],
"Nonzero", [Nonzero],
"Deviance", [Deviance],
"Lower", [Lower],
"Upper", [Upper],
"Chosen", [Chosen]
)
| Lambda | Nonzero | Deviance | Lower | Upper | Chosen |
|---|---|---|---|---|---|
| 0.587607236285896 | 0 | 3.69980580459434 | 3.29805292735439 | 4.10155868183428 | (blank) |
| 0.211175091471473 | 2 | 2.77097359456228 | 2.5083988117379 | 3.03354837738666 | (blank) |
| 0.0758923929185371 | 3 | 2.27696812724918 | 2.11191372845161 | 2.44202252604675 | (blank) |
| 0.0272743118649468 | 3 | 1.85203863051635 | 1.65755346589623 | 2.04652379513647 | 1se |
| 0.00980187946511145 | 3 | 1.83747058120452 | 1.64239450632105 | 2.03254665608799 | min |
| 0.00352261283526835 | 3 | 2.11840951075657 | 1.76524565133098 | 2.47157337018215 | (blank) |
| 0.0012659614139681 | 3 | 3.00436369919286 | 1.83894181354295 | 4.16978558484277 | (blank) |
| 0.000454962942736797 | 3 | 4.88043283981263 | 1.86778478316533 | 7.89308089645993 | (blank) |
| 0.000163505204013226 | 3 | 6.97484052524515 | 1.8783452285005 | 12.0713358219898 | (blank) |
| 5.87607236285896e-05 | 3 | 9.16808782599264 | 1.88271840466881 | 16.4534572473165 | (blank) |