SURVIVAL.COX.PATH OpenBI
A penalized Cox model's coefficients along the penalty's path.
Group: Survival analysis. OpenBI’s own: DAX has no function of this name.
Syntax
SURVIVAL.COX.PATH ( <table>, <duration>, <event> [, <options>] [, <entry>], <name>, <covariate> [, <name>, <covariate>] … )
| Parameter | Type | Description |
|---|---|---|
table | Table | The rows to follow, one a subject such as a customer, a policy or a machine: a table of the model or a table expression, as the filters leave it. |
duration | Number evaluated for each row | How long each row was followed: until its event, or until we stopped seeing it. In any unit (days, months, years), the same for every row. |
event | Number evaluated for each row | 1 or TRUE when the row's duration ended in the event (the customer left); 0 or FALSE when it did not, so all we know is that it lasted at least that long (censored). |
options | Text optional | Settings in one text, name=value with commas between: alpha (1 when left out), nlambda (100) and ratio. |
entry | Number optional, evaluated for each row | When each row came under observation: it is at risk from just after its entry to the end of its duration. An expression, not a text, given right before the first name. |
name | Text repeatable | A covariate's name, in quotes: the result calls it so. |
covariate | Expression repeatable, evaluated for each row | The covariate's value on each row: a number, or a text for a factor whose levels are each compared with the first. |
Returns
Table. A row a lambda and term, the largest lambda first. The columns: Lambda, Log lambda, Variable, Coefficient and Deviance ratio.
Remarks
- The fit is the elastic net for Cox's model, as R's glmnet computes it (Efron's handling of ties): alpha=1 is the lasso, which can drop terms altogether; alpha=0 is ridge, which only shrinks them.
- The terms are put on a common scale before the penalty, as glmnet does, and the coefficients come back on the covariates' own scale.
- The path runs over nlambda values of lambda (100 by default, from 5 to 500), from the smallest that keeps every term at 0 down to ratio times it (0.0001, or 0.01 when there are fewer rows than terms).
- Plot Coefficient against Log lambda, a line a Variable: the lasso's picture of which terms come in first as the penalty eases.
- There is no cross-validation here; to choose a lambda, use
SURVIVAL.COX.CV.
Compared with DAX
An OpenBI function: DAX has none like it, so a measure or query that uses it works in OpenBI only, not in Power BI or Analysis Services.
Examples
Five steps of the lasso
At the largest lambda every coefficient is 0; at the smallest they are nearly the plain Cox fit's.
EVALUATE
VAR Customers =
DATATABLE (
"Plan", STRING, "Months", INTEGER, "Left", INTEGER, "Spend", INTEGER, "Tickets", INTEGER,
{
{ "Basic", 3, 1, 20, 4 }, { "Basic", 5, 1, 25, 3 }, { "Basic", 6, 0, 40, 1 }, { "Basic", 8, 1, 30, 5 },
{ "Plus", 9, 1, 35, 2 }, { "Basic", 10, 0, 45, 0 }, { "Basic", 12, 1, 28, 3 }, { "Plus", 14, 0, 60, 1 },
{ "Plus", 15, 1, 38, 4 }, { "Basic", 16, 0, 50, 1 }, { "Plus", 18, 1, 42, 2 }, { "Plus", 20, 0, 55, 0 },
{ "Basic", 22, 1, 33, 2 }, { "Plus", 24, 0, 65, 1 }, { "Plus", 25, 1, 48, 3 }, { "Basic", 27, 0, 36, 0 },
{ "Plus", 30, 1, 52, 1 }, { "Plus", 32, 0, 70, 0 }, { "Basic", 34, 0, 44, 2 }, { "Plus", 36, 0, 58, 1 },
{ "Basic", 4, 1, 22, 2 }, { "Plus", 7, 1, 30, 3 }, { "Basic", 13, 1, 26, 1 }, { "Plus", 28, 0, 62, 2 }
}
)
RETURN
SURVIVAL.COX.PATH ( Customers, [Months], [Left], "nlambda=5", "Plan", [Plan], "Spend", [Spend], "Tickets", [Tickets] )
| Lambda | Log lambda | Variable | Coefficient | Deviance ratio |
|---|---|---|---|---|
| 0.587607236285896 | -0.531696519772035 | Plan: Plus | 0 | 0 |
| 0.587607236285896 | -0.531696519772035 | Spend | 0 | 0 |
| 0.587607236285896 | -0.531696519772035 | Tickets | 0 | 0 |
| 0.0587607236285896 | -2.83428161276608 | Plan: Plus | 1.44260285345084 | 0.571507499768426 |
| 0.0587607236285896 | -2.83428161276608 | Spend | -0.21985282689778 | 0.571507499768426 |
| 0.0587607236285896 | -2.83428161276608 | Tickets | 0.413132345741105 | 0.571507499768426 |
| 0.00587607236285896 | -5.13686670576013 | Plan: Plus | 3.99196805115304 | 0.677222441997696 |
| 0.00587607236285896 | -5.13686670576013 | Spend | -0.515679672710595 | 0.677222441997696 |
| 0.00587607236285896 | -5.13686670576013 | Tickets | 0.746627278631462 | 0.677222441997696 |
| 0.000587607236285896 | -7.43945179875417 | Plan: Plus | 4.67645505696084 | 0.680812505120616 |
| 0.000587607236285896 | -7.43945179875417 | Spend | -0.600164383645542 | 0.680812505120616 |
| 0.000587607236285896 | -7.43945179875417 | Tickets | 0.851388731480517 | 0.680812505120616 |
| 5.87607236285896e-05 | -9.74203689174822 | Plan: Plus | 4.76274928876033 | 0.680860017238365 |
| 5.87607236285896e-05 | -9.74203689174822 | Spend | -0.610812663918499 | 0.680860017238365 |
| 5.87607236285896e-05 | -9.74203689174822 | Tickets | 0.864545444112366 | 0.680860017238365 |
The elastic net
With alpha=0.5, half lasso and half ridge.
EVALUATE
VAR Customers =
DATATABLE (
"Plan", STRING, "Months", INTEGER, "Left", INTEGER, "Spend", INTEGER, "Tickets", INTEGER,
{
{ "Basic", 3, 1, 20, 4 }, { "Basic", 5, 1, 25, 3 }, { "Basic", 6, 0, 40, 1 }, { "Basic", 8, 1, 30, 5 },
{ "Plus", 9, 1, 35, 2 }, { "Basic", 10, 0, 45, 0 }, { "Basic", 12, 1, 28, 3 }, { "Plus", 14, 0, 60, 1 },
{ "Plus", 15, 1, 38, 4 }, { "Basic", 16, 0, 50, 1 }, { "Plus", 18, 1, 42, 2 }, { "Plus", 20, 0, 55, 0 },
{ "Basic", 22, 1, 33, 2 }, { "Plus", 24, 0, 65, 1 }, { "Plus", 25, 1, 48, 3 }, { "Basic", 27, 0, 36, 0 },
{ "Plus", 30, 1, 52, 1 }, { "Plus", 32, 0, 70, 0 }, { "Basic", 34, 0, 44, 2 }, { "Plus", 36, 0, 58, 1 },
{ "Basic", 4, 1, 22, 2 }, { "Plus", 7, 1, 30, 3 }, { "Basic", 13, 1, 26, 1 }, { "Plus", 28, 0, 62, 2 }
}
)
RETURN
SURVIVAL.COX.PATH ( Customers, [Months], [Left], "alpha=0.5, nlambda=5", "Plan", [Plan], "Spend", [Spend], "Tickets", [Tickets] )
| Lambda | Log lambda | Variable | Coefficient | Deviance ratio |
|---|---|---|---|---|
| 1.17521447257179 | 0.16145066078791 | Plan: Plus | 0 | 0 |
| 1.17521447257179 | 0.16145066078791 | Spend | 0 | 0 |
| 1.17521447257179 | 0.16145066078791 | Tickets | 0 | 0 |
| 0.117521447257179 | -2.14113443220614 | Plan: Plus | 0.345816246267727 | 0.438162185739789 |
| 0.117521447257179 | -2.14113443220614 | Spend | -0.106350475780543 | 0.438162185739789 |
| 0.117521447257179 | -2.14113443220614 | Tickets | 0.397053618504519 | 0.438162185739789 |
| 0.0117521447257179 | -4.44371952520018 | Plan: Plus | 2.53007080741934 | 0.63943350935419 |
| 0.0117521447257179 | -4.44371952520018 | Spend | -0.334252864727361 | 0.63943350935419 |
| 0.0117521447257179 | -4.44371952520018 | Tickets | 0.56582539368134 | 0.63943350935419 |
| 0.00117521447257179 | -6.74630461819423 | Plan: Plus | 4.1967264789389 | 0.678943242694114 |
| 0.00117521447257179 | -6.74630461819423 | Spend | -0.540152339761071 | 0.678943242694114 |
| 0.00117521447257179 | -6.74630461819423 | Tickets | 0.784476346634892 | 0.678943242694114 |
| 0.000117521447257179 | -9.04888971118827 | Plan: Plus | 4.69886901230135 | 0.680832179609879 |
| 0.000117521447257179 | -9.04888971118827 | Spend | -0.602843724502354 | 0.680832179609879 |
| 0.000117521447257179 | -9.04888971118827 | Tickets | 0.855574987788656 | 0.680832179609879 |