The Development of the Prognostat Tool for Survival Prediction in Palliative Care Patients

Background: Since a common question arises toward end of life about prognosis and that clinicians vary widely in the accuracy of their survival predictions, it is important that advancements be developed, one of which is to design or improve prognostic tools.

Purpose: To test the reliability of the new Prognostat tool for survival prediction in palliative care patients

Design: Prospective multi-site mixed methods study with data collection, survey and focus groups.

Setting: Prognostat form completed during first assessment by a palliative physician or nurse on admission to a palliative care unit or first ward or home consult.

Results: Four sites were involved with Prognostats completed on 422 patients by 24 palliative clinicians. Of these, the median age was 70 years, 83.2% cancer/16.8% non-cancer, and 29 were censored. Only 5 of 15 factors tested were significant in the multivariable model including clinician prediction of survival, Palliative Performance Scale (PPSv2), primary illness, gender and delirium. Non-significant factors were illness trajectory, age, Charlson Index, tiredness, weight loss, loss of appetite, peripheral edema and skin breakdown. The final model with 5 variables had a Harrell's C of 0.78 demonstrating good predictive discrimination of the model. Focus groups liked the calculator table and nomogram graph in preference to KM-graphs, and the Prognostat was felt to be somewhat valuable, easy and non-invasive tool. Limitations include defining the term 'palliative,' chosen prognostic time intervals and small validation group.

Conclusion: The preliminary Prognostat is shown in this study of patients already registered or referred to palliative care to be a significant predictor of survival and may assist in clinical prognostication. Surprisingly, several factors such as Charlson comorbidities were not significant factors. For practical use, clinicians preferred the table and graph nomograms for quick estimation of survival. Further study is warranted to validate the tool.

Prognostication is a challenging yet important issue for those with progressive advanced and life-limiting illnesses. At some point, the patient or family may ask "Doc, how long do I have?" [1] and our response affects decisions about continuing or stopping treatments, timing of travel of family to visit, location of care and service needs to name a few. Several tools are currently in use which employ various parameters to estimate survival including the Palliative Prognostic Score (PaP) [2-8], Palliative Performance Index (PPI) and Prognosis in Palliative Care Study (PiPS) [9-12].

The Palliative Performance Scale (PPSv2) has also been shown to be a significant predictor of survival for palliative care patients [4,9,11,13-18]. Yet, the clinician’s prediction of survival (CPS) is also important albeit with varying accuracy [19-21]. Many clinicians are able to categorize patients in terms of how sick they are or how long they have to live [22] with some discriminatory abilities but as Justice [22] notes they are not well calibrated [23,24]. To improve precision several laboratory marker indices have been incorporated into some prognostic tools such as low albumin [25], elevated lactate dehydrogenase [25], elevated white cell count combined with lymphocytopenia [26-28], levels of B12, CRP [24,29] and others.

The Palliative Performance Scale (PPS) is known to be a significant predictor of survival for palliative care patients [4,9,11,13-17,30,31]. Although PPS contains both functional assessment and other components of level of consciousness and intake, there are likely also other important factors. The Prognostat was developed to incorporate and test several additional factors reported in the literature.

The Charlson Co-morbidity Index (CCI) is a commonly used tool in chronic disease that includes 19 co-morbidities which have been shown to have prognostic significance value in multiple studies [32-40]. To our knowledge CCI has not yet been incorporated into specific palliative prognostic tools and thus warrants testing. Also, the general trajectory of an illness is an important factor that clinicians use in daily assessment and has shown to have some predictive ability particularly if a sudden functional decline occurs [30]. The Australian 'illness phases' utilizes five aspects of an illness including pre-death categories of stable, unstable, deteriorating, terminal and also bereavement [41,42]. Since palliative care patients face gradual, progressive or sudden decline with the emergence of symptoms leading to death, the inclusion of a trajectory variable into prognostic tools is important to study.

In general, the intent of such tools is not to predict the actual date of death but to involve a probabilistic approach where the 'likelihood' of survival is improved using grouped patient data and prediction graphs and tables. However, Stiel notes that scoring systems on prognosis of survival time seem to be least useful for those patients with neither a good nor a poor prognosis where in those situations both physicians’ estimates as well as the existing prognostic instruments lack precision [43]. In other words, clinicians seem to predict quite well if the patient is very close to death or if they are very stable and doing well; it is the ‘in-between’ that remains challenging. Much attention has recently focused on the ‘surprise question’ stated as "Would you be surprised if your patient died within the next 6 or 12 months?". This appeared somewhat useful in cancer [44], renal [45,46] and ICU [47] patients.

The Prognostat is a new tool combining several factors and designed to be easy, quick and non-invasive. The primary research question is whether the Prognostat is a significant predictor of survival and secondly, do new variables such as CCI and illness trajectory add significant value to survival prediction in palliative patients?

The Prognostat exploratory model combined PPS with clinician prediction of survival(CPS) [4,20,48-50], nineteen co-morbidity factors incorporated into the weighted Charlson Co-morbidity Index (CCI) [35,37,40,51,52], seven symptoms [53-55] and four Australian illness ‘phases’ [42,56,57]. These variables are defined in Table 1 and were selected by an interdisciplinary palliative expert group of 12 physicians and nurses. Further Delphi-type feedback was gained from two workshops [58] where it was also recommended that prognostic tools be quick, easy to use and preferably non-invasive, so laboratory markers were excluded as being minimally invasive. The CPS was categorized into clinically relevant time blocks. The outputs to be explored in the Prognostat included a life expectancy table calculator, nomogram and Kaplan-Meier graphs similar to our prior work [59].

The palliative patient was defined as either being the first admission to a palliative care unit or hospice, or received a palliative consultation at home or hospital. A Prognostat collection form (Supplementary XX) was used by palliative physicians and nurses to obtain data from patients including their survival predictions at the time of first admission or consult. Data was entered into Microsoft (MS) Access, anonymized and extracted for analysis.

Descriptive statistics, Kaplan-Meier survival curves and log-rank statistics were computed. The Cox proportional hazards model was fitted to the survival outcomes incorporating the Prognostat component variables together with the demographic variables, age, gender, location and disease. The statistical significance of Prognostat components for predicting survival were assessed using Wald tests for the corresponding (log)-hazard ratios. A backwards elimination process was used to formulate a final survival prediction model. Survival nomograms were generated using Harrell’s Design library for R [60] which incorporates Cox model analyses to compute weights that are applied to the Prognostat components to determine an overall Prognostat Score. Computations were performed using R2.12.1 and SPSS V17.

Sample size was calculated using Harrell's [60] recommendations of ten events (deaths here) per model parameter. We thus required over 300 deaths to accurately estimate the initial variables with parameters including PPS 9, CPS 5, illness trajectory 4, seven symptoms, CCI 4, age, gender, primary illness and location. Once the total number of cases was met (8 months), recruitment was stopped and four additional months were allowed to follow survival on the remaining cases.

The study participants were also sent a short satisfaction survey which was used along with three focus groups to assess utility of the Prognostat. This qualitative data was thematically analyzed and used along with three separate focus group sessions. Invitations for these focus groups included all study participants in the three city locations. The questions and results c89an be seen in the Supplementary data regarding clinical relevance and utility.

Ethics approval was received by the Health Research Boards of the University’s of Victoria (#J2009-38) and Alberta (#Pro00007680), with some funding from the Canadian Institutes for Health Research (CIHR) ) Partnership in Health System Improvement (PHSI) grant on Timely Access to End-of-Life Care (PHSI#33928-54150) and the Victoria Foundation’s Myre & Winnifred Sim fund.

Four locations participated: two with Palliative Care Units (PCU) only (Nanaimo, Saanich) and two with combined PCU, home and ward consult services (Victoria, Edmonton) but one PCU withdrew after a few cases. There were 25 physician and 20 nurse participants. There were 518 prognostats completed in 422 patients but in those additional situations, assessments were performed by both physician and nurse independently and a few patients had repeat assessments at a later time point or different location of care. The analysis was conducted on only the 422 patient first assessments. Demographic characteristics are shown in Table 2. The average age was 70 years, with the primary disease of cancer 83.6%, non-cancer 16.4%. The time from first Prognostat completion until death (or censoring) averaged 38 days (median 10 days). There were 29 Prognostats censored at study closure.

Table 3 shows the frequencies and percentages for the clinical variables used in the study. In the interest of parsimony, we aggregated levels of PPS into clinically relevant groups: PPS 10-20%, PPS 30-40%, PPS 50% and PPS 60% or higher. CPS categories were collapsed at each end from focus group discussion. A backwards elimination process was used to formulate a survival prediction model. Factors found not to be significant in the multivariable Cox model include: illness trajectory, location of care, dyspnea on exertion, age, Charlson Index, persistent tiredness, weight loss, skin pressure sore, loss of appetite and peripheral edema. Statistically strong factors include CPS, PPS, gender, primary illness and, delirium. P-values for these variables included in the survival prediction model are listed in Table 4. Harrel's C-stat is 0.78 which shows good predictive discrimination of the model.

The final Prognostat nomogram in table calculator format is given in Table 5. Each variable has a weighted value and these are summed to create a total points score. By then locating that score on the percentile table, the likelihood of dying is shown by 10^th, 25^th, median, 75^th and 90^th percentile predictions.

Prognostat predictions were considered accurate if the observed survival time fell between the estimated 25^th and 75^th percentiles. The CPS was considered accurate if actual survival time fell within the CPS interval in the Prognostat that the clinician indicated. From this, 42% of CPS values were accurate compared to 69% accuracy using the Prognostat. The final Prognostat model was validated using external prospective data. There were 63 patients in the validation set for whom descriptive statistics are provided in Table 2 and 3. Since the validation model only collected 'points', we could not distinguish colorectal from lung cancer in Table 2. For the validation set, 69% of CPS values were accurate compared to 84% accuracy using the Prognostat.

A survey and three focus groups were held totalling 15 palliative physicians and 11 palliative nurses to discuss preliminary results, and from which several themes emerged. The tool was felt to be "somewhat or very" easy (79%), quick (83%), "somewhat or very" useful (78%) and practical (62%). Overall, the Prognostat was felt to be a valuable tool which enhances a clinician's prediction of survival. The nomogram and the calculator table were the preferred methods to use the tool, and KM-graphs least preferred. Specific information on the survey and nomgram graph is found in Supplementary data.

The Prognostat improved standalone reports of survival prediction for both clinician prediction (CPS) and PPS scores [31,50,53]. Primary illness, gender and delirium added further significance while surprisingly; the Charlson Index [33] was not significant in our model. Although other studies in cancer and non-cancer have shown the CCI to be a significant factor, an external palliative physician not involved in this study (personal communication) suggested that in far advanced illness these factors are ‘washed out’ by more significant changes in functional status and clinician prediction. Further study is warranted to clarify this.

The cancer categories were selected using the four with the highest mortality in Canada. The 'other' cancers accounted for more cases but varied highly so they were grouped for statistical purposes. Although the 71 non-cancer cases were only 16.8% of the total, that is similar to the number of lung cancer patients. Further, on the nomogram, 'non-cancer' is significant and lies between cancer of the prostrate and other-cancers. This suggests that non-cancer illnesses can be fitted into prognostic tools and that such patients do not always live longer than cancer patients. A larger sample size is needed to test significance for non-cancer illnesses as well as delineate among other primary cancers (eg. Harris et al) [61].

Neither the Charlson CCI nor Australian trajectory phases were significant in the 422 cases. Interestingly however, we initially analysed the 518 cases which included some 'repeat' cases and both parameters were significant. The repeat cases were removed from the final analysis to preserve statistical accuracy of ‘first’ assessment predictions. Further studies using prospective serial assessments are imperative. An earlier model of the Prognostat was less predictive but did not include CPS [62].

An ongoing issue in this field is to adequately define what constitutes the term 'palliative patient.' We based this on direct involvement with "palliative experts" either by referral for consultation or admission to a palliative care unit or hospice. Such referrals usually occur in advanced illness which fits the shorter lengths of stay and survival seen in many palliative prognostic tools [6-11,63-65]. However more recent evolution in this field frequently uses a ‘palliative approach’ which implies a much broader interpretation of the term palliative, makes identification and prognostication more challenging. The 'surprise question' is often utilized about the 'risk' of dying but appears limited on its own as a survival prediction tool [66-70].

Clinician prediction of survival (CPS) remains a valuable aspect in prognostication being the most significant of all variables in this study. Accuracy was substantively improved by 15%-27% in the two datasets using the Prognostat with an overall accuracy of 69-84%. Additionally it provides quantiles of risk which may be clinically informative. Also, the percentile scores were highly appreciated in showing a clinically practical range of survival predictions including both 10% and 90% blocks.

Since the final Prognostat model was reduced from 15 variables to 5, there is need for a larger prospective validation study. Although definitions were provided, the focus groups noted some ambiguity with several symptoms such as tiredness, weight loss and dyspnea.

The prognostic time intervals were also at question. Some focus group participants suggested using simple groupings of days, weeks or months but there was no agreement. The issue is similar to variations seen in other tools such as PaP [6,7] and PiPS [12]. As with most palliative care studies in advanced stage illness, patients had a shorter overall prognosis; we had insufficient cases in those living six months or more category and thus were collapsed into >4 months category. A larger study is needed to gain significant longer-term survivors.

Although our overall Kaplan-Meier mean and median survival times are short at 44 and 12 days respectively, these are similar to several other palliative prognostic tools. For example, the PaP tool is based on 30-day survival [7] and PiPS categories’ median survivals are 5, 33 and 92 days [12]. Therefore the findings may not apply to early stage illness where patients are not admitted to palliative programs or not referred for palliative consultation.

Finally, the patients were already deemed palliative, generally with far advanced illness and either seen in consultation by palliative physicians or admitted to a palliative care unit. Therefore, the findings may or may not apply to patients who are only "at risk of dying," "not palliative" or not seen by palliative expert clinicians.

The Prognostat appears to be a significant predictor of survival for patients already deemed palliative but requires further validation. The study adds support for probabilistic tools using a calculator nomogram with a range of survival percentiles within the individual patient parameters. At least for now the Charlson comorbidities, Australian phases, tiredness, appetite and weight loss are not shown to be the predictors of survival but need to be tested in a larger prospective validation study.

The nomogram table calculator is preferred by most clinicians over KM-graphs. The Prognostat is a simple, non-invasive tool that can assist clinicians in prognostication for palliative patients. Future study should include serial calculations for trajectory analysis and repeat of the Australian phases may demonstrate value in larger cohorts and comparison among professionals. Branco study.

The authors declare there are no competing financial interests.

In addition to the study authors, we express our appreciation to the collaborator physicians and nurses for their time, expertise and support including Physicians: Pablo Amigo, Peter Battershill, David Blair, Debra Braithwaite, Brian Calvin, J. Faily, Manny Fritsch, Susan Germain, Lloyd Hildebrand, James Houston, Brad Hunter, Naseem Janmohamed, Christine Jones, Ryan Liebscher, Robin Love, Doreen Oneschuk, Gail Sager, Maurice Saunders, Ellen Skinnarland, Geoff Spry, Jack Sturmwind and Jim Wilde; and Nurses: Susan Becker, Angela Bruce, Christine Cosack, Catherine Coulter, Caite Meagher, Jill DeGoey, Theresa Downing, Judy Dupuis, Susan Geyssen, Rob Hayward, Carol Johnson, Phin Leow, Ellen MacLeod, Tracy McConnell, Kath Murray, Phung Pam, Lorraine Pardy, Carol Pare, Violet Ramey and Wendy Simmons. Special thanks to Analyst Ju Yang and also to the following Programs: Victoria Hospice Palliative Care Unit and Palliative Response Team, Nanaimo Regional Hospital Palliative Care Unit, Saanich Peninsula Hospital Palliative Care Unit and the Edmonton Zone Palliative Care Program.