Skip to main content

Table 3 Suggestions for questions and comments on QUIPS risk of bias domains and our team’s proposed answers

From: Elaborating on the assessment of the risk of bias in prognostic studies in pain rehabilitation using QUIPS—aspects of interrater agreement

RoB domain and corresponding “prompting items”

Suggestions of questions and comments where consensus is needed.

Our team’s answers to the questions from a pain rehabilitation perspective, which will produce a “yes”

1. Study participation

For a “yes”, …

 a) Adequate participation in the study by eligible persons.

a) What is adequate?

a) … the participation should be at least 67%. Our experience is that in this field of those being assessed for participation not all are eligible and therefore we permitted a somewhat lower participation rate compared to what can be used in other fields [12].

 b) Description of the source population or population of interest

b) When is the lack of such a description related to bias?

b) This prompting item was not taken into account, since only a few studies reported information of the source population.

 c) Description of the baseline study sample

c) What is important to know about the study sample?

c) … basic information available regarding gender, age, socioeconomic status together with some disease-related information (pain, disability, comorbidities) and information on relevant outcome data .

 d) Adequate description of the sampling frame and recruitment

d) What is the minimal information on the sampling frame and recruitment procedure and what is an “incorrect sampling frame”?

d) … information available on the patients’ recruitment (from which health service) together with a description of how the data collection was performed.

 e) Adequate description of the period and place of recruitment

e) What is the minimal information needed?

e) … there should be information available on the beginning and end of the data collection, the setting, and the name of a geographical place or hospital. Additionally, it could be more important to know if the patient was filling in questionnaires with/without influence of the caregivers.

 f) Adequate description of inclusion and exclusion criteria

f) What is the minimal information needed?

f) … at least 1 inclusion and 1 exclusion criterion should be given.

2. Study attrition

For a “yes”, …

 a) Adequate response rate for study participants

a) What is adequate?

a) … the response rate should be at least 67%. The experience from this field is that due to the relative long follow-up time (6mo) and differences in follow- up time (12, 18mo), a relative larger loss of participants at follow-up compared to other fields could be expected.

 b) Description of attempts to collect information on participants who dropped out

b) What is the minimal information needed?

b) … information available on the methods and timing.

 c) Reasons for loss to follow-up are provided

c) What is the minimum information needed?

c) … any information available on the reasons for drop-outs

 d) Adequate description of participants lost to follow-up

d) What is adequate? Do we need to see the analyses or is it enough with one sentence?

d) … any information available on gender, age and disease-related information for drop-outs.

 e) There are no important differences between participants who completed the study and those who did not

e) What is important? Demographic differences (hampering generalization) or illness-related differences (hampering validity?)

e) … there should be no differences between the participants and non-participants in regard to demographic and illness-related variables, such as levels of pain intensity, disability, absence from work.

3. Prognostic factor (PF) measurement

For a “yes”, …

 a) A clear definition or description of the PF is provided

a) What is the minimal information needed?

a) … there should be a clear definition of the PF, e.g. information on which question(s) were used, how the data was collected, how the variable was constructed, etc.

 b) Method of PF measurement is adequately valid and reliable

b) What is adequate? When is an instrument valid/reliable?

b) … there should be a reference to a reliability/validity study or information on these features in the paper and this paper should cover the field of chronic pain rehabilitation. In the field of rehabilitation of chronic pain we suggest that when different prognostic factors are included with different RoB, this issue should be noted and solved in the synthesis phase of the SR/MA (e.g. making decisions of excluding those invalid instruments, or downgrading the level of evidence).

 c) Continuous variables are reported or appropriate cut points are used

c) What is appropriate?

c) … the cut-offs used should NOT be based on distribution of the data, but on established cut-offs in the field of chronic pain rehabilitation.

 d) The method and setting of measurement of PF is the same for all study participants

d) Is it OK if not the same, but two reliable measurements are used?

d) …the PF should be the same, but also could be different for different study participants if both measures are reliable (e.g. VAS or NRS when measuring pain). However, both instruments should be valid for the use in the field of chronic pain.

 e) Adequate proportion of the study sample has complete data for the PF

e) What is adequate?

e) … there should be at least 67% available with complete data. It is important also to check if there is different data available for different prognostic factors measured simultaneously, which could indicate differential loss-to follow-up.

 f) Appropriate methods of imputation are used for missing PF data

f) What is appropriate?

f) … there should be some kind of imputation, but even if no imputation was done, it could be a “yes” if at least 67% of the study sample had complete data.

4. Outcome measurement

For a “yes”, …

 a) A clear definition of the outcome is provided

a) What is the least information needed?

a) … there should be a clear definition of the outcome measure available, e.g. information on the question(s) used, how the data was collected, how the variable was constructed, etc. In the field of rehabilitation of chronic pain we suggest that the IMMPACT recommendations on outcome measures should be used [13].

 b) Method of outcome measurement is adequately valid and reliable

b) What is adequate? When is an instrument valid/reliable?

b) … there should be a reference to a reliability/validity study or information on these features in the paper. Note the population on which the reliability/validity study was performed should correspond to the population of interest, in this case patients with chronic pain.

 c) The method and setting of outcome measurement is the same for all study participants

c) Is it OK if not the same, but two reliable measurements are used?

c) … the outcome measures should be the same, but could also be different for different study participants if both measures are reliable (e.g. VAS or NRS when measuring pain). However, both instruments should be valid for the use in the field of chronic pain.

5. Study confounding

For a “yes”, …

 a) All important confounders are measured

a) Which confounders are of importance?

a) … there should be at least one confounder taken into account. However, in a broad review in the field of chronic pain with multifactor associations between prognostic factors and outcome it is difficult to know if one should use the results of the multivariate analyses in the MA or if the univariate results should be used. On one hand, the univariate analyses could be confounded by other factors, hence a multivariate analysis should be the appropriate choice. However, in a multivariate analysis, the PF of interest could have been removed or the results could have been altered through interaction of other factors, leading to a lack of data or overestimation of the PF of interest in a MA. In our current study on prognostic factors for functioning in patients with chronic pain we included both uni- and multivariate estimates in the MA, but indicated this in the forest plots and used sensitivity analyses for re-analysing the results.

 b) Clear definitions of the important confounders measured are provided

b) What is the minimal information needed?

b) … there should be a clear definition of the confounder measure available, e.g. information on the question(s) used, how the data was collected, how the variable was constructed, etc.

 c) Measurement of all important confounders is adequately valid and reliable

c) What is adequate? When is an instrument valid/reliable?

c) … there could be a reference to a reliability/validity study or information on these features in the paper and the confounder should be valid for the use in the field of chronic pain.

 d) The method and setting of confounding measurement are the same for all study participants

d) Is it OK if not the same, but two reliable measurements are used?

d) … the confounder measures should be the same, but could also be different for different study participants if both measures are reliable (e.g. VAS or NRS when measuring pain).

 e) Appropriate methods are used if imputation is used for missing confounder data

e) What do we accept as an appropriate method when handling missing data?

e) … there could be any kind of imputation, but even if no imputation was done, it could be a “yes” if at least 67% of the study sample had complete data.

 f) Important potential confounders are accounted for in the study design

f) What do we accept as an appropriate study design?

f) … there should have been some kind of randomization or confounders included in the analyses. Since blinding of researchers and patients to treatment is nearly impossible in the field of chronic pain rehabilitation, we accepted randomization only.

 g) Important potential confounders are accounted for in the analysis

g) If not, do we judge here or on point 6c) or on both points?

g) see point f) and point 6c)

6. Statistical Analysis and Reporting

For a “yes”, …

 a) Sufficient presentation of data to assess the adequacy of the analytic strategy

a) What is sufficient?

a) … there should be enough information available to understand the statistical methods applied, so that the rater can determine whether the methods used were correct.

 b) Strategy for model building is appropriate and is based on a conceptual framework or model

b) What is appropriate? If there is no model described, is there a risk of bias?

b) We decided that this point was NOT to be considered, but more emphasis was put on point 6a)

 c) The selected statistical model is adequate for the design of the study

c) What is adequate?

c) … there should be some form of statistical analyses description available, resulting in information on the effect of the PF on the outcome.

 d) There is no selective reporting of results

d) Is there a need to check pre-publications such as protocols for this information?

d) … all variables (outcomes and PF) that are described in the method section should be included in the result section with words or in numbers (tables, figures).

  1. MA meta-analysis, VAS visual analogue scale, NRS numeric rating scale