How are the trials rated ?

Each trial is rated using a scale depending on the type of study design used. Randomised Controlled Trials (including pseudo randomized controlled trials), non-Randomised Controlled Trials and Case Series are rated using the PEDro Scale. At present, the PsycBITE™ scale for Single Subject Design is under development. Systematic Reviews are not rated for quality at present.

To ensure consistency with rating, two independent reviewers rate the trials that are included in PsycBITE™. These ratings are conducted by members of the PsycBITE™ team, as well as volunteer raters from any allied-health or medical discipline. All raters have undergone training and assessment to qualify as a rater for PsycBITE™. Our volunteer raters are considered members of the PsycBITE™ Raters Network. If you would like to become a member of the PsycBITE™ Raters Network, please send us your contact details.

It is important to note that each rating is based on information that has been explicitly stated in the text of the article. For example, if on a literal reading of the trial report it is possible that a particular criterion is not satisfied, then it cannot receive a point for this criterion. Therefore, it is possible that a low rating may reflect the absence of detail rather than the quality of the study.


Back to Top

The PEDro Scale

All group studies on PsycBITE™ (RCTs, non-RCTs and Case Series) are rated using the PEDro Scale, with some clarification to criterion definitions making them unambiguously related to trials on PsycBITE™.
The PEDro scale consists of 11 criteria. The first item relates to the external validity (specifically the participant selection criteria). The remaining 10 items (criteria 2 - 10) assess the internal validity of each trial and whether the trial contains sufficient statistical information to make it interpretable. Thus, the internal validity of each trial is ranked based on a total score out of 10 (i.e., excluding criterion 1).
The PEDro scale is based on the Delphi list developed by Verhagen and colleagues. The Delphi list is a list of trial characteristics that was thought to be related to trial "quality" by a group of clinical trial experts (for details see Verhagen et al, Journal of Clinical Epidemiology 51: 1235-41, 1998). The PEDro scale contains additional items on adequacy of follow-up and between-group statistical comparisons. One item on the Delphi list (the item on eligibility criteria) is related to external validity, so it does not reflect the dimensions of quality assessed by the PEDro scale. This item is not used to calculate the method score that is displayed in the search results (which is why the 11 item scale gives a score out of 10). This item has, nevertheless, been retained so that all Delphi list items are represented on the PEDro scale.
The reliability of the PEDro scale for rating quality of randomised controlled trials has been published in Physical Therapy 83(8):713-21, 2003 August. The abstract can be viewed here.



Back to Top

Randomised Controlled Trials

Randomised Controlled Trials (RCT) are rated using the PEDro scale. A maximum of 10 points on the 11-item scale can be obtained, (criterion 1 is related to external validity and is not counted).
An RCT compares at least two treatments/interventions (one of which can be a no-treatment control or a wait-list control condition), which involves random allocation (participants are randomly allocated to groups for either the intervention/treatment being studied or control/placebo using a random mechanism, such as coin toss, random number table, or computer-generated random numbers) and the outcomes are compared. Pseudo or intended-to-be-randomised trials are also included in this category where participants are allocated to groups for intervention/treatment or control/placebo using a non-random method (such as alternate allocation or by odd or even hospital numbers).

Back to Top

Non-Randomised Controlled Trials

Non-Randomised Controlled Trials (non-RCT) are rated using the PEDro scale. A maximum of 8/10 points can be obtained for internal validity on the 11-item scale. Criterion 1 is related to external validity and is not counted, criteria 2 and 3 (random and concealed allocation) are not applicable to these types of trials and therefore cannot receive points for these items.

A non-RCT is similar to an RCT in that it compares at least two treatments/interventions (one of which can be a no-treatment control or a wait-list control condition) with the exception that participants have not been randomly allocated to groups. Participants with the outcome or disease are selected and information is obtained about the previous exposure to the treatment/intervention or other factor being studied.


Back to Top

Case Series

Case Series are rated using the PEDro scale. A maximum of 2/10 points can be obtained for internal validity on the 11-item scale (criteria 8 and 11). Criterion 1 is related to external validity and is not counted, criteria 2, 3,4,5,6,7,9 and 10 are not applicable to these types of trials and therefore cannot receive points for these items.


Case Series refers to a group/s of participants who are exposed to one treatment/intervention where outcomes are measured in participants before and after exposure to the treatment / intervention.


Back to Top

Single Subject Design

The PsycBITE™ scale for rating the methodological quality of Single Subject Design is currently under development
Back to Top

How do the ratings from one research design compare with another? - Designation of Levels of Evidence

The different research designs on PsycBITE™ span the gamut of levels of evidence as described by the National Health and Medical Research Council, (see table). It is noted that "a methodologically poor randomised trial (level II) may, however, provide a weaker basis for a recommendation than a high-quality observational study without randomisation, which can provide level III-2 evidence" (NHMRC, 1999, p.15). This means that an RCT (level II) with a score of 3 may be less methodologically sound than a non-RCT with a score of 6. For Further information on Levels of Evidence see the NHMRC website.


Back to Top

How to become a PsycBITE™ rater

Thank you for your interest in becoming a rater for PsycBITE™. Our training packages are currently under development and will be available over the next few months. Please send us your contact details so that we can notify you when the training packages are completed.

	RCT, non RCT and Case Series Training Kit (disabled for now)
	RCT, non RCT and Case Series Accuracy Test (disabled for now)
	Single Subject Design Training Kit (disabled for now)
	Single Subject Design Accuracy Test (disabled for now)

Back to Top

PsycBITE™ - ©2004