This funnel plot shows the change from baseline to the terminal assessment for experimental groups and control groups as a standardized effect size, Hedges' g.
This Hedge's g is the terminal mean minus the baseline mean divided by the pooled standard deviation. Thus, a Hedges' g = 1 means that a group improved by 1 standard deviation from the begining
of an intervention to the end. These effect-sizes are plotted against the standard error of the effect size on the y-axis. This axis is inverted so that "up" indicates greater precision.
Plotting the data this way shows the bias in effect-sizes across studies. Small, imprecise studies are most likely to produce large effects, whereas larger trials with more precision generally
produce more modest (but still postive!) effect-sizes.
You can also interact with this figure by toggling between plotting all outcomes (i.e., multiple outcomes per study) or the main outcome of each study (i.e., one outcome per study, which was the
"primary outcome" if specified in the original study). You can click and drag anywhere on the plot to zoom in on a specific area. You can also click on a datapoint to get more information
about that trial! (Note that if an interaction takes you someplace you don't want to go, you can click "Reset Visualization" at the bottom right or refresh the page.
In the funnel plot above, we can see tremendous variation in the effect sizes of different RCTs.
This variation in effect-sizes results from a multitude of factors. For instanace, differences in the study population
(e.g., age of patients, stroke etiology, other medical conditions), in the intervention (e.g., frequency, duration,
intensity, method of delivery), in the control group (e.g., How was the control condition matched to the intervention?),
and the outcomes (e.g., Was there sufficient blinding of assessors? Was it lower extremity or upper extremity? A measure
of function, activity, or participation?). Because variation in these parameters may explain variation in treatment
outcomes, it is very important to quantitatively document how interventions are being applied.
In the visualizations below, we present some variables related to the sample population (e.g., days from stroke to the
start of the intervention, average age) and to the nature of the intervention (e.g., invervention duration and estimated
time scheduled for therapy). Note that these visualizes are all "tied" to each other, so you can click on "ctrl" in the
first panel, and all of the other panels will update to show only Control groups.
A single RCT might have multiple experimental and control groups. There 489 total groups in the SCOAR database, 285 identified as experimental and 204 identified as control. Click on one of the bars in the plot above to view data specifically for experimental or control groups. To return to plotting all data, click "Reset Visualization".
We know that timing is a critical factor in stroke rehabilitation, but when are most studies conducted? Here, we have groups of studies a occuring within 90 days following stroke (the "acute" phase), less than one year, or more than one year following stroke (the "chronic" phase). Click on a point above to see only studies from that time window.
A major concern with RCTs is their external validity. That is, how well do the obtained results generalize to people outside of the study? Given that effects obtained in younger participants may be different from effects obtained in older participants, it is important to see how many studies are conducted in each at range and if age moderates treatment effects in some way.
The duration of an intervention is a critical component of therapy to consider and this is likely to change from region to to region based on differences in health care policies and the availability of care. Thus, it is important to see the distribution of therapy durations to look for consistencies, but also gaps, across trials. Critically, it is also important to explore how the duration and dose of therapy (next panel) might change as a function of other variables like time-post stroke or age.
The importance of the dose of therapy and the shape of potential dose-response curves is a critical topic in rehabilitation. This visualization allows you to see what the dose of therapy has historically been across trials. As outlined in our review paper the most consistently reported metric for the dose of therapy is the time scheduled for therapy. Here we present three different estimates of time scheduled for therapy (max, min, and the 50% compromise between the two). Please refer to the SCOAR review paper or the data-dictionary below for more information about these calculations.
In the bubble chart below, you can plot time scheduled for therapy (as either time_min, time_50, or time_max), against the terminal Hedges' g on the y axis. You can also choose to plot all outcomes from all studies, or plot only the main outcomes (one per group). The color of the bubbles shows seven different outcome types, and you can filter the results based on outcome type by clicking on the labels to the right. (Note that an explanation of these labels is provided in the SCOAR review paper.) You can click and drag on the image to zoom, and clicking on a data point will give you additional information about that group/study.
By toggling through the different outcome types, you can see how these data show different potential dose response relationships as a function of what is being measured. However, numerous other factors are going to affect the dose-response relationship, such as the timing of the intervention, the type of therapy received, and the age of the patients. Other variables are going to affect the dose-response curve, but timing, group, and age are readily available in SCOAR. You can filter the bubble chart by interacting with the other plots below. For instance, to see on control groups, click on the "ctrl" bar. For filtering based on the age of the patients, you can drag and select an interval of ages on the age chart. The filters can always be removed by clicking on "Reset Visualization". For more information for how to use the filters, please see the "Video" section.
Data:
The SCOAR database includes many categorical and numerical values that were extracted from over two hundred randomized controlled trials in stroke therapy. The coding/calculation of some of the variables is quite detailed and we would directed interested readers to the SCOAR review paper. The basic variables and the most essential variables that were used in our visualizations are briefly explained below. The full database (as of 2016-03-31), the data dictionary, a full reference list of all trials, and the Creative Commons license for SCOAR are available from Github.
Author: The last name/surname of the first author. Entered all lowercase. In the case of follow-up studies being combined with an original publication the format will be “surname1/surname2”.
Year: A four digit number indicating the year of publication. In the case of follow-up studies being combined with an original publication the format will be “year1/year2”.
Time Max, Time 50, Time Min: Given the issues with different types of therapy, it is not always clear how time was spent in therapy. In CIMT, for instance, participants might spend 5 hours per day under constraint for 5 days per week, for 4 weeks (100 total hours of constraint). To resolve this issue we have created three different calculations of total time:
Time_MAX: a total time calculation where 100% of constraint time is counted as time in therapy.
Time_50: a total time calculation where 50% of constraint time is counted as time in therapy (we consider this calculation to be the most plausible as some, but not all, constraint time is counted).
Time_MIN: a total time calculation where 0% of constraint time is counted as time in therapy.
Days ps: Patients' chronicity. That is, the average time, in days, from the patient's stroke to the beginning of the intervention.
Age base: Average age, in years, of the experimental/control group at the baseline assessment.
Term g: The terminal Hedges' g (i.e., terminal Cohen's d multiplied by the correction factor). Subtraction was arranged so that positive values always reflect improvement from baseline.
Fu g: The follow-up Hedges' g (i.e., follow-up Cohen's d multiplied by the correction factor). Subtraction was arranged so that positive values always reflect improvement from baseline.
Term vg: Variance of the terminal Hedges' g.
Standard error: square root of Term vg.
Group:A categorical variable indicating whether the data come from an experimental group, "exp", or a control group "ctrl" as described in the original study.
Group id: Numeric value identifying an independent group of participants. Note that multiple groups of participants will come from the same study and that a single group might be measured on several outcomes in the database (i.e., there may be multiple outcomes per group).
Base n: Given the description in the text, this is the number of participants whose data contribute to the baseline mean (base_m) and baseline standard deviation (base_sd) calculations. Note that this is not necessarily the number of participants randomized to each group, depending on how the authors conducted their analysis.
Outcome Name: The name of the outcome measure being recorded on that row. At the moment (2016-01-31) we have one outcome measure per study, but as the data base grows, we will have more. Common abbreviations include: