Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial open data platform and infrastructure that supports research on pragmatic trials. It shares clean trial data and ratings using PRECIS-2 permitting multiple and varied meta-epidemiological studies that compare treatment effects estimates across trials that employ different levels of pragmatism as well as other design features.
Background
Pragmatic trials are increasingly acknowledged as providing evidence from the real world for clinical decision making. The term "pragmatic" however, is not used in a consistent manner and its definition and assessment need further clarification. The purpose of pragmatic trials is to guide clinical practice and policy decisions, rather than confirm a physiological or clinical hypothesis. A pragmatic trial should try to be as similar to the real-world clinical environment as is possible, including the recruitment of participants, setting up and design, the delivery and implementation of the intervention, determination and analysis of outcomes and primary analysis. This is a major difference from explanatory trials (as described by Schwartz and Lellouch1) which are designed to provide more complete confirmation of the hypothesis.
Truely pragmatic trials should not be blind participants or the clinicians. This could lead to bias in the estimations of treatment effects. Practical trials should also aim to enroll patients from a wide range of health care settings to ensure that their findings are generalizable to the real world.
Additionally studies that are pragmatic should focus on outcomes that are important to patients, such as quality of life or functional recovery. This is particularly relevant in trials that involve the use of invasive procedures or potential serious adverse events. The CRASH trial29, for example, focused on functional outcomes to compare a two-page report with an electronic system for the monitoring of hospitalized patients with chronic heart failure, and the catheter trial28 focused on symptomatic catheter-associated urinary tract infections as the primary outcome.
In addition to these aspects pragmatic trials should also reduce trial procedures and data-collection requirements to cut costs and time commitments. Finally, pragmatic trials should seek to make their findings as relevant to actual clinical practice as they can by making sure that their primary analysis follows the intention-to treat approach (as described in CONSORT extensions for pragmatic trials).
Many RCTs which do not meet the criteria for pragmatism but have features that are contrary to pragmatism, have been published in journals of different types and incorrectly labeled pragmatic. This could lead to false claims about pragmatism, and the usage of the term should be standardised. The development of the PRECIS-2 tool, which offers an objective and standard assessment of pragmatic characteristics is a great first step.
Methods
In a pragmatic study it is the intention to inform clinical or policy decisions by showing how an intervention can be integrated into routine care in real-world situations. Explanatory trials test hypotheses regarding the cause-effect relation within idealized environments. In this way, pragmatic trials could have a lower internal validity than explanation studies and are more susceptible to biases in their design analysis, conduct, and design. Despite these limitations, pragmatic trials can contribute valuable information to decisions in the context of healthcare.
프라그마틱 플레이 -2 tool evaluates the degree of pragmatism within an RCT by scoring it across 9 domains, ranging from 1 (very explicative) to 5 (very pragmatic). In this study, the recruit-ment, organization, flexibility in delivery, flexible adherence and follow-up domains scored high scores, but the primary outcome and the method for missing data were below the practical limit. This suggests that a trial can be designed with well-thought-out pragmatic features, without harming the quality of the trial.
It is difficult to determine the degree of pragmatism within a specific trial since pragmatism doesn't have a binary characteristic. Certain aspects of a study may be more pragmatic than other. A trial's pragmatism could be affected by changes to the protocol or logistics during the trial. Koppenaal and colleagues discovered that 36% of the 89 pragmatic studies were placebo-controlled, or conducted prior to licensing. They also found that the majority were single-center. They aren't in line with the standard practice and can only be considered pragmatic if their sponsors accept that the trials aren't blinded.
A typical feature of pragmatic studies is that researchers attempt to make their findings more relevant by studying subgroups within the trial. However, this often leads to unbalanced results and lower statistical power, which increases the risk of either not detecting or incorrectly detecting differences in the primary outcome. This was a problem in the meta-analysis of pragmatic trials due to the fact that secondary outcomes were not corrected for differences in covariates at the baseline.
Furthermore practical trials can be a challenge in the gathering and interpretation of safety data. It is because adverse events are typically self-reported, and therefore are prone to errors, delays or coding variations. Therefore, it is crucial to improve the quality of outcome assessment in these trials, in particular by using national registries instead of relying on participants to report adverse events in a trial's own database.
Results
While the definition of pragmatism does not require that all clinical trials be 100% pragmatic, there are benefits of including pragmatic elements in trials. These include:
By incorporating routine patients, the results of trials can be more quickly translated into clinical practice. However, pragmatic studies can also have drawbacks. The right type of heterogeneity, like could allow a study to generalise its findings to many different settings or patients. However the wrong type of heterogeneity could decrease the sensitivity of the test, and therefore decrease the ability of a study to detect even minor effects of treatment.
Numerous studies have attempted to classify pragmatic trials using various definitions and scoring systems. Schwartz and Lellouch1 have developed an approach to distinguish between explanatory trials that confirm a clinical or physiological hypothesis, and pragmatic trials that help in the selection of appropriate therapies in real-world clinical practice. The framework was comprised of nine domains that were scored on a scale ranging from 1 to 5 with 1 being more informative and 5 indicating more practical. The domains included recruitment of intervention, setting up, delivery of intervention, flexible adherence and primary analysis.
The original PRECIS tool3 was built on the same scale and domains. Koppenaal et al10 created an adaptation to this assessment, dubbed the Pragmascope which was more user-friendly to use in systematic reviews. They found that pragmatic systematic reviews had a higher average scores in the majority of domains, with lower scores in the primary analysis domain.
The difference in the main analysis domain could be explained by the fact that most pragmatic trials analyse their data in an intention to treat manner while some explanation trials do not. The overall score was lower for systematic reviews that were pragmatic when the domains on organisation, flexible delivery, and follow-up were merged.
It is important to note that the term "pragmatic trial" does not necessarily mean a low-quality trial, and there is an increasing rate of clinical trials (as defined by MEDLINE search, however it is neither sensitive nor specific) that employ the term "pragmatic" in their abstracts or titles. The use of these terms in abstracts and titles could indicate a greater understanding of the importance of pragmatism, but it isn't clear if this is reflected in the content of the articles.
Conclusions
As appreciation for the value of real-world evidence grows widespread the pragmatic trial has gained traction in research. They are clinical trials randomized that evaluate real-world alternatives to care rather than experimental treatments under development. They involve patients which are more closely resembling the ones who are treated in routine care, they use comparators which exist in routine practice (e.g. existing drugs), and they rely on participant self-report of outcomes. This approach can overcome the limitations of observational research like the biases that come with the reliance on volunteers and the limited availability and the coding differences in national registry.
Other advantages of pragmatic trials are the ability to utilize existing data sources, and a greater chance of detecting meaningful changes than traditional trials. However, they may be prone to limitations that compromise their validity and generalizability. For instance, participation rates in some trials may be lower than expected due to the healthy-volunteer effect and financial incentives or competition for participants from other research studies (e.g., industry trials). Many pragmatic trials are also restricted by the need to recruit participants quickly. Additionally some pragmatic trials lack controls to ensure that the observed differences aren't due to biases in the conduct of trials.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs that self-described themselves as pragmatist and published up to 2022. They assessed pragmatism by using the PRECIS-2 tool, which includes the eligibility criteria for domains as well as recruitment, flexibility in adherence to intervention and follow-up. They discovered that 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or more) in at least one of these domains.
Trials with a high pragmatism rating tend to have broader eligibility criteria than traditional RCTs which have very specific criteria that are not likely to be present in clinical practice, and they comprise patients from a wide range of hospitals. The authors argue that these traits can make pragmatic trials more meaningful and useful for daily practice, but they don't necessarily mean that a trial conducted in a pragmatic manner is free of bias. The pragmatism principle is not a fixed attribute and a test that does not have all the characteristics of an explanatory study could still yield valid and useful outcomes.