Module 4: Content Guidelines

Health care research is usually performed with the conscious, albeit long-term, intention of influencing health care policies and practices. In other words, researchers are usually looking for better ways to care for people. However, the results of their research are not likely to be implemented if others cannot understand the research and its implications. Indeed, there are many fine examples of important health discoveries which have not been put into practice. Since policy-makers and practitioners do not willfully do a bad job, we have to conclude that they failed to use research findings because the did not "get the message". In other words, they did not understand the research or they were not convinced by it.

This problem – sometimes referred to as the  "know-do gap" - had been recognized for a long time. Almost from the beginning of biomedical and health publication, critics have been arguing that we need to do a more effective job of communication. 

An important landmark in the move to more effective communication of research results was the establishment of content guidelines for manuscripts, an effort which began in earnest in the late 1970’s. By the early 1990’s two groups have developed independently with the purpose of forming broadly-accepted content guidelines for clinical trials. These two groups were the Asilomar Working Group (named for the meeting place in California) and Standards of Reporting Trials (SORT) Group.

Out of a meeting held in 1993, an initial SORT statement was published with a 32-item checklist and flow diagram which investigators were encouraged to use in reporting RCTs.

Sensibly, the Asilomar and SORT groups met in 1996, in Chicago to see if they could work collaboratively. Merging the recommendations of both groups resulted in the Consolidated Standards of Reporting Trials (CONSORT) Statement, which was published in 1996. As a single policy endorsed by the major medical journals of the time, CONSORT was quickly adopted by many journals as the standard against which manuscripts would be judged.

As we have discussed previously, CONSORT does not impose any limitations on the design or execution of a study – researchers are still free to conduct research as they see fit. What CONSORT does do is to require transparency in reporting. Authors are encouraged to provide enough detail in their writing that readers can judge for themselves whether or not the results are meaningful (and relevant to their own clinical circumstances). Given the intent of CONSORT, it is not surprising that people quickly began to compare manuscripts pre- and post-CONSORT. Indeed, within a few years it did become apparent that manuscripts were becoming more informative. This, in itself, is a good thing. However, it also appears that there was a second benefit – that the quality of research itself was improving. It seems that as authors were becoming obliged to report their studies more completely, they were also becoming more careful about design and execution.

This improvement in reporting and in the research effort itself has not been overwhelming or uniform. As one would understand, it will take time for content guidelines to become known, understood and implemented. Chiropractic literature has been slow to show the benefit of CONSORT, and CONSORT has not become well established in many non-English speaking countries. Nonetheless, the effort has been seen as generally successful and has led to the development of content guidelines for other types of research.

With the involvement of members of the CONSORT group, QUOROM (quality of reports of meta-analyses of randomised controlled trials) was produced in 1999, and MOOSE (meta-analyses of observational studies in epidemiology) in 2000.

In 2001, a revised version of CONSORT was published with an explanation to clarify the criteria for the 22 items on the checklist. 

In late 2001 and early 2002, an independent group published recommendations for a modified CONSORT checklist for use with trials of acupuncture. This set of recommendations was called STRICTA – standards for reporting interventions in controlled trials of acupuncture.

Now the content guidelines were coming fast and furious. 2003 saw the publication of STARD (standards for reporting of diagnostic accuracy) and STROBE (strengthening reports of observational studies in epidemiology). 

In 2004, two extensions to CONSORT were published. These dealt with cluster randomized trials – trials in which groups of participants rather than individuals are randomized – and the reporting of harm during RCTs. These extensions came from and are endorsed by the CONSORT group. 

2004 also saw the publication of the TREND Statement intended to improve the reporting quality of nonrandomized evaluations of behavioral and public health interventions. The TREND group is independent of CONSORT.

In 2005, we saw the first appearance of GNOSIS : guidelines for neuro-oncology : standards for investigational studies – reporting of phase 1 and phase 2 clinical trials. In 2007, an extension was published for reporting surgically based therapeutic clinical trials.

In 2006, there were two more extensions to CONSORT. These dealt with trials for herbal medicines and non-inferiority trials. This year also saw 2 "unofficial" recommendations of CONSORT versions. These were RedHot – reporting data on homeopathic treatments – and recommended revisions for trials of occupational therapy.

The following year produced ORION –outbreak reports and intervention studies of nosocomial infection – billed as a CONSORT equivalent for infection control studies. 2007 also saw the expansion of STROBE to cover 3 common designs in epidemiological studies.

In 2008, there were two more important extensions to CONSORT – one dealt specifically with the contents of abstracts for journal articles and conferences, and the other dealt with non-pharmacological trials.

Clearly the business of producing content guidelines is booming. Furthermore, there seems to be something of an over-representation of TM/CAM genre: acupuncture, herbalism and homeopathy. This may reflect the incongruency between TM/CAM clinical practice and the design of clinical trials in medicine. It also speaks to the concern of the respective professions to promote quality standards in research.

With so many content guidelines, and more appearing all of the time, it may be difficult to keep up! A response to this, arising out of the CONSORT group, is the formation of the EQUATOR network which acts as something of a clearinghouse for content guidelines in health care research.