While the official definitions of these terms given by various regulatory agencies are clear and concise to an experienced researcher, the language used can be a little off putting.

The European Medicines Agency defines these trials as:

Superiority Trial
This is a trial with the primary objective of showing that the response to the investigational product is superior to a comparative agent (active or placebo control).

Equivalence Trial
This is a trial with the primary objective of showing that the response to two or more treatments differs by an amount which is clinically unimportant. This is usually demonstrated by showing that the true treatment difference is likely to lie between a lower and an upper equivalence margin of clinically acceptable differences.

Non-Inferiority Trial
This is a trial with the primary objective of showing that the response to the investigational product is not clinically inferior to a comparative agent (active or placebo control).

I think it is perhaps better to understand these types of trials with an example. So, to put these definitions into context; clinical trials most often involve comparing a new treatment with an existing treatment or a new drug against a placebo or existing drug. The goal of these trials is to show that the new drug or treatment is efficacious. Efficacy in this context can mean a number of things depending on the test hypothesis. Take for example a trial where we are comparing a new drug to a control; depending on our test hypothesis, efficacy could be showing that the new drug is at least as good as the control, or it could be proving that the new drug is better than the control or it could even be to confirm that the new drug is equivalent to the control. Such hypotheses are widely referred to in publications and texts as non-inferiority trials, superiority trials and equivalence trials respectively.

Continuing with this example: if we let ∆ be the difference between the new drug and the control, and assuming that the values to the right of zero correspond to a better response with the new drug so that the values to the left indicate that the control is better. Also, letting represent the largest difference that is clinically acceptable so values greater than this value would be significant in practice. Then, if we were to declare that the two drugs were equivalent then a two-sided confidence interval (CI) on the difference between the two drugs should lie within the interval to . Similarly, if we were to declare that the new drug was non-inferior to the control then a one sided CI of the difference should be greater than , or if we were to declare that the new drug was superior to the control then the CI of the difference should be greater than zero.

We can see from the above that it is possible to start out with a non-inferiority trial but end up showing superiority. Conversely it is also possible that you start out with a superiority trial but end up showing non-inferiority. However, it must be noted that there are potential implications in making such post-hoc changes which I won’t be going into here but if you are interested I strongly recommend the July 2000 EMA report titled Points to Consider on Switching between Superiority and Non-Inferiority. It is also important to note, as has been inferred from the above, that a non-inferiority trial is essentially a one sided equivalence trial or conversely an equivalence trial can be described as two one sided tests (TOST).

Which hypothesis you use depends entirely on the question your trial is addressing. Hopefully when you now go to research your trial you will be able to refine your search to the trials that best suit your hypothesis!

A study that has a sample size which is too small may produce inconclusive results and could also be considered unethical, because exposing human subjects or lab animals to the possible risks associated with research is only justifiable if there is a realistic chance that the study will yield useful information. Similarly, a study that has a sample size which is too large will waste scarce resources and could expose more participants than necessary to any related risk. Thus an appropriate determination of the sample size used in a study is a crucial step in the design of a study.

More recent studies analysing the power of published papers has shown that, even still, there are large numbers of papers being published that have insufficient power. With the availability of software such as nQuery and nTerim which can calculate appropriate sample sizes for any given power such issues should not be arising so often today.

The Prevention and Treatment of Missing Data in Clinical Trials Blog
November 1st – 2nd 2011, Iselin, New Jersey
by Andrew Grannell

This was a short course, organised be Daniel Scharfstein and held in New Jersey, aimed at statisticians in pharmaceutical industry to inform and discuss the new National Academy of Science on Statistics’ report on how to prevent and treat missing data in clinical trials. Several members of the panel attended the short course to discuss the new recommendations. Also in attendance were two representatives from the Food and Drug Administration, Robert O’Neill (Senior Statistical Advisor to the CDER, FDA) and Thomas Permutt (Director at the Division of Biometrics II, CDER, FDA).

On the first morning, the course started off with Robert O’Neill giving some background information on how the panel was formed and what its task was. The panel from the National Academy of Science on Statistics were tasked with developing some recommendations for the FDA on how to approach the prevention and treatment of missing data in clinical trials. They were asked to prepare “a report with recommendations that would be useful for FDA’s development of guidance for clinical trials on appropriate study designs and follow-up methods to reduce missing data and appropriate statistical methods to address missing data for analysis of results.”

The next speaker was the chairman of the panel, Roderick Little (Associate Director for Research and Methodology, and Chief Scientist, U.S. Census Bureau). He presented a very good and clear overview on the design, conduct and analysis issues that currently exist. Following the break, Thomas Permutt gave great insight into his opinion on where the FDA’s position is on the prevention and treatment of missing data. He felt they were moving in the right direction towards reviewing new methods on handling missing data analysis. Daniel Scharfstein then presented an overview of some of the practical case studies they were going to present, which helped put this course into perspective on a very practical level.

After lunch, both Jay Seigal (Chief Biotechnology Officer and the Head of Pharmaceutical Global Regulatory Affairs, Johnson & Johnson) and James Neaton (Professor of Biostatistics, School of Public Health, University of Minnesota) gave very interesting talks on the prevention of missing data. Jay spoke about the design aspects that could help reduce missing data and James spoke about methods in how to efficiently approach the data management and site personnel side of collecting data for clinical trials.

For Day 2, Daniel had organised a much more intense day of presentations, involving the heavy theory behind these very impressive methods of handling missing data analysis. In that respect, the second day was very appropriately kicked off with a presentation by Rod Little on defining missingness, missing data patterns, inverse probability weighting (IPW), likelihood methods, multiple imputation and sensitivity analysis.

Daniel Scharfstein gave a very interesting and comprehensive presentation on the methodology of sensitivity analysis and gave great weighting to its importance. His presentation outlined that sensitivity adds a certain degree of verification to results obtained. The first case study of the course was given by Joe Hogan (Professor and Director of Graduate Studies, Department of Biostatistics, Brown University). He outlined a very detailed and comprehensive approach to the analysis of missing data in this case. It helped put the material covered during the first day into perspective when approaching a real clinical trial. The second case study was presented by Daniel Scharfstein. Again, with this case study a very detailed and comprehensive approach was outlined and explained very well.

The last presentation of the short course was given by Rod Little. He outlined any other considerations such as clarification of specific characteristics encountered in clinical trials and collecting data. There were also a string of very interesting Q&A sessions after each presentation. There were very open and engaging discussions held between the panel and the attendees on various topics encountered in the pharmaceutical industry today. To sum up, it was a very successful course and gave great insight into the future of prevention and treatment of missing data in the pharmaceutical industry.

Free Webinar

Date: Tuesday 6th December
Time: 10AM – 11 AM GMT
Duration: 60 minutes
Speaker: Andrew Grannell, Senior Statistician, Statistical Solutions Ltd.

Click Here to Register

Demand is expected to be high. Please register early.

Key Webinar Learnings:

• Learn about unique and innovative methods of approaching missing data analysis and how to deal with a biased response
• Explore new ways a graphically representing missing data, both pre- and post- analysis
• Gain insight into how multiple imputation can improve and enhance your overall analysis
• Find out how sensitivity analysis can strengthen your understanding of your results

After a calm flight from Cork, we arrived on a very pleasant and sunny day in Bristol. On the way to Wills Hall, the location of the conference, we drove through the city and passed through the vast, outreaching Durdham Downs. It was an amazing open area of grassed lands and forest. The area which we were staying in was inspiring and really set the tone for the conference.

Being the first ASC conference we have attended, there was an air of anticipation about the talks being given and the discussions to be had. The theme for this years’ conference was “Shifting the Boundaries of Research”. The first keynote speaker was Tom Ewing (Kandar Operations), started off proceedings in true marketeering fashion. He gave a great and lively presentation appropriately named, “Twilight of the Respondents”, on the story of respondents and how they are transforming as well as the way market researchers interact with them. He discussed various influences over respondents and researchers’ approaches to the information collect from respondents. He also highlight a major issue of, can market researchers actually trust respondents to assess themselves and their impulses correctly. He raised a very valid issue, that now, due to the influences of social media and “big data”, there are a lot more resources out there to assess respondents on.

Andrew Grannell at ASC Bristol

The second keynote speaker (which rounded off the day’s events), Jeffrey Henning (Affinova), gave a very impressive presentation on Crowd-Shaped Surveys. I felt this talk followed on nicely from the mornings talk by Tom Ewing. This presentation highlighted how nowadays, market researchers are employing methods that allow surveys to adapt based on the responses obtained to date. Therefore potentially, asking more important questions on the “real” topics as opposed to the perceived ones. Not only will the questions be formulated based on the previous responses but also the options to choose from. Thus quickly converging on the relevant topics and optimising both responses gathered as well as total respondents.

Day 1 also highlighted very interesting developments in mobile applications and the developments of language processes on the web. David Birks (Winchester School of Art) and his colleague, Deborah Wilson (University of Winchester) presented their multimedia application called Talking Walls, designed to recreate a time slice of any heritage or historical building. The various potential application areas were discussed as well as employing the KubeMatrix format aimed towards the market research online community. Maggie Little (Language Connect) presented their examination if the overlap between market research and language services. She highlighted the degree to which various languages are used on the internet across the globe. Maggie gave a very interesting insight into how market researchers approach the need for translation when conducting surveys.

By the time Thursday afternoon rolled around, many great talks had been given and it was my turn to step up to the plate. It was a very appropriate setting in the chapel of Wills Hall, where I stood by the altar to preach the word of Multiple Imputation to those who had the stamina (or patience) to listen. I gave a brief introduction to missing data/nonresponse and examples of the many ways in which it can come about. I went on to outline traditional methods in dealing with nonresponse before introducing the concept of imputation. There was a great discussion afterwards on applying imputation to both survey design (matrix sampling) and dealing with nonresponse in market research.

Eoghan Murphy at ASC Bristol

The second day started off just as the first, with a very impressive and charismatic presentation from Glen Watson (Office for National Statistics). He gave a great presentation of what was involved in organising and carrying out the England & Wales 2011 Census. He described how they went about implementing their first online census and what was involved with that aspect. He outlined the goals they had aimed to achieve and even mentioned how they incorporated imputation to account for nonresponse!!

The keynote presentation was swiftly followed by a very interesting talk on behavioural economics from Ken Parker (Discovery Research). He outlined a very important concept of what is going through the consumers mind when they decide to part with their money. He also highlighted the issues facing market researchers when trying to figure out that decision process, quoting John Wanamaker, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half”.

Unfortunately we missed the final presentations of the conference as we had to make the flight home. Overall we were really pleased with the conference. The presentations we were able to attend were eye-opening as to how the market research world is evolving. Also discussions we had with various attendees, from all sorts of backgrounds, at the booth were very interesting and exciting, when one considers the potential of what’s capable in the near future. Til’ next time…..

Andrew Grannell

Kevin & Andrew at JSM Miami

At JSM, our U.S.A. Manager, Kevin Connolly and our Cork-based Customer Support Manager, Andrew Grannell staffed our booth. Right from the very beginning, the Statistical Solutions booth was extremely popular with the delegates. We would like to think that this popularity was mainly due to the demos of SOLAS that Andrew gave, but in truth it was more likely that the free Statistical Solutions FRISBEES, drove all the traffic to our booth. These great Miami Beach give-away’s were so much in demand that we had completely run out by Tuesday afternoon!! But any late-comers were not disappointed as we still had SOLAS Flashlights and pens to give-away.

On the more serious side of the exhibition booth, the new pre- and post-imputation graphics in SOLAS 4.0 drew great interest from delegates, with many opting to sign up for our free 30 day trial. nQuery Advisor was a popular as ever and the new nTerim utility certainly added to the interest. Many visitors to our booth were delighted to see that BMDP is still going strong and everyone was delighted to hear that we have great plans for its future. (More about this in a few weeks time).

No sooner had we packed up our exhibition booth in Miami, we had to turn our attention to the next big statistics event, ISI Dublin. As an Irish-based statistical software company, we were very excited at the prospect of welcoming over 2,500 statisticians to our capital city. Initially, we were somewhat worried that the notorious Irish weather would dampen the delegates delight at being in the “Emerald Isle”. But we need not have worried because despite being officially classed as the coldest Irish Summer for over 50 years, the sun actually shone for every day of the Congress. As attendees poured into the newly opened, award-winning Convention Centre Dublin, Mary Byrne, CEO, Helen Murphy, Sales & Marketing Director and Andrew Grannell, warmly welcomed visitors to the Statistical Solutions booth. With attendees from almost every country in the world, there was no doubt that the event was in every sense a World Congress.

Mary Byrne CEO Statistical Solutions, with Prof. Donald B. Rubin at the Statistical Solutions booth at ISI 2011, Dublin

Monday, August 22nd was the highlight day for us, with the arrival of the “Dream Team”. Sir David Cox, Prof. Donald B. Rubin, Prof. Peter Huber and Prof. Stephen Stigler reflected on the past, present and future of statistics. As you can imagine, the debate was extremely lively and stimulating and continued long after the scheduled session ended. Prof. Rubin then visited the Statistical Solutions booth for a demo of SOLAS for Missing Data Analysis and is pictured here with CEO, Mary Byrne. Wednesday August 24th was the special Water Theme day with all talks and presentation focusing on this critical global issue.

Our next event is the ASC (Association of Survey Computing) International Conference in Bristol on September 22-23. Andrew Grannell will present a paper on multiple imputation and it will also be the first outing for new customer support statistician Eoghan Murphy (pronounced Owen).

SOLAS for Missing Data Analysis – 9 Imputation Techniques

BMDP Statistical Software – Original & Best General Stats Package

nTerim – Sample Size for Group Sequential Trials

EquivTest – Equivalence Testing & Confidence Intervals

Take a Free Trial Now!!

The first morning session focused on the challenges in Educational and Health Research, with Jutta Von Maurice (University of Bamberg, NEPS) giving a detailed description of the NEPS Project and the missing data issues that have arisen during this very large study. Trivellore E. Raghunathan (University of Michigan), then went on to describe how multiple imputation can be used to reduce the size of questionnaire by employing Split Questionnaire Survey Designs. Nathaniel Schenker (CDC), then described how multiple imputation has been used throughout the CDC for a variety of multiple problems.

After coffee, the next session focused on Missing Data Problems in Complex Situations, when speakers Joerg Drechsler (IAB Inst. for Employment Research), Elizabeth R. Zell (CDC), Stef van Buuren (Utrecht University) and Florian Meinfelder (University of Bamberg) all gave great insights into how they have used multiple imputation.

After lunch, we heard more about the Applications of Incomplete Data Techniques. Volker Bosch (GfK, Nurnberg), Sara Kleyer (University of Bamberg and Christian Assmann (University of Bamberg, NEPS) all outlined serious problems they faced in attempting to analyse incomplete data.

In the final session of the day, Andrew Grannell of Statistical Solutions gave a demo of the new features in SOLAS 4.0 for Missing Data Analysis and showed how some of the problems outlined earlier could have been approached using SOLAS. Finally, the day concluded with detailed remarks from Donald B. Rubin on each of the presentations with suggestions on how to proceed and perhaps improve on approaches taken. He sincerely thanked and congratulated Prof. Raessler for organizing such as worthwhile event and looked forward to next year’s  5^th AID Workshop.