Optimization Trials

Considerations for the Optimization phase of MOST

What experimental designs are used for optimization trials in MOST?

There are many different approaches to experimentation that can be used in the optimization phase.  Factorial experiments, fractional factorial experiments, sequential multiple-assignment randomized trials (SMARTs), micro-randomized trials, system identification, or any other suitable approach can be used.  MOST does not require any particular approach to experimentation in the optimization phase, only that the approach selected is highly efficient, in other words, is the best one according to the resource management principle.  The choice of approach depends on the type of intervention that is to be optimized, the exact empirical information that is needed, and the resources that are available to conduct the experimentation.

What are the most important considerations when choosing an experimental design for an optimization trial?

First, it is important to select an experimental design that addresses the research questions at hand. Different designs address subtly different research questions.

Second, it is important to select an experimental design that is efficient, that is, makes the best use of available resources to address the key scientific questions. Different kinds of resources may be available in different situations. In some areas of research, subjects are expensive or hard to get. In others, the overhead costs associated with each experimental condition may be high. Different experimental designs demand different kinds of resources, so a design that is very economical in one setting may be expensive in another. (See Chapter 6 of Collins, 2018.)

Where do pilot studies fit into MOST?

The answer to this question depends on your definition of the term “pilot study.” Unfortunately there is a lot of confusion about this, because different people, even people from the same field, may use different definitions of this term.

Those of us working in MOST use the definition of pilot study that was suggested by Leon, Davis, & Kraemer (2011).  This article defines a pilot study as a study aimed at examining feasibility in preparation for a more formal study.  According to this definition, pilot studies do not form the basis for classical hypothesis testing.  Conversely, if you are planning to conduct classical hypothesis testing, you are not doing a pilot study.  By this definition, optimization trials are never pilot studies, because they are serious, fully powered, carefully conducted, and tightly controlled experiments.

Conducting a pilot study to try out individual intervention components before conducting an optimization trial is a good idea. This is something careful intervention scientists have always done before conducting an RCT, and it should be done as part of the preparation phase in MOST, before conducting the component selection experiment.

Reference

Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research, 45, 626–629.

Does the MOST framework include use of the randomized controlled trial (RCT)?

Definitely. As figure 1 shows, generally an RCT is used to complete the evaluation phase of MOST.

Overview of MOST. 1. Preparation Phase - Purpose: Lay groundwork for optimization. Activities: a. Derive/revise conceptual model. b. Identify set of candidate components. c. Conduct pilot tests. d. Identify optimization criterion. 2. Optimization – Purpose: Build optimized intervention. Activities: a. conduct optimization trial(s). i. Factorial experiment ii. Fractional factorial experiment iii. SMART iv. Micro-randomized trial v. System identification vi. Other b. Identify intervention that meets optimization criteria 3. Optimized intervention expected to be sufficiently effective? Yes = continue to Evaluation. No = return to Preparation (due to the Resource Management Principle.” 4. Evaluation Phase – Purpose: Confirm effectiveness of optimized intervention. Activity: Random controlled trial

Figure 1 is drawn from page 24 of
Collins, L. M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: the multiphase optimization strategy (MOST). Springer.

Is there such a thing as a “MOST design”?

It depends on what it meant by the term “design.”  If “design” is referring to an overall approach to or framework for conducting research, then the answer is yes.  However, if “design” is referring to a particular experiment design, then the answer is no. There are many different experimental designs that can be considered for the optimization trial. Occasionally people call a factorial design a MOST design. PLEASE DO NOT DO THIS. Factorial experiments are used in many contexts besides MOST. We definitely did not invent factorial experiments; they were invented by R.A. Fisher in the 1920’s!

Return to MOST home page.

Read practical questions about using MOST.

 

Last updated: May 6, 2020