Dear Faculty Members and Fellow Students,

You are cordially invited to attend my thesis defense.

Title: Minimum Energy Designs: Extensions, Algorithms, and Applications

Advisors: Dr. Roshan Vengazhiyil and Dr. Jeff Wu

Committee Members:

  Dr. Brani Vidakovic

  Dr. Benjamin Haaland

  Dr. William Myers (Procter & Gamble)

Date and Time: Thursday, June 2, 2016, 11:00AM - 12:30PM.

Location: ISyE Groseclose 304.

Abstract:

Minimum Energy Design (MED) is a recently proposed technique for generating deterministic samples from any arbitrary probability distribution. The idea originated from space-filling designs in computer experiments. Most space-filling designs look for uniformity in the region of interest. In MED, some weights are assigned in the optimal design criterion so that some areas are preferred over the other areas. With a proper choice of the weights, the MED can asymptotically represent the target distribution.

In this dissertation, we improve and extend MED in three different aspects. The dissertation consists of three chapters. In Chapter 1, we propose an efficient approach that uses MED to construct proposals for an independence sampler in a Monte Carlo Markov chain, which integrates MED with Monte Carlo techniques. The MED criterion is generalized and a fast algorithm for constructing MEDs is developed in Chapter 2. Finally, in Chapter 3, we propose a new type of MEDs and a new modeling method for robust parameter design in computer experiments.

Monte Carlo (MC) and Markov Chain Monte Carlo (MCMC) methods have found wide application in studying and analyzing complex systems, among which Metropolis-Hastings algorithm is commonly used. Traditional Metropolis-Hastings proposals, which move locally, are not efficient to sample from complex distributions with multiple modes. Existing tempering methods generate multiple chains at different temperatures, but how to efficiently transfer the mixing information from high to low temperature chains is unknown and is a challenging problem. In the first chapter, we propose a new approach to construct proposals for independence sampler using the idea of MED. Between two adjacent temperatures, MED points are selected to keep and transfer the mixing information. Final samples are generated by independence sampler with the exploratory proposals constructed by the selected MED points. Simulations and a real data example show that the proposed approach is more stable and efficient than existing tempering methods, which can save a large number of function evaluations.

When evaluations on the posterior distribution become expensive, traditional MC/MCMC methods are infeasible because of the requirement of large samples. MED is a good way to overcome this problem. It can be viewed as a ``deterministic’’ sampling method that avoids repeated sampling in the same places, which dramatically decreases the number of required samples. However, MED has two limitations, which are improved in this chapter. One is its efficiency in integration. The integration error rate using MED points is low and can be worse than MC in high dimensional cases. In Chapter 2, we define a generalized distance and use it to generalize the MED criterion. With a proper choice of the tuning parameter, the efficiency of the generalized MEDs is greatly improved. The other limitation is the construction algorithm. An MED is constructed by a one-point-at-a-time greedy algorithm, where a global optimization is required in each iteration. The function evaluations are too many to make MED competitive to MC/MCMC methods. In Chapter 2, we develop a fast algorithm for constructing MEDs with much less function evaluations. In each iteration, the algorithm constructs simplexes to search the optimal MED point while keeping all the evaluated points as a candidate list for finding good starting points in the next iteration. The proposed algorithm is shown to have better performance with much less function evaluations.

Space-filling designs, commonly used in computer experiments try to spread out points uniformly in the experimental region. However, in robust parameter design, when the objective is to achieve robustness against noise factors, uniformity is no longer needed in the space of noise factors. This is because noise factors usually follow non-uniform distributions such as normal distribution. It makes more sense to place points in the high probability regions where more “actions” take place. In Chapter 3, we develop new design and modeling methods for robust parameter design experiments. In the design part, a new design based on the generalized MED criterion is proposed, where different tuning parameters are used for control and noise factors. Since the design points are not equally-spaced, stationary covariance functions can lead to numerical instability in computation and tend to perform poorly in prediction. In the modeling part, we propose a simple but efficient nonstationary Gaussian process that takes into account of the experimental design structure to solve this potentially difficult problem. Both the proposed design and model are demonstrated to improve the performance over conventional methods using simulated examples and a real example on oral care packaging process.