Parallel Computing: A Tutorial for Statisticians,

The purpose of this article is to provide statisticians a brief introduction to parallel computing. We begin by discussing a few basic notions which are fundamental to parallel processing. Next some important aspects of hardware for parallel computers are reviewed. We then provide a brief analysis of system performance including a statistical approach to the performance of one kind of distributed computing system. Next there is a discussion of a particular form of parallel iteration which we have found generally useful followed by a discussion of several statistical applications. We conclude with a review of some of the difficulties of programming parallel systems and mention one programming system we have used which helps overcome these problems. We recommend Bertsekas and Tsitsiklis 1989 for the reader who is interested in further details on many of these topics.