Examples and Problems in Mathematical Statistics

[Book Description]

Provides the necessary skills to solve problems in mathematical statistics through theory, concrete examples, and exercises With a clear and detailed approach to the fundamentals of statistical theory, Examples and Problems in Mathematical Statistics uniquely bridges the gap between theory andapplication and presents numerous problem-solving examples that illustrate the relatednotations and proven results. Written by an established authority in probability and mathematical statistics, each chapter begins with a theoretical presentation to introduce both the topic and the important results in an effort to aid in overall comprehension. Examples are then provided, followed by problems, and finally, solutions to some of the earlier problems.In addition, Examples and Problems in Mathematical Statistics features: * Over 160 practical and interesting real-world examples from a variety of fields including engineering, mathematics, and statistics to help readers become proficient in theoretical problem solving * More than 430 unique exercises with select solutions * Key statistical inference topics, such as probability theory, statistical distributions, sufficient statistics, information in samples, testing statistical hypotheses, statistical estimation, confidence and tolerance intervals, large sample theory, and Bayesian analysis Recommended for graduate-level courses in probability and statistical inference, Examples and Problems in Mathematical Statistics is also an ideal reference for applied statisticians and researchers.

[Table of Contents]

Preface                                            xv

        List of Random Variables                   xvii

        List of Abbreviations                      xix

    1 Basic Probability Theory

        Part I Theory 1                            1   (1)

      1.1 Operations on Sets                       1   (1)

      1.2 Algebra and σ-Fields               2   (2)

      1.3 Probability Spaces                       4   (2)

      1.4 Conditional Probabilities and            6   (2)

      Independence

      1.5 Random Variables and Their               8   (4)

      Distributions

      1.6 The Lebesgue and Stieltjes Integrals     12  (9)

        1.6.1 General Definition of Expected       12  (5)

        Value: The Lebesgue Integral

        1.6.2 The Stieltjes-Riemann Integral       17  (2)

        1.6.3 Mixtures of Discrete and             19  (1)

        Absolutely Continuous Distributions

        1.6.4 Quantiles of Distributions           19  (1)

        1.6.5 Transformations                      20  (1)

      1.7 Joint Distributions, Conditional         21  (5)

      Distributions and Independence

        1.7.1 Joint Distributions                  21  (2)

        1.7.2 Conditional Expectations: General    23  (3)

        Definition

        1.7.3 Independence                         26  (1)

      1.8 Moments and Related Functionals          26  (9)

      1.9 Modes of Convergence                     35  (4)

      1.10 Weak Convergence                        39  (2)

      1.11 Laws of Large Numbers                   41  (3)

        1.11.1 The Weak Law of Large Numbers       41  (1)

        (WLLN)

        1.11.2 The Strong Law of Large Numbers     42  (2)

        (SLLN)

      1.12 Central Limit Theorem                   44  (3)

      1.13 Miscellaneous Results                   47  (59)

        1.13.1 Law of the Iterated Logarithm       48  (1)

        1.13.2 Uniform Integrability               48  (4)

        1.13.3 Inequalities                        52  (1)

        1.13.4 The Delta Method                    53  (2)

        1.13.5 The Symbols op and Op               55  (1)

        1.13.6 The Empirical Distribution and      55  (1)

        Sample Quantiles

        Part II Examples                           56  (17)

        Part III Problems                          73  (20)

        Part IV Solutions To Selected Problems     93  (13)

    2 Statistical Distributions                    106 (85)

        Part I Theory                              106 (1)

      2.1 Introductory Remarks                     106 (1)

      2.2 Families of Discrete Distributions       106 (3)

        2.2.1 Binomial Distributions               106 (1)

        2.2.2 Hypergeometric Distributions         107 (1)

        2.2.3 Poisson Distributions                108 (1)

        2.2.4 Geometric, Pascal, and Negative      108 (1)

        Binomial Distributions

      2.3 Some Families of Continuous              109 (9)

      Distributions

        2.3.1 Rectangular Distributions            109 (2)

        2.3.2 Beta Distributions                   111 (1)

        2.3.3 Gamma Distributions                  111 (1)

        2.3.4 Weibull and Extreme Value            112 (1)

        Distributions

        2.3.5 Normal Distributions                 113 (1)

        2.3.6 Normal Approximations                114 (4)

      2.4 Transformations                          118 (2)

        2.4.1 One-to-One Transformations of        118 (1)

        Several Variables

        2.4.2 Distribution of Sums                 118 (1)

        2.4.3 Distribution of Ratios               118 (2)

      2.5 Variances and Covariances of Sample      120 (2)

      Moments

      2.6 Discrete Multivariate Distributions      122 (3)

        2.6.1 The Multinomial Distribution         122 (1)

        2.6.2 Multivariate Negative Binomial       123 (1)

        2.6.3 Multivariate Hypergeometric          124 (1)

        Distributions

      2.7 Multinomial Distributions                125 (5)

        2.7.1 Basic Theory                         125 (2)

        2.7.2 Distribution of Subvectors and       127 (2)

        Distributions of Linear Forms

        2.7.3 Independence of Linear Forms         129 (1)

      2.8 Distributions of Symmetric Quadratic     130 (2)

      Forms of Normal Variables

      2.9 Independence of Linear and Quadratic     132 (1)

      Forms of Normal Variables

      2.10 The Order Statistics                    133 (2)

      2.11 t-Distributions                         135 (3)

      2.12 F-Distributions                         138 (4)

      2.13 The Distribution of the Sample          142 (2)

      Correlation

      2.14 Exponential Type Families               144 (2)

      2.15 Approximating the Distribution of       146 (45)

      the Sample Mean: Edgeworth and

      Saddlepoint Approximations

        2.15.1 Edgeworth Expansion                 147 (2)

        2.15.2 Saddlepoint Approximation           149 (1)

        Part II Examples                           150 (17)

        Part III Problems                          167 (14)

        Part IV Solutions To Selected Problems     181 (10)

    3 Sufficient Statistics and the Information    191 (55)

    in Samples

        Part I Theory                              191 (1)

      3.1 Introduction                             191 (1)

      3.2 Definition and Characterization of       192 (8)

      Sufficient Statistics

        3.2.1 Introductory Discussion              192 (2)

        3.2.2 Theoretical Formulation              194 (6)

      3.3 Likelihood Functions and Minimal         200 (2)

      Sufficient Statistics

      3.4 Sufficient Statistics and Exponential    202 (1)

      Type Families

      3.5 Sufficiency and Completeness             203 (2)

      3.6 Sufficiency and Ancillarity              205 (1)

      3.7 Information Functions and Sufficiency    206 (6)

        3.7.1 The Fisher Information               206 (4)

        3.7.2 The Kullback-Leibler Information     210 (2)

      3.8 The Fisher Information Matrix            212 (2)

      3.9 Sensitivity to Changes in Parameters     214 (32)

        3.9.1 The Hellinger Distance               214 (2)

        Part II Examples                           216 (14)

        Part III Problems                          230 (6)

        Part IV Solutions To Selected Problems     236 (10)

    4 Testing Statistical Hypotheses               246 (75)

        Part I Theory                              246 (1)

      4.1 The General Framework                    246 (2)

      4.2 The Neyman--Pearson Fundamental Lemma    248 (3)

      4.3 Testing One-Sided Composite              251 (3)

      Hypotheses in MLR Models

      4.4 Testing Two-Sided Hypotheses in          254 (2)

      One-Parameter Exponential Families

      4.5 Testing Composite Hypotheses with        256 (4)

      Nuisance Parameters---Unbiased Tests

      4.6 Likelihood Ratio Tests                   260 (11)

        4.6.1 Testing in Normal Regression         261 (4)

        Theory

        4.6.2 Comparison of Normal Means: The      265 (6)

        Analysis of Variance

      4.7 The Analysis of Contingency Tables       271 (4)

        4.7.1 The Structure of Multi-Way           271 (1)

        Contingency Tables and the Statistical

        Model

        4.7.2 Testing the Significance of          271 (2)

        Association

        4.7.3 The Analysis of 2 x 2 Tables         273 (1)

        4.7.4 Likelihood Ratio Tests for           274 (1)

        Categorical Data

      4.8 Sequential Testing of Hypotheses         275 (46)

        4.8.1 The Wald Sequential Probability      276 (7)

        Ratio Test

        Part II Examples                           283 (15)

        Part III Problems                          298 (9)

        Part IV Solutions To Selected Problems     307 (14)

    5 Statistical Estimation                       321 (85)

        Part I Theory                              321 (1)

      5.1 General Discussion                       321 (1)

      5.2 Unbiased Estimators                      322 (6)

        5.2.1 General Definition and Example       322 (1)

        5.2.2 Minimum Variance Unbiased            322 (1)

        Estimators

        5.2.3 The Cramer-Rao Lower Bound for       323 (3)

        the One-Parameter Case

        5.2.4 Extension of the Cramer--Rao         326 (1)

        Inequality to Multiparameter Cases

        5.2.5 General Inequalities of the          327 (1)

        Cramer--Rao Type

      5.3 The Efficiency of Unbiased Estimators    328 (3)

      in Regular Cases

      5.4 Best Linear Unbiased and                 331 (4)

      Least-Squares Estimators

        5.4.1 BLUEs of the Mean                    331 (1)

        5.4.2 Least-Squares and BLUEs in Linear    332 (2)

        Models

        5.4.3 Best Linear Combinations of Order    334 (1)

        Statistics

      5.5 Stabilizing the LSE: Ridge Regressions   335 (2)

      5.6 Maximum Likelihood Estimators            337 (4)

        5.6.1 Definition and Examples              337 (1)

        5.6.2 MLEs in Exponential Type Families    338 (1)

        5.6.3 The Invariance Principle             338 (1)

        5.6.4 MLE of the Parameters of             339 (2)

        Tolerance Distributions

      5.7 Equivariant Estimators                   341 (5)

        5.7.1 The Structure of Equivariant         341 (2)

        Estimators

        5.7.2 Minimum MSE Equivariant Estimators   343 (1)

        5.7.3 Minimum Risk Equivariant             343 (1)

        Estimators

        5.7.4 The Pitman Estimators                344 (2)

      5.8 Estimating Equations                     346 (3)

        5.8.1 Moment-Equations Estimators          346 (1)

        5.8.2 General Theory of Estimating         347 (2)

        Functions

      5.9 Pretest Estimators                       349 (1)

      5.10 Robust Estimation of the Location       349 (57)

      and Scale Parameters of Symmetric

      Distributions

        Part II Examples                           353 (28)

        Part III Problems                          381 (12)

        Part IV Solutions Of Selected Problems     393 (13)

    6 Confidence and Tolerance Intervals           406 (33)

        Part I Theory                              406 (1)

      6.1 General Introduction                     406 (1)

      6.2 The Construction of Confidence           407 (1)

      Intervals

      6.3 Optimal Confidence Intervals             408 (2)

      6.4 Tolerance Intervals                      410 (2)

      6.5 Distribution Free Confidence and         412 (2)

      Tolerance Intervals

      6.6 Simultaneous Confidence Intervals        414 (3)

      6.7 Two-Stage and Sequential Sampling for    417 (22)

      Fixed Width Confidence Intervals

        Part II Examples                           421 (8)

        Part III Problems                          429 (4)

        Part IV Solution To Selected Problems      433 (6)

    7 Large Sample Theory for Estimation and       439 (46)

    Testing

        Part I Theory                              439 (1)

      7.1 Consistency of Estimators and Tests      439 (1)

      7.2 Consistency of the MLE                   440 (2)

      7.3 Asymptotic Normality and Efficiency      442 (2)

      of Consistent Estimators

      7.4 Second-Order Efficiency of BAN           444 (1)

      Estimators

      7.5 Large Sample Confidence Intervals        445 (1)

      7.6 Edgeworth and Saddlepoint                446 (2)

      Approximations to the Distribution of the

      MLE: One-Parameter Canonical Exponential

      Families

      7.7 Large Sample Tests                       448 (1)

      7.8 Pitman's Asymptotic Efficiency of        449 (2)

      Tests

      7.9 Asymptotic Properties of Sample          451 (34)

      Quantiles

        Part H Examples                            454 (21)

        Part III Problems                          475 (4)

        Part IV Solution Of Selected Problems      479 (6)

    8 Bayesian Analysis in Testing and             485 (78)

    Estimation

        Part I Theory                              485 (1)

      8.1 The Bayesian Framework                   486 (5)

        8.1.1 Prior, Posterior, and Predictive     486 (1)

        Distributions

        8.1.2 Noninformative and Improper Prior    487 (2)

        Distributions

        8.1.3 Risk Functions and Bayes             489 (2)

        Procedures

      8.2 Bayesian Testing of Hypothesis           491 (10)

        8.2.1 Testing Simple Hypothesis            491 (2)

        8.2.2 Testing Composite Hypotheses         493 (2)

        8.2.3 Bayes Sequential Testing of          495 (6)

        Hypotheses

      8.3 Bayesian Credibility and Prediction      501 (1)

      Intervals

        8.3.1 Credibility Intervals                501 (1)

        8.3.2 Prediction Intervals                 501 (1)

      8.4 Bayesian Estimation                      502 (4)

        8.4.1 General Discussion and Examples      502 (1)

        8.4.2 Hierarchical Models                  502 (2)

        8.4.3 The Normal Dynamic Linear Model      504 (2)

      8.5 Approximation Methods                    506 (7)

        8.5.1 Analytical Approximations            506 (2)

        8.5.2 Numerical Approximations             508 (5)

      8.6 Empirical Bayes Estimators               513 (50)

        Part II Examples                           514 (35)

        Part III Problems                          549 (8)

        Part IV Solutions Of Selected Problems     557 (6)

    9 Advanced Topics in Estimation Theory         563 (38)

        Part I Theory                              563 (1)

      9.1 Minimax Estimators                       563 (2)

      9.2 Minimum Risk Equivariant, Bayes          565 (5)

      Equivariant, and Structural Estimators

        9.2.1 Formal Bayes Estimators for          566 (2)

        Invariant Priors

        9.2.2 Equivariant Estimators Based on      568 (2)

        Structural Distributions

      9.3 The Admissibility of Estimators          570 (31)

        9.3.1 Some Basic Results                   570 (5)

        9.3.2 The Inadmissibility of Some          575 (7)

        Commonly Used Estimators

        9.3.3 Minimax and Admissible Estimators    582 (2)

        of the Location Parameter

        9.3.4 The Relationship of Empirical        584 (1)

        Bayes and Stein-Type Estimators of the

        Location Parameter in the Normal Case

        Part II Examples                           585 (7)

        Part III Problems                          592 (4)

        Part IV Solutions Of Selected Problems     596 (5)

References                                         601 (12)

Author Index                                       613 (4)

Subject Index                                      617