An update on modeling dose response relationships: accounting for correlated data structures and heterogeneous variance in linear and non-linear mixed models

Abstract Text: Advanced methods for dose-response assessments are used to estimate concentrations of a nutrient that optimize a given outcome of interest, thereby determining nutritional requirements for optimal performance. Traditionally, many dose-response methods use a fixed-effects framework that assumes mutually independent observations with homogeneous variances. Yet, experimental data often present a design structure that includes correlations between observations (e.g. blocking, nesting, etc.), as well as heterogeneity of variances that can mislead inference if disregarded. Our objective is to demonstrate practical implementation of computationally-intensive linear and non-linear mixed models methodology to describe dose-response relationships accounting for correlated data structure and heterogeneous variances. To illustrate, we model data from a randomized complete block design study to evaluate the Standardized Ileal Digestible (SID) Trp:Lys dose-response on G:F of nursery pigs. A base linear mixed model was fit to explore the functional form of G:F relative to Trp:Lys and assess model assumptions, in particular residual homoscedasticity. Next, we fitted 3 competing dose-response mixed models to G:F, namely a quadratic polynomial (QP), a broken-line linear (BLL) ascending model, and a broken-line quadratic (BLQ) ascending model, all of which included heteroskedastic specifications, as dictated by the base model, and used parameter estimates from the base model as initial values. The GLIMMIX procedure of SAS (Version 9.4) was used to fit the base and QP models and the NLMIXED procedure was used to fit the non-linear models. We further illustrate the use of a grid-search approach to facilitate convergence and parameter estimation in non-linear mixed models, as this seemed to be the most common implementation problem. Model fit between competing dose-response models was compared using maximum-likelihood-based Bayesian Information Criterion (BIC). The QP, BLL, and BLQ models fitted on G:F of nursery pigs yielded BIC values of  353.7, 343.4, and 345.2, respectively, indicating a better fit of BLL followed closely by BLQ. The BLL breakpoint estimate of the SID Trp:Lys was 16.5% (95% CI: [16.1, 17.0]%), whereas the BLQ estimate was 16.0% (95% CI: [15.5, 16.6]%). Importantly, accounting for heterogeneous variance enhanced inferential precision as the breadth of the CI for mean breakpoint decreased by approximately 44%, from [15.8, 17.4]% to [16.1, 17.0]% SID Trp:Lys. In summary, we illustrate the use of linear and non-linear mixed models for dose-response relationships accounting for heterogeneous residual variances, discuss important diagnostics and their implications for inference, and provide practical recommendations for computational troubleshooting.

Keywords: linear and non-linear mixed models, dose-response, heterogeneous variance