Near-infrared (NIR) optical properties of turbid media, e.g., tissue, can be accurately quantified noninvasively using methods based on diffuse reflectance or transmittance, such as frequency domain photon migration (FDPM). Factors which govern the accuracy and sensitivity of FDPM-measured optical properties include instrument performance, the light propagation model, and fitting algorithms used to calculate optical properties from measured data. In this article, we characterize instrument, model, and fitting uncertaintics of an FDPM system designed for clinical use and investigate how each of these factors affects the quantification of NIR absorption (μa) and reduced scattering (μs′) parameters in tissue phantoms. The instrument is based on a 500 MHz, multiwavelength platform that sweeps through 201 discrete frequencies in as little as 675 ms. Phase and amplitude of intensity modulated light launched into tissue, i.e., diffuse photon density waves (PDW), are measured with an accuracy of ±0.30° and ±3.5%, while phase and amplitude precision are ±0.025° and ±0.20%, respectively. At this level of instrument uncertainty, simultaneous fitting of frequency-dependent phase and amplitude nonlinear model functions derived from a photon diffusion approximation provides an accurate and robust strategy for determining optical properties from FDPM data, especially for media with high absorption. In an optical property range that is characteristic of most human tissues in the NIR (5×10−3<μa<5×10−2 mm−1, 0.5<μs′<2 mm−1), we theoretically and experimentally demonstrate that the multifrequency, simultaneous-fit approach allows μa and μs′ to be quantified with an accuracy of ±5% and ±3%, respectively. Although exceptionally high levels of precision can be obtained using this approach (<1% of the estimated absorption and scattering values), we show that the absolute accuracy of optical property measurements is highly dependent on specific factors associated with instrument performance, model function relevance, and details of the fitting strategy used to calculate μa and μs′. © 2000 American Institute of Physics.