The BitMap exercise is the largest multi-laboratory comparison of DO instruments to date, encompassing 12 institutions and 28 systems. The study aims to establish a culture of PAS in the DO community and propose a common methodology that can be adopted in other environments. The work has yielded simple numeric values, called synthetic indicators, for each of the tests employed. The indicators enable simple comparison across the gamut of instruments enrolled.
To compare the performance of these instruments, the researchers used three internationally adopted protocols — BIP, MEDPHOT, and NEUROPT — to challenge each DO system. The BIP protocol characterized the most basic optical performances of each instrument whereas the MEDPHOT protocol characterized how well each instrument could recover homogenous optical properties, that is, absorption and reduced scattering coefficients. The NEUROPT protocol tested how well each system could detect inhomogeneities in a sample by focusing on measures related to contrast.
Additionally, researchers agreed on three different “phantom” kits, each of which was tailored for one of the protocols. A phantom refers to an artificial structure that is typically used for calibration and testing and that emulates certain properties of human tissue.
The experiments consisted of executing an assortment of relevant tests from each protocol on its respective phantom kit using each of the DO instruments. Results yielded information on which instruments and techniques show the best performance, reproducibility of result, and the amount of variability between measurements made using different systems.
The researchers found a substantial difference in hardware performances across different systems, which helped them identify some critical issues related to performance assessment in DO. They plan to deploy the entire data set gathered through Action 1 into an open data repository (Action 2). This step would enable analysis and comparison of specific aspects of the DO systems, which is part of the aim of project Action 3.
“Photon migration through the human body is complicated by the biological variability, but not the basic physics underlying it all,” said senior author Antonio Pifferi from Politecnico di Milano. “We can disentangle the uncertainties and artifacts produced by the instruments and analysis tools from the biological variability, with great benefit for clinical use.”
The researchers anticipate the efforts enabling more powerful and more reliable DO technology with greater accuracy and enabling more convenient diagnostics.
The research was published in the Journal of Biomedical Optics (www.doi.org/10.1117/1.JBO.27.7.074716).