Abstract: Pictures of typical specimens selected from most kinds of morphological laboratory tests were digitized and their diagnostic reliability was evaluated. The quality of most of the properly prepared digitized pictures were almost the same as slide film, but some of them required higher pixel density. In addition, there were large variations in the reproduction of colors among the displays, which may inadvertently cause erroneous diagnoses. Because some standardization of color images is indispensable to prevent this serious problem, a comprehensive model was assumed and possible methods were discussed.
On the basis of the RGB system, color-matching technologies, color calibrators for displays and color charts used to adjust displays have been introduced to prevent such diagnostic accidents. Multispectral imaging is a more promising solution that can reproduce precise colors and compensate for differences in illuminant conditions.
Unfortunately, the technologies used for digital color imaging will not be adequately standardized for medical application in the near future. Therefore, another temporary and practical solution based on the concept of 'diagnostic equivalence', in which a set of typical medical images with authorized diagnoses is used as a practical calibrator for common displays, should be considered at present.
Intensive use of multimedia technology is rapidly progressing and digital images will soon be common not only in morphological laboratory diagnosis, but also in every other medical subfield, to which the expertise presented here is expected to be efficiently applied.

1. Introduction

Many critical decisions in medicine are made based on morphological evidence observed in various color images, therefore accurate as well as precise recording and reproducing of colors is essential. In clinical pathology and laboratory medicine, laboratory information systems, which are widely used already, encourage the spread of digital imaging, therefore many investigations on digital color imaging have been already made in this field.
In this paper, in addition to the experiences related to color reproduction and digital imaging in morphological laboratory diagnosis, discussions and trials of the standardization of digital color imaging based on such experience, which are expected to be applied to other subfields of medicine, are reviewed.


2. Studies on diagnostic reliability

One of the leading comprehensive research studies on the diagnostic reliability of digitized color images has been promoted by the Morphological Internet Survey Research Project Team [1-6]. The Ministry of Science, Education and Culture of Japan organized it in 1998, and the team consists of nine researchers from seven universities and 29 co-researchers from various fields of laboratory medicine and clinical pathology in Japan (for details, see its website at http://survey.umin.ac.jp/). The original purpose was proper application of the Internet to the control survey of morphological laboratory tests (Fig. 1). But since the preliminary experiment revealed the urgent necessity for standardization of color information of digitized images, how to calibrate and how to verify the color accuracy of display equipment became its main subjects.
To investigate digitized images sent via the Internet, instead of ordinary slide film, which are properly used for the control survey of morphological laboratory tests, pictures of typical specimens from urinalysis, hematology, microbiology, immunology, cytology, chromosome analysis, physiology and ultrasonography were digitized and their diagnostic reliability was evaluated using various display equipment including an extra high resolution (QSXGA, 200 pixel per inch) LCD.
The problems revealed included inaccurate color reproduction, rough gradations of color and insufficient density of pixels, with varying degrees of relevance as regards their seriousness in these areas. Namely, the quality of most of the digitized pictures was almost the same as slide film, but some of them needed to be observed on the QSXGA display to be viewed at proper quality. In hematological diagnosis, the colors of stained dyes themselves, as well as their change in color caused by various chemical reactions with the components of each blood cell, are considered extremely important. In addition, there are large variations in reproduction of colors among displays, which may incidentally cause erroneous diagnoses (Fig. 2).

3. A process model of morphological diagnosis [3]

3.1 Diagnosis using conventional images
To investigate the problems concerning color reproduction, a comprehensive model was assumed, in which the process of medical diagnosis based on conventional visual data was broken down into the following steps.
(A1) The subject of observation emanates proper light waves of various wavelengths. This step is affected by the characteristics of the surface of the subject and the illumination. Penetrated light or invisible electromagnetic waves converted into visible light, etc. require more complicated processes, but light waves finally emanated are functions of the characteristics of the subject and the illuminants.
(A2) The light waves pass through an iris, a lens and a vitreous body, and reach the retina. This step is affected by the color of the iris and the transparency of the path of the light waves.
(A3) The rod cells convert the brightness of the light waves and the three kinds of cone cells convert the color of the light waves into frequency modulated pulse signals each and transmit them to the optic nerves. There may be many interactions between these sensor cells, but most processes are unclear. One important fact is that adaptation of the human retina to surrounding illumination is greatly affected by the signals produced by the sensor cells in the aspects of both color and brightness.
(A4) The signals are transmitted to the brain and converted into image data. This step is mentioned as visual sensation and includes the specialized process of various image abstraction.
(A5) Each image is connected to some concepts accumulated in the memory and given particular meanings. This step is mentioned as visual recognition and most of its processes have been only fragmentarily elucidated.
(A6) Each concept is compared to the experiences accumulated in the brain, and deduced by the knowledge established in the brain. This step is mentioned as decision making and the study of the relationship between visual recognition and medical diagnosis is as yet in a hypothetical stage.

3.2 Diagnosis using digitized images
On the other hand, the process of medical diagnosis based on digital images is broken down as follows.
(B1) Same as (A1).
(B2) The light waves are converted into digitized image data that are composed of color information of each pixel represented by the RGB system and the coordinates of the pixel on a two-dimensional plane. In this step, not only large parts of information of the original light are lost, but also various biases that depend on the characteristics of the equipment used for A-D conversion are added.
(B3) The digitized image data are stored or transferred for use at another time or place.
(B4) According to the color information of each pixel of the digitized image data, the red, green and blue dots of the display equipment corresponding to the pixel emanate light of each color with appropriate combination of intensities to give the same stimulus to the retina as the original color. Because there is a large discrepancy between the wavelengths of red, green and blue light and the peaks of the response curve of three types of the cone cells, some type of approximation is indispensable in this step.
(B5) The light waves pass through an iris, a lens and a vitreous body, and reach the retina. This step is affected by the color of the iris and the transparency of the path of the light waves. Unlike natural light, these light waves include only three bands of the wavelength. Therefore, these influences may cause different results from (A2).
(B6~9) Same as (A3~6)
The steps from (B2) to (B5) considerably affect the quality of the original images and could possibly cause wrong diagnoses when observing digitized images instead of conventional analog ones.


4. Accurate color reproduction [3,5,6]

At present, we have several techniques for compensating for the deterioration of color accuracy caused by the steps from (B2) to (B5) of the process model, though they have technological limitations.
Colors reproduced by CRT displays and some LCD displays can be calibrated with proper commercial equipment. But others such as plasma displays, digital projectors and head mounted displays, which could possibly also be used to observe the medical images, cannot yet be calibrated.
As the range and the variety of colors synthesized by the output devices are very different, it is impossible to equalize the physical color specifications among them. The technique of adjusting the colors of digital images according to the characteristics of each device so as to be sensed by human eyes as equally as possible is called color management. It was developed initially in the desktop publishing field and must be modified to satisfy medical requirements. Namely, the former should make printed images similar to the original ones, and the latter should make displayed images similar to the standard images acquired in the brains of experienced people.
Colors transmitted to a distant location and reproduced on a display there are possibly affected by variations of illumination, characteristics of the camera and distortion produced by transmission. To compensate for these influences, a color chart taken simultaneously with the object is used to adjust the color values of the displayed images to reproduce the same colors as those of the original chart. But this technique cannot reproduce the same colors on a display under different illumination. Also, human sensing of color is affected by the illumination surrounding the displays and there is no way to compensate this kind of influence.


5. A road map for standardizing medical color images

5.1 Multispectral imaging [7]
One of the major causes of color problems concerning medical diagnosis is the essential limitation of conventional RGB systems, which record colorimetric values of only three primary colors, red, green and blue. Multispectral imaging, which makes it possible to record the spectral reflectance of objects for accurate color reproduction, will provide an important solution to these problems.
One of its unique advantages is the ability to reproduce precise colors under various illumination. A multichannel image of the object is taken using a multispectral camera and the spectral reflectance of the object is deduced from the image using its statistical characteristics. Simultaneously, spectral power distribution of the illumination used to take the image is measured. The image data, from which the component of the illumination is removed, are converted with the spectral power distribution of the illumination used for observation to reproduce same colors under it.
According to previous studies, not so many medical images should require more precise color representation than possible with the RGB system. But in such cases, an adequate number of principal components to represent the required colors should be clarified using the techniques of multispectral imaging.

5.2 Diagnostic equivalence [3,5,6]
It will take still more time to establish a comprehensive theory to manage colors in medical imaging and it is not practical that every medical terminal should be equipped with expensive displays exclusively for medical use. Therefore some simple and inexpensive methods to calibrate common displays should be provided as soon as possible.
To meet this demand, the novel concept of 'diagnostic equivalence' was introduced, which means that two displays reproducing colors differently are considered medically equivalent if the same diagnoses are made observing medical images on both of them. And also physical differences between a digitized image and the original image can be allowed as far as they do not affect the medical diagnosis. To verify this kind of equivalence, a set of typical medical images with their diagnoses pre-decided by authorities are proposed to be used as a practical calibrator. Medical professionals can evaluate and adjust their own displays by comparing the diagnoses made observing the images using their displays with the authorized ones. Although this is not a complete solution, if this approach works successfully, most of the practical problems can be prevented.
The concept of diagnostic equivalence is very practical, but requires a huge trial-and-error process to be put into practice. When we succeed in accumulating a number of concrete cases, the boundary between digitized medical images used properly and ones causing erroneous diagnoses can be expected to be made so clear that some quantitative index for prediction will be available. For this purpose, the multispectral imaging discussed above is expected to play an important role by providing precise methods for investigation.


6. Conclusions

Digital color imaging in medicine is a new research frontier and just took the first step of its growth in morphological laboratory diagnosis. The best present recommendation to decrease erroneous diagnoses is that each digitized medical image should be displayed using at least both a CRT and a flat panel display, followed by examination and revision by specialists of the respective fields before putting them into practical use [4].
Further investigation, especially on the possibilities of multispectral imaging, requires a broad range of international collaboration among specialists not only from medical fields but from engineering fields. To promote this collaboration, the Digital Biocolor Society (http://biocolor.umin.ac.jp/) was recently established after two symposia focused on this theme and anyone interested is invited to join it.


Acknowledgments

A part of this study was supported by a Grant-in-Aid from the Ministry of Science, Education and Culture of Japan (Grant No. 10672172, entitled 'Internet-based Control Surveys for Morphological Laboratory Tests', 1998-1999). I am grateful to all the members of the Morphological Internet Survey Research Project Team and the Digital Biocolor Society for their cooperation.


References

1.

Masahiro NISHIBORI, Kiichi ITOH, Kiyoaki WATANABE, Harushige KANNO and Yasuhiro OHBA : Use of WWW in a Control Survey for Morphological Laboratory Tests. Proceedings of the Ninth World Congress on Medical Informatics, 803, 1998 (http://mn.umin.ac.jp/medinfo98-803.html)

2.

Edited by Masahiro NISHIBORI : A Final Report of the Research Entitled 'Internet-based Control Surveys for Morphological Laboratory Tests' supported by a Grant-in-Aid from the Ministry of Science, Education, Sports and Culture of Japan (Grant No. 10672172, 1998-1999), 2000 (http://mn.umin.ac.jp/work20000331.html)

3.

Masahiro NISHIBORI : Color Representation of Digital Imaging in Medicine. XX World Congress of Pathology and Laboratory Medicine, 167-172, 1999 (http://mn.umin.ac.jp/work19990921.html)

4.

Masahiro NISHIBORI : Problems and Solutions of Digital Imaging in Morphological Diagnosis. The Proceedings of the 2nd Symposium of the 'Color' of Digital Imaging in Biology and Medicine, 8.1-8.5, 2000 (http://biocolor.umin.ac.jp/sympo200004/proc18.pdf)

5.

Masahiro NISHIBORI : Does Color Reproduction of Digital Images Affect Diagnoses? Proceedings of the 19th Joint Conference on Medical Informatics 336-337, 1999 (http://mn.umin.ac.jp/work19991125.html)

6.

Masahiro NISHIBORI : Standardization of Color in Medical Imaging. Japanese Journal of Medical Informatics 20(2) : 165-167, 2000 (http://mn.umin.ac.jp/work20000603.html)

7.

Masahiro NISHIBORI : The Role of Multispectral Imaging in Medicine. Proceedings of International Symposium on Multispectral Imaging and Color Representation for Digital Archives, 114-116, 1999 (http://mn.umin.ac.jp/work19991021.html)