Digital Color Imaging in Biomedicine

1. Introduction

In recent years, signal digitization has reached the pathological laboratory of hospitals. This is particularly apparent in the availability of excellent input devices such as high-precision CCD cameras, easy-to-use image-processing applications enabling high quality processing of images, and filing equipment all at reasonable costs. Such developments are beneficial for the staff working in the pathological laboratory of a hospital, who have had to deal with huge quantities of image data of pathological tissues, because digital-signal devices will enable them to easily digitize and file the pictures taken each day of pathological tissues excised for inspection in the operating room and anatomical theater.
Furthermore, telepathology as well as PACS as applications of telemedicine are attracting attention. Indeed, in some general hospitals, telepathology or remote diagnosis based on pathological images transmitted through a computer network has already been implemented. This has also been made possible by the digitization of image signals and the spread of networks with high bit-rate signal transmission channels.
In this paper, we outline the role of color in the morphological expression of a disease, predict the advantages and disadvantages for hospital pathologists in utilizing the digitized color images of pathological tissues for daily examinations, and describe the function of color in the automatic classification of pathological tissues based on computer processing, a technique we have developed to effectively exploit the digitized visual information obtained from pathological tissues.

2. Color in morphological pathology

Modern pathology is based on humanﾕs highly developed capacity of visual information processing as well as newly developed techniques such as staining, optics and electron microscopes. Pathology investigates the morphological expression of diseases, and pathological diagnosis has contributed greatly to medical science.
Many fields of study, such as biology for example, first classify various substances based on morphological differences, and then specify objects for study and investigate the relationship among them. It is important for the development of studies that morphological information is analyzed in detail, substances are preserved in the form of "specimens", and obtained knowledge is conveyed correctly to contemporary and future researchers. Museums are thus of vital importance.
Development of morphological pathology also is based on various morphological analyses of pathological organs, as well as conveying the morphology of the pathological organs that are studied. Efforts have been made to preserve pathological organs by fixatives such as formalin, even though some information such as color is lost. Many old hospitals and universities with hospitals have museums containing pathological specimens that stand quietly in the corner.
To document the color of pathological organs which is lost in preservation, pathologists use common words and color expression, or draw rough illustrations using colors or pigments.
Before the photograph was introduced in pathology, academic artists had made detailed sketches for valuable cases. These methods had been used to preserve color information that would otherwise be lost. The significance of color in pathomorphological observation is clear from the fact that as soon as color printing techniques were developed, colored illustrations were immediately published in pathology-related magazines in cooperation with artists.
However, the development and progress of photography had led some academic artists to learn to take photographs. Color films were developed and popularized after the Second World War. Before the war, efforts were made to make photographs look as close as possible to the original object through the use of subtle differences in light and shade of black and white. Since the development of color photographs, technical development of films for better reproduction of natural colors has progressed in parallel with the development of illumination techniques as well as studies and popularization of techniques for taking color photographs of pathological organs and tissues.
It has thus become possible to preserve the pathomorphology of pathological organs including information on color, and to convey imaging information through photographs. These developments have greatly contributed to the dissemination and recording of pathological knowledge. It has become possible to preserve the forms and colors of pathological specimens without requiring a large museum. Provided a system for managing photographs is established, pathological photographs can be fully utilized to exchange knowledge and carefully control daily pathological diagnosis.

3. Current conditions of uses of pathological digital imaging

However, the establishment of a system for managing photographs requires enormous human and physical resources. In the past as well as present, hospital management and its application have not been well established in many hospitals in Japan. The most common medical system model involves a clinic consisting of a physician and a nurse, and there is insufficient investment in the resources necessary to manage products, biological materials such as pathological organs and information.
However, digitization of information such as electronic medical charts is attracting attention in Japan as a means of overcoming the problems of lack of human resources and poor management.
In the field of pathological tests, there is insufficient space in many hospitals for organ preservation. Therefore, to preserve information on the forms and colors of pathological organs, images must be filed digitally, even if this means sacrificing the quality of color photographs which accurately reproduce natural colors and forms. Such medical economics and operation systems lie in the background of current developments in Japan.
Recognizing the problems found in many hospitals in Japan, in 1994 we decided to digitally preserve macroscopic pathological images, when our hospital systems were upgraded. We studied systems for inputting and filing images that were on the market at that time and chose a device that was easy to use and met our budget. Although much time is taken for input, the quality of images in terms of actual forms and colors has rarely been problematic. Digital filing enables us to eliminate most problems including loss of macroscopic photographs, taking extra photographs to be delivered to the clinical side, and loss of macroscopic images due to simple errors related to photographing procedure such as rewinding films.
However, there still are problems. When examining macroscopic images, it is important to recognize subtle changes in color such as, jaundice, congestion, changes in exudation and effusion, and pigmentation. Although some of this analog information is lost during the process of digitization, in practice pathological diagnosis means histopathological diagnosis or cytological diagnosis. Thus, subtle color changes of organs are not regarded as important, and the current system is acceptable.
With respect to microscopic images, which are used for histopathological diagnosis or cytological diagnosis, it is possible to preserve specimen slides for a relatively long period and in limited space, and so there is little need for digital filing of microscopic images at present.

4. Telepathology

From November 17 through 30, 1998, medical officers from the Self-Defense Force participated in the International Emergency Relief Project that was initiated in response to the request for disaster relief by the Government of Honduras. At that time, images of intractable dermal ulcers of the crus filmed by a simple digital camera were transmitted via the Internet to dermatologists at the Japan Self Defense Forces Central Hospital, Mishuku, Tokyo for diagnosis and consultation on treatment. This is an example of the application of telecommunications to medicine. Strong interest in telepathology has been expressed by the Hygiene Department of the Self-Defense Force. However, telepathology is not currently implemented at any facilities affiliated with the Self-Defense Force including the National Defense Medical College.
In pathological and cytological diagnosis, the quality of microscopic digital images is an issue in the field of telepathology. Histopathological and cytological diagnoses are performed on images that are digitized and reproduced on monitors at remote areas through high-speed communication circuits. When we observe microscopic pathological tissues and cytological images, we usually see images of pigments binding on pathological tissues and cells. Subtle changes in these artificial colors created in the process of digitization are unlikely to become a problem. In fact, in the field of telepathology, which is used mostly only for microscopic images, it has been found that remote pathological diagnosis is effective. There are estimated to be 18 facilities that currently employ telepathology in Japan, where rapid histopathological diagnosis during operation is carried out almost daily, mostly at affiliated hospitals that do not have full-time pathologists. Remote pathological diagnosis has now progressed from the experimental and introductory stage to the stage of popularization and expansion. Currently, excessive workload on individual pathologists are being discussed through experiences of remote pathological diagnosis.
Regarding H.E. staining, pathologists and technicians tend to have their own favorite colors, and each facility may use pigments made by different manufacturers and staining methods. In remote pathological diagnosis at present, communication is performed among affiliated hospitals, and pathologists become "accustomed" to the colors that are transmitted, so there are few troubles. However, when consultation is performed by electronic communication on a nation-wide basis, a significant problem may arise. The AFIP attached to the U.S. Army has set up a home page for histopathology and is encouraging active participation in world-wide telepathology to accumulate images of valuable cases. Requests for standardization of staining methods may arise from such movements. Although some manufacturers claim that image processing software can be easily used to alter colors to suit each pathologistﾕs preferences or to meet standards, excessive expectations should not be placed on the software.
The key issue of concern in remote pathological diagnosis is whether or not the diagnosed portions are turned into specimens correctly. To the end, pathological diagnosis is made by examining histological images. However, usually, not all areas can be made into histological specimens, and so appropriate portions have to be cut out. The appropriateness of portions is decided by detailed macroscopic observation of organs and palpation of lesions. The cutting out is of major importance when making a rapid diagnosis. Currently, surgeons or trained clinical laboratory technicians assist the pathologists. However, the fact that pathologists cannot perform detailed macroscopic observation or examination of lesion by palpation may remain a significant problem.
We borrowed a telepathology device from AutoCyte Co. of the US and conducted a simulation. The simulation and sales materials of other companies demonstrated that devices marketed in the US are highly satisfactory and performed well in practice.
However, there is a problem that when a CCD camera with super-fine pixels is used, the input image files require more memory capacity than is available on ordinary personal computers, and that preservation and transmission take too long. It is therefore necessary to develop a system for utilizing the full potential of cameras. In addition, images have to be compressed for transmission, and depending upon the memory medium, compression may also be necessary for preservation, so an appropriate compression technology for images is needed for histopathological diagnosis.

5. Problem of color that arose during our experiments on developing a device for automatically classifying pathological tissues, and solutions

Devices for automatically screening cytological diagnosis are commercially available. In automatic image processing, the quantity and quality of color of specimens are the problems, when the basic characteristics of images are extracted. Most automatic diagnostic devices that have been developed so far employ standardized or specified stains and slide glasses.
Our experiments on the computerized classification of histopathological tissues of mammary tumors demonstrated that sensitivity and specificity increased when color was one of the characteristic values, confirming the importance of color information even for microscopic images. When color, average of chromaticity and spatial dispersion were employed, some tissue patterns showed good results.
In addition, with respect to color, experiments of development of cognitive capacity of patterns demonstrated that the use of absolute color information made computer processing easier. However, in reality, colors of histopathological specimens vary greatly even for the same tissue because 1) the conditions of producing specimens vary, and 2) properties of tissues themselves vary. Even when RGB changes, if each factor changes at the same ratio, the chromaticity (r/(r+g+b), etc.) does not change. Thus, the use of chromaticity facilitated processing.
In extraction of regions, better results were obtained by using relative color information (such as color differences from surrounding areas) rather than using absolute color information. For instance, in extraction of nuclear regions, at first, output values of neurons of our automatic classification system using neural networks were simply threshold-processed. Then, automatic regulation of threshold values was installed so that the largest color difference was obtained between nuclear regions and non-nuclear regions following threshold processing. This method produced better results of extracting nuclear regions.
In experiments of isolation of Helicobacter pylori in tissues of gastritis, the background color overlapped the color of the bacteria. Color differences among surrounding areas (relative color information) were important rather than just absolute color information. The color information of the target region itself was not useful. Among problems concerning color, there is a problem of illumination. In the case of isolation of H. pylori, most parameters including extraction of regions were processed using relative information. Consequently, illumination was not a problem, and it was not necessary to standardize intensity. However, when classifying mammary tumors, absolute color information was fairly important, and so intensity had to be standardized. In detail, first an empty region in an image was extracted, and then the intensity of the empty region was modified to a prescribed fixed value by multiplying each value of R, G, and B. That is, the color of an empty region was assumed to be that of the illumination. Images were converted so that the color of the empty region was standardized. Thus, images were assumed always to have the same illumination. In practice, for brighter intensity in images, when the intensity value of pixels was standardized to the empty intensity value, satisfactory results were obtained. However, when the intensity of illumination at one place in an image was abnormally different from others, it did not produce good results. For instance, in case of a bright center with dark surroundings, and the difference became too great, and the surroundings were converted darker, and empty places in surroundings were recognized as not-empty. Ideally, standardization should be performed at each place using empty intensity values. However, canceling each other by computer processing takes too much calculation time, so it would be better to standardize at the time of filming. This would reduce the burden on software processing when 1) the conditions of producing specimens (thickness, staining etc.) must be standardized, and 2) the conditions of illuminating the microscope should be standardized (color temperature and intensity of microscopic field are standardized without specimens).

6. Conclusion

We are losing opportunities to discuss how to advance digitization of medical digital information. However, we must continually strive to improve information systems such that the patient has priority over logic of medicine and treatment as it is now. We must attempt to solve problems related to color in disease, such as the importance to patients of color as medical information in the management of disease, and how color be managed as useful information.