Molecular Vision 2022; 28:492-499
<http://www.molvis.org/molvis/v28/492>
Received 01 April 2022 | Accepted 29 December 2022 | Published 31 December 2022
Molecular Vision 2022; 28:492-499
<http://www.molvis.org/molvis/v28/492>
Received 01 April 2022 |
Accepted 29 December 2022 |
Published 31 December 2022
Kent R. Barter,1 Hélène Paradis,2 Robert L. Gendron,2 Josué A. Lily Vidal,3 Oscar Meruvia-Pastor1
1Department of Computer Science, Memorial University, St. John's, A1B 3X5, NL, Canada; 2Faculty of Medicine, Memorial University, St. John's, A1B 3V6, NL, Canada; 3Department of Ophthalmology, College of Medicine, University of Saskatchewan, Saskatoon, SK, S7K 0M7, Canada
Correspondence to: Robert Gendron, Memorial University, Biomedical Science, 300 Prince Philip Parkway, St. John's, NL A1B 3V6, Canada; Phone: +1 (709)-864-3359; FAX: +1 (709)-864-6007; email: address: rgendron@mun.ca
Spectral domain-optical coherence tomography (SD-OCT) has become an essential tool for assessing ocular tissues in live subjects and conducting research on ocular development, health, and disease. The processing of SD-OCT images, particularly those from non-mammalian species, is a labor-intensive manual process due to a lack of automated analytical programs. This paper describes the development and implementation of a novel computer algorithm for the quantitative analysis of SD-OCT images of live teleost eyes. Automated segmentation processing of SD-OCT images of retinal layers was developed using a novel algorithm based on thresholding. The algorithm measures retinal thickness characteristics in a large volume of imaging data of teleost ocular structures in a short time, providing increased accuracy and repeatability of SD-OCT image analysis over manual measurements. The algorithm also generates hundreds of retinal thickness measurements per image for a large number of images for a given dataset. Meanwhile, heat mapping software that plots SD-OCT image measurements as a color gradient was also created. This software directly converts the measurements of each processed image to represent changes in thickness across the whole retinal scan. It also enables 2D and 3D visualization of retinal thickness across the scan, facilitating specimen comparison and localization of areas of interest. The study findings showed that the novel algorithm is more accurate, reliable, and repeatable than manual SD-OCT analysis. The adaptability of the algorithm makes it potentially suitable for analyzing SD-OCT scans of other non-mammalian species.
The biology of the visual systems of many non-mammalian vertebrate species has been relatively understudied at the multi-dimensional imaging level. Meanwhile, spectral domain-optical coherence tomography (SD-OCT) has become an essential methodological approach for studying ocular tissues in live mammalian subjects [1-3]. In the last several years, SD-OCT has been used to study the ocular structures of small teleosts, such as zebrafish (Danio rerio) [4]. More recently, larger marine teleosts, such as lumpfish (Cyclopterus lumpus), have been used as model organisms [5,6]. The use of automated medical imaging technologies instead of manual analytical procedures has become more prevalent in biomedical research [7,8]. However, the processing of SD-OCT images for teleosts and similar species is labor intensive and of limited utility due to a lack of automated analytical programs [9]. Segmentation analysis—one of the automated analytical tools currently available for SD-OCT imaging of mammals—allows automated, precise measurement of the anatomic layers of the retina. Such an approach would facilitate analyses of retinal development and health in non-mammalian species.
The lumpfish is a visually-guided, predatory North Atlantic teleost fish that has become an important cleaner fish species in food fishery aquaculture for seeing and eating sea lice from Atlantic salmon [10-12]. The lumpfish visual system is vital for survival in the wild. Furthermore, since lumpfish genomic resources are public, this species has become a unique and feasible model organism for studying the comparative biology of the vertebrate visual system [5,6]. Since lumpfish have recently been declared threatened in the wild (Committee on the Status of Endangered Wildlife in Canada (COSEWIC), Government of Canada) [13]. New insights into the physiologic systems upon which lumpfish rely for sensing their environment could lead to a better understanding of this species in the context of changing ocean environments.
In recent studies of the lumpfish visual system, we described several ocular features that set lumpfish apart from other teleosts and mammals [5,6]. Furthermore, we found that lumpfish possess a unique retinal partitioning anatomy. Although we have presented the first SD-OCT imaging data from lumpfish retinas [5], the data analysis and segmentation software accompanying SD-OCT instrumentation is specific to mammalian retinas and cannot be used to perform segmentation analysis on retinal tissues from teleosts, such as lumpfish or zebrafish.
The zebrafish is a freshwater teleost species that is recognized for its fully defined genome, ease of culture, and utility as a model organism [14]. Similar to lumpfish, the zebrafish visual system is important for survival in the wild and in culture. SD-OCT imaging has been applied to zebrafish [15]; however, retinal segmentation software built for this or other teleost species has not been described.
In the present study, we addressed the lack of automated segmentation analysis for teleosts by developing and implementing a novel computer algorithm for the quantitative analysis of large volumes of retinal SD-OCT imaging data of teleosts, including lumpfish and zebrafish. This tool differs from other retinal segmentation algorithms in that it was created for the purpose of segmenting non-mammalian teleost species. It is also unique because it uses thresholding instead of traditional graph theory or neural network-based methods. This method was designed to interpret individual images and provide the user with a new perspective on retinal segmentation. The algorithm allows precise measurement of the retinal architecture of live teleost ocular tissues with a high-throughput capacity and creates new research algorithm for studying these tissues.
The Institutional Animal Care Committee of Memorial University approved the use of cultured lumpfish [1] (Department of Ocean Sciences, Memorial University) and C57BL/6J mice (The Jackson Laboratory, Bar Harbor, ME) of mixed gender in the present study (protocols 17–03-RG and 17–01-RG). Retinae from adult mice (n=20), lumpfish 100 days post hatch (dph; n=10) and lumpfish 250 dph (n=6) were noninvasively imaged by SD-OCT essentially as described previously [5] and detailed below. The lumpfish were randomly selected from three holding tanks containing 150 specimens each. Mouse retinal scans were selected based on the quality of the images and used for comparison.
The fish were transported from the Dr. Joe Brown Aquatic Research Building (JBARB), Department of Ocean Sciences Memorial University to the SD-OCT imaging laboratory (Faculty of Medicine Memorial University) in a cooler box with fresh oxygenated tank seawater, acclimated for 30 minutes, and then lightly sedated with 40 mg/l tricaine methanesulfonate (Sigma, Oakville, ON) in tank water. For imaging, the mice were lightly sedated with 100mg/kg ketamine and 10 mg/kg xylazine, and tropicamide was applied to their eyes to dilate their pupils. Tear Gel (Alcon, Mississauga, ON) was applied to keep the external eye surfaces of the mice and the fish moist. The fish gill areas were wrapped in gauze soaked with tank water and fish were placed in a silicone cradle with the imaged eye facing toward the SD-OCT bore lens (90-BORE-G4-M G4 mouse lens).
En face SD-OCT images were acquired from the live anesthetized animals using a Leica Envisu R2210 SD-OCT instrument (Bioptigen, Durham, NC). Scans were acquired in a rectangular volume (1 × 1000 × 100, 1.8mm × 1.8mm mode). The OD and OS eyes of each animal were scanned. Zebrafish SD-OCT scans (rectangular volume, 1 × 1000 × 100 mode) were kindly provided by Dr. J. Vance (The Spective Group, Raleigh-Durham, NC). One pixel corresponded to 1.62 µm. SD-OCT B-scans were manually analyzed using the OpenLap software ruler tool. Specifically, 10 retinal thickness measurements were captured for each selected B-scan image.
OpenCV was used to build an image segmentation program for analyzing and interpreting SD-OCT B-scan images. The thickness characteristics of the retina of lumpfish were determined by conducting a segmentation analysis which separated the retinal layers into 3 groups (the nerve fiber layer [NFL] with the ganglion cell layer [GCL], the inner plexiform layer [IPL] with the inner nuclear layer [INL], with the outer plexiform layer [OPL], with the outer nuclear layer [ONL] and with the inner photoreceptor segments [IS]; and the outer photoreceptor segments [OS] with the retinal pigment epithelium [RPE]) and which processed the data. The algorithm and code are available via a GitHub repository. The images were processed in three phases, with the result of the previous phase serving as the input for the next one. First, the images were loaded and refined by excluding image artifacts and eliminating as much noise as possible using non-local means denoising, followed by a Gaussian blur, a Median blur, and truncated thresholding. These processes helped reduce possible retina identification errors and refined each image so that the program could correctly identify the lumpfish retinal layers. Second, the refined images were processed using a novel algorithm to find the retina located in the input image. Third, the data was analyzed and outputted.
The segmentation function of the algorithm was based on thresholding, which uses the red, green, blue (RGB) pixel values contained in an image when the threshold values are given as user input. The thresholding was accomplished by accepting any value that met a certain requirement and rejecting all others. Three main variables were used to determine the location of the retina and its layers. The first variable represented the minimum acceptable pixel value of white, the second variable corresponded to the minimum gap between two white pixels, and the final variable was the maximum allowed gap between two white pixels.
The algorithm operated by assessing each column of pixels and storing the location of the white values that passed the thresholding function. The first and last threshold values of a column of pixels were highlighted in blue to delimit the total thickness of the retina (Figure 1). The program was designed to identify a large gap between two white pixels based on the second and third thresholding variables. The last value before the gap and the first value after the gap were highlighted in green. The middle of these two points, deemed the smoothing line, was highlighted in red and saved as a separate output; this line served as an additional evaluation tool for revealing the retinal curvature. Thickness calculations were taken directly from thickness measurements, which interpreted the data in the highlighted columns of pixels in each image. Only the highlighted values in each image were measured; all sections that failed to meet one or more of the thresholding values used in the algorithm were excluded.
The program can be adjusted by modifying one or more of the thresholding variables to suit the input image set. The optimization of the thresholding settings can be tested on an individual B-scan image before the segmentation of complete SD-OCT datasets is initiated. The number of measurements made by the algorithm is determined by the number of columns of pixels found by the thresholding algorithm contained in the retina; the maximum number is based on the width of the image in pixels. Once all measurements are made, the image is saved, storing the positions of all measured points.
To prevent possible errors, the program was designed to complete a functional analysis to determine whether the retina is missing from the image or if the image contains a large amount of noise or artifacts. This is performed by determining whether the image contains the correct number of pixels with a certain intensity above a threshold. In these cases, the images will be rejected, and they will not be processed by the image processing algorithm. Another feature to prevent false measurements is a minimum measurement thresholding variable, which excludes measurements created when the program chooses the same point or points that are too close together by preventing those measurements from entering the dataset. In addition, the user can delimit the height and the width of the image to be processed to avoid large artifacts. This feature can also be used to avoid fields of speckle noise, thereby improving the overall quality of the image segmentation.
The program’s data analysis component was designed to process the stored image dataset into a finished data output that facilitates data reproducibility by reporting the program’s internal settings. In the present study, the B-scan sequence numbers were recorder at the beginning of each data entry. The data output consisted of the mean retinal thickness for each B-scan image with the number of points measured for each retinal layer (Appendix 1). Using these measurements, the thickness of the segmented layers of each SD-OCT scan was reported as a percentage of the thickness to the whole retina.
The SD-OCT data produced by the image segmentation program was interpreted by a heat map program designed to represent the relative thickness of the whole retina B-scan. Specifically, all thickness values (in micrometers) of a complete set of B-scan images from a specimen were analyzed, and each measurement was assigned a hue on a color scale. The minimum thickness values were represented by blue, and the scale gradually increased in thickness, as represented by green, yellow, orange, and, finally, red, which represented the maximum thickness values. Each color ranged from a grade of 1 to 11 to create a color gradient representing the program processes of the whole dataset of measurements taken from the total retina thickness for each SD-OCT B-scan. The ends of the scale were marked by their respective values in micrometers placed above the color gradient. To avoid errors, any value that did not exactly match the color of the gradient was assigned to the nearest color of the range.
In addition to a 2D heat map, a 3D representation of the heat map data was constructed based on a 3D surface map that used the same data and color range as the original 2D representation. The 3D surface was constructed using a triangular surface mesh that displayed the outer surface of the retina, giving the user a better perspective on the surface and shape of the retina in the vicinity of the optic nerve head.
Two-tailed Student’s t-tests were used to analyze the statistical significance of the difference between the measurements of retinal thickness of B-scans assessed manually versus those assessed by algorithm processing. The measurements were considered statistically different if the p-values were equal to or less than 0.05.
Previous studies have used SD-OCT instrumentation in combination with original manufacturer equipment (OME) software to analyze teleost retinal parameters, such as the retinal radius or retinal layers [15,16]. However, this paper contains the first description of a custom algorithm designed to automate the SD-OCT data analysis in order to measure retinal architecture in marine teleosts. In the present study, we designed an image segmentation program to scan and interpret SD-OCT retinal B-scan images to determine the overall thicknesses of the retina and its sublayers in teleosts. Figure 1A and Figure 2A contain representative images of SD-OCT retinal B-scans of a juvenile cultured lumpfish (100 dph) and a zebrafish, respectively. The image segmentation program identified the following three layers of the lumpfish retina: the NFL/ GCL; the IPL, INL, OPL, and ONL with IS; and the OS with RPE (Figure 1B,C). The automatic segmentation of the zebrafish retina also identified three layers: the NFL-GCL; the IPL and INL; and the OPL with the ONL-IS, OS, and RPE (Figure 2B,C). The program also delimited the middle point of the retinal layers within the green lines. This smoothing line, depicted as a red line in Figure 1D, facilitated visualization of the retinal curvature. Although the program can segment specific clusters of retinal layers, some sublaminar layers were not segmented in the present study, as indicated in Figure 1 and Figure 2.
We designed the program to output the retinal layer thicknesses of each SD-OCT B-scan in the form of a data file in Excel or comma-separated values (CSV) format. Appendix 1 presents the data for lumpfish that formed the basis for the thickness measurement example in Figure 1C. The data file contained the retinal layer thickness measurements, the number of measurements or readings per B-scan, and internal settings associated with dataset processing. In addition, the program output included the thickness of the segmented retinal layers of each SD-OCT scan as a percentage of the thickness of the whole retina. As shown in the table, the retinal thickness measurement analysis of the representative juvenile lumpfish eye revealed that NFL/GCL was 14.9% of the volume of the retina, the inner layers with the outer layers and IPL-INL-OPL-ONL-IS volume percentage were 61.9%, and OS/RPE was 23.3%.
The program includes error prevention measures to ensure data accuracy and thresholding variables to enable adaptability to any SD-OCT dataset. Since thresholding is used, any input produces the same output when all settings remain the same. As with any scientific process, error mitigation is important. To avoid possible retina interpretation errors, the program has features such as adjustable thresholding methods, which can exclude information from detection and placement in the data. In addition, thresholding variables and functional analysis can prevent the appearance of a significant errors in the dataset. As the program makes hundreds of measurements per B-scan, an acceptable number of errors can occur without affecting the results. In contrast, when manual analysis is used to examine 10 measurements per B-scan (Figure 1E), a single error can greatly affect the results.
The program was designed as a replacement for the manual analysis of SD-OCT B-scan data of the teleost retina. The present study’s findings indicated that this algorithm is a reliable replacement for manual measurement analysis of lumpfish B-scans (Figure 3).
The algorithm analysis results were equivalent to the manual measurements in the corresponding zebrafish B-scan images (Figure 4). Since the algorithm processed information at a faster rate than the manual analysis, more specimens can be included in an individual study. Having the capacity to process more data and take more measurements per image combined with the capacity to perform data analysis increases the amount of reliable and repeatable SD-OCT data produced from a given set of B-scan images as well as the power of the analysis. Moreover, the turnaround time between the production of data and the results is greatly reduced when using the algorithm instead of manual analysis.
Additional program features include the generation of 2D and 3D heat maps representing the relative thickness of the retina throughout the whole SD-OCT retinal scan from individual B-scans. A representative 2D heat map of the whole SD-OCT retinal scan of a juvenile cultured lumpfish (100 dph) is shown in Figure 5, and a 3D rendition is shown in Figure 6. The heat maps include the specific specimen identifier, the number of B-scans processed, and a color gradient display that corresponds to different retinal thickness levels. Such heat maps are particularly useful for rapid visual identification of areas of the retina with different thickness levels. Heat maps are also useful for identifying retinal architecture and the position of the optic nerve head, as depicted in the top right section of Figure 5. The overlay of the heat map with the volume intensity projection (en face image) generated by the SD-OCT instrument demonstrated how the shape of the retina displayed in the heat map corresponded to the volume presented in the intensity projection of (Figure 5). The overlay illustrates the heat map’s relevance in representing the shape of the retina through visualization of its thickness characteristics corresponding to the anatomy seen in the en face image.
In a previous study, we used the OME 3D rendering software available on the Leica Envisu R2210 SD-OCT instrument (Bioptigen, Durham, NC) and found novel anatomic shape characteristics in the lumpfish retina [5]. In another study, we used a custom built in vivo ophthalmoscopy image capture methodology and found that the inner retinal surface of the lumpfish has distinctive streaks [17]. The present study marks the first time we have had the ability to perform 3D thickness heat mapping. Moreover, the 2D and 3D heat maps generated by the algorithm in this study further confirm the rather abrupt changes in thickness in the lumpfish retina, which we have observed histologically [5] and which are not typically observed in mammalian retinae.
The peripheral areas of the lumpfish retina are relatively thin, while the more central areas abruptly thicken in regions closer to the optic nerve tract. These characteristics prompt questions about the biologic significance of these types of variations in retinal morphology in live lumpfish and how these features might be related to the distinctive streaks in lumpfish retina that we have previously described [17]. Moreover, they provide the impetus for comparative assessments of dynamic retinal shape and morphology in different marine teleosts.
The algorithm used in the present study can correctly and automatically process lumpfish and zebrafish SD-OCT images and identify the thicknesses of the segmented retinal layers. In addition, 2D and 3D heat mapping could be useful for in vivo comparative biology assessments of teleost retinal anatomy. These heat maps are especially relevant because, until now, only manual SD-OCT image analysis has been used to produce retinal anatomy imaging data.
The core concepts of the algorithm can be expanded to analyze a different number of retinal layers, enabling its use in examinations of other teleosts or even other species. This algorithm is available to the retinal research community to facilitate basic discovery science on the biology of the retina. Huckenpahler et al. recently described the use of OCT to assess the degeneration and regeneration of specific cone photoreceptors in zebrafish and the visible detection of multiple sublaminae on OCT images of zebrafish retinae [18]. Although the lumpfish retina might present species-specific challenges because it is larger and more structurally complex than the zebrafish retina, the image segmentation program could be expanded to enable the identification of discrete bands of retinal sublaminae in lumpfish and other teleosts, leading to better imaging and understanding of the structure and cellular connectivity of the retinae of teleosts.
Iterative approaches to achieve a more detailed subdivision of the retinal layers [18] could involve extracting and isolating the sublaminae followed by running a separate segmentation of the extracted layers. Alternatively, we could use graph-based segmentation, in accordance with (Automated Segmentation of Retinal Optical Coherence Tomography Images-UWSpace) or adapt the combined graph-based segmentation and machine learning approach described in [19]. However, such approaches might be more complicated if different programs need to be produced for different species. Regardless, the development of this technology could be aided by SD-OCT technology expansion and improvement. For example, the ability to produce higher resolution images would allow more retinal anatomy features to be distinguished using computer vision techniques.
Appendix 1. Retinal layer thickness measurements from SD-OCT retinal B-scans of a juvenile lumpfish.
We are grateful for the expert help from D. Boyce and staff at the Joe Brown Aquatic Research Facility of Memorial University. We also thank Dr. J. Vance (The Spective Group, Raleigh-Durham, NC) for zebrafish SD-OCT scan data. This work was supported in part by the Canada Foundation for Innovation John R. Evans Leaders Fund; Memorial University of Newfoundland Dean of Medicine Transition Funds, School of Graduate Studies, and Medical Research Foundation grant; and the Ocean Frontier Institute grant Vitamin research Fund.