Stitching and Tiling Image Sets
Digital image processing allows us to see immense amounts of detail within an image. Intensity, spatial and morphometric details are easily captured and evaluated nowadays thanks to easily affordable, highly sensitive and very precise imaging systems. Coupled with image analysis software programs, extracting quantifiable information is a snap.
But it is the nature of research to push technology and it is no longer enough to collect and evaluate data from a single image when multiple images tell a bigger story. Creating a large composite image from several individual acquisitions allows us to see, literally, the bigger picture. Increasingly, it’s the bigger picture we are interested in evaluating. Fortunately, hardware and software technologies converge perfectly to provide the basis for creating perfectly aligned tiled and stitched images.
Fig. 1- Biopsy of transplanted human kidney. Sections cut from archival paraffin embedded tissue blocks stained with hemotoxylin and eosin. Images captured with an Aperio ScanScope (Aperio Technologies, Vista CA). Courtesy Tibor Nadasdy MD, Department of Pathology, The Ohio State University Medical Center.Image tiling crosses a number of disciplines including satellite imagery, radiology and microscopy. The field of microscopy is where we’ll spend the majority of the discussion, but the same techniques may be applied elsewhere. The software tools described here are found in Media Cybernetics’ Image-Pro Plus version 5.0 and higher software.
Collecting an Image Set
Tiled images are especially useful when a specimen is larger than the microscope or camera field of view. Excellent examples are plant and animal tissue, as well as metallurgical samples. Although changing magnification to increase the field of view can help, in the end we’ll lose resolution which, in turn, may hide or obscure important information. To overcome the tradeoff between magnification and field of view when acquiring image tiles, a scanning stage can be used. These stages are extremely useful in repeating a calibrated step-size with the step between adjacent fields being closely matched to the microscope’s field of view. Matching this step size is easily accomplished by using a stage micrometer to calibrate the camera and microscope objective. If you know the physical size of the field of view, it then becomes very easy to reproduce step sizes with an automated stage. Synchronizing image capture, stage movement and the order in which the individual planes are acquired gives you all the necessary information required to construct a composite image.
Putting the Pieces Together
Once you have acquired the image planes it is time to assemble them. To accomplish this, Image-Pro Plus uses two different algorithms.
User Defined (Manual) - This method allows the user to input X and Y offsets between adjacent tiles in the horizontal and vertical direction, assuming that the images have been collected in a regular fashion.
Fourier (Automatic) - This method uses image cross correlation, also known as ‘Fourier Correlation’ to determine how each pair of images in the tiling best match up. To understand how image correlation works, imagine sliding one image over its adjacent images and multiplying the images together on a pixel by pixel basis. The best match occurs when the sum of the multiplied images is at a peak. As an example, imagine two images with a background intensity of 10, and a single pixel peak intensity of 50. If the peaks don’t match up, the contribution of the peaks is (10*50+10*50) = 1000. If the peaks do match, the contribution including the remaining backgrounds is (10*10+50*50) = 2600. The higher number represents the better match between the images.
Image-Pro Plus supports full Fourier correlation as well as a “phase only” Fourier correlation. The phase method matches just the edges in adjacent images, rather than their intensities. Phase only correlation may work better for some brightfield images, or in images where there are large background variations that match each other more strongly than the objects you would like to align.
Either tiling method will generate a set of positions for the tiles and their overlaps which are then used to put the images together into a mosaic.
Both alignment methods make use of three techniques to handle the overlap of the tiles – Overlap in order, Trim to fit and Gradient Blend.
Overlap in order (None) - Just as the name implies, tiles are overlapped from upper left to lower right, with later images overlapping previous ones.
Trim to Fit - This cuts the image pairs at a point halfway through their overlap, butting the edges together. In effect, a line is calculated to lie halfway between adjacent images, and the input tiles are cut and joined along those lines. If there are multiple images, the mosaic pixels are taken from the tile with the closest center, allowing 4-way splits.
Auto vs. Manual- Which is Best?
Manual tiling is useful when you are trying to get an impression of what the overall tiled image will look like, or when highly precise stage movements make it unnecessary to align and stitch edges.
Auto tiling is very useful when you’re not sure of the accuracy of stage movement, or when blending edges is a must for publications or reports. Also useful when you have a minimal amount of image detail to work with.
This is probably the best method to use when checking for tiling errors due to acquisition settings or lack of image detail.
Fig. 2- Two areas of interest extracted from the lower right of Fig. 1
Fig 3. Stitched areas of interest from Fig 2. The region outlined in black is shown in magnified view in Figures 4 a and b.
Fig 4. In the Gradient Blend example (a), smooth transitions and no evidence of overlap between adjacent edges are evident. In (b) no transitions or alignment are used, resulting in a mismatch between edges.
Gradient Blend - The most computationally intensive method, gradient blending calculates a smooth weighted blend from one side of the overlapping tiles to the other. The effect reduces issues like varying background intensities and provides the smoothest edge transitions between tiles.
An Example of Gradient Blending
Two images are horizontally tiled and the overlap region is calculated. On the left side of the overlap the weighting is 100% to the pixels from the left tile, while on the right, the weighting is 100% from the right tile. The weighting shifts in a linear fashion so that at the midpoint the mosaic pixels are a 50-50% blend of the left and right tiles. When more than two tiles cross, the mosaic is calculated from a weighted blend of all overlaps, with the percentages distributed according to how far from the edge of each tile the overlap occurs.
A Survey of Iage Registration Techniques.
A.G. Brown: ACM Computing Surveys. Vol.24, pp.326-276. 1992.
C. Kuglin and D. Hines. "The Phase Correlation Image Alignment Method," Proc. Int. Conf. Cybernetics and Society. pp. 163-165. 1975.
E. Kang, I. Cohen, G. Medioni "A Graph-Based Global Registration for 2D Mosaics, 15th International Conference on Pattern Recognition(ICPR). Barcelona, Spain. 2000.
R. Raskar, J. van Baar, J.X. Chai.
"A Low-Cost Projector Mosaic with Fast Registration." Mitsubishi Electric Research Laboratories, Cambridge MA, USA. ACCV2002: The 5th Asian Conference on Computer Vision, 23-25, Melbourne, Australia. January 2002.
M. McGuire. “An Image Registration Technique for Recovering Rotation, Scale and Translation Techniques”. NEC Tech Report, Feb. 1998.