A reconfigurable real-time morphological system for augmented vision

Abstract

There is a significant number of visually impaired individuals who suffer sensitivity loss to high spatial frequencies, for whom current optical devices are limited in degree of visual aid and practical application. Digital image and video processing offers a variety of effective visual enhancement methods that can be utilised to obtain a practical augmented vision head-mounted display device. The high spatial frequencies of an image can be extracted by edge detection techniques and overlaid on top of the original image to improve visual perception among the visually impaired. Augmented visual aid devices require highly user-customisable algorithm designs for subjective configuration per task, where current digital image processing visual aids offer very little user-configurable options. This paper presents a highly user-reconfigurable morphological edge enhancement system on field-programmable gate array, where the morphological, internal and external edge gradients can be selected from the presented architecture with specified edge thickness and magnitude. In addition, the morphology architecture supports reconfigurable shape structuring elements and configurable morphological operations. The proposed morphology-based visual enhancement system introduces a high degree of user flexibility in addition to meeting real-time constraints capable of obtaining 93 fps for high-definition image resolution.

1. Introduction

Studies indicate that there are a globally estimated 124 million individuals who suffer from significantly reduced vision that cannot be considerably improved using conventional corrective devices [1]. Visual impairments differ per individual and can be caused by a wide range of visual disorders such as age-related macular degeneration, glaucoma or cataracts, each with varying degrees of severity. Individuals who suffer visual sensitivity loss from visual disorders are often referred to as being visually impaired or low-vision individuals, as they often experience some level of vision. The characteristics of the visual sensitivity deficit are dependent on the disorder and can range between central visual acuity loss and peripheral visual field deficit (e.g. tunnel vision). Many visually impaired individuals experience a loss of visual perception to high spatial frequencies resulting in blurred vision [2].

These low-vision individuals struggle with important basic daily tasks such as reading, facial recognition, object detection and general mobility [3]. Traditional visual aid devices typically utilise optical magnification to counteract the reduced visual clarity, i.e. blurred vision, by increasing the objects' perceived visual size, which can be useful. However, the associated magnification side effects of spatial misrepresentation and information sampling limit the application and degree of aid provided [4]. Recent research utilising traditional optical methods produced an optoelectronic device consisting of cascaded modulating prisms to replicate and enhance microsaccadic eye jitter movement among spatial frequency-limited visually impaired individuals; however, this device is relatively large and heavy, limiting its widespread application and general use [5].

Advances in embedded devices throughput and power efficiency enable the development of novel real-time augmented vision, consisting of a relatively small, lightweight and wearable head-mounted display (HMD) embedded device to aid the visually impaired. The HMD visual aid device should implement configurable algorithm parameters to enable subjective user customisation per task. Efficient algorithm design allows real-time operation in low-cost devices with longer operating times on a portable power supply while reducing power consumption and minimising heat dissipation.

A proposed conceptual HMD visual aid system implemented on a field-programmable gate array (FPGA) is presented in Figure 1, demonstrating an edge overlay approach to enhance visual perception among low-vision individuals who perceive the world in low spatial frequencies, represented using a Gaussian low pass filter applied to perceived images. The proposed system camera captures real-world image scenes with a typical field of view, processed by an FPGA to provide edge augmentation for presentation on an HMD in real-time. The image-extracted edges are obtained by evaluating two morphological operations [6] in parallel. Overlaying the gradient components onto the original image augments the perceived scene boundaries and features for visually impaired individuals. The algorithm architecture is configurable through the use of a soft microprocessor, allowing flexible customisation as required per task suitability and subjective preference, for the edge-enhanced technique demonstrated the user could change various parameters such as edge colour and strength. The conceptual hardware system will be suitable for implementation within other various applications, such as a television enhancement module [7].

Figure 1

Conceptual overview of a morphological high spatial frequency augmented vision system on FPGA for low-vision aid.

The contributions of the work presented in this paper describe a detailed morphology [6] FPGA architecture, implementing reconfigurable shape structuring elements (SE) and efficient morphology operators capable of obtaining dilation and erosion from the same function. The FPGA design can be reconfigured during operation, obtaining a large degree of flexibility. Applying the presented architecture for augmented vision obtains a highly flexible edge overlay visual enhancement system capable of obtaining multiple morphology edge gradient types with user-tunable edge size and magnitude while meeting real-time constraints for video graphics array (VGA), extended graphics array (XGA) and high-definition (HD) resolutions.

2. Previous work on visual enhancement and embedded devices

Currently, the most notable contributions in augmented vision for the visually impaired and development of visual enhancement embedded devices are reported by Peli et al. [8–13] and Atabany et al. [14, 15]. However, the work presented on embedded devices are severely limited in hardware technical details while opting to focus more on psychophysical testing of fixed visual enhancement algorithms.

2.1 Visual enhancement algorithms

Visual image enhancement algorithms predominantly focus on enhancing high spatial frequency components for augmented low-vision aid. The high spatial frequency components of an image are where sudden large changes in pixel value occur over a short period, such as an objects edges or fine detail within an image. This has led to edge detection, a common digital image processing technique, being a critical underlying process in high spatial frequency enhancement. Peli et al. originally proposed augmented vision models utilising visual multiplexing techniques such as the wideband enhancement algorithm that superimposed modified image edges over the original image, enhancing the high spatial frequency components [8, 9]. Further work includes modifying the discrete cosine transform to increase the mid-range spatial frequencies of JPEG images [10], which was then extended to include MPEG video enhancement [11] and recently a modified edge detection algorithm for contour shape enhancement applied to image luminance channels [12]. Wolffsohn et al. demonstrated a significant increase of perceived image quality of television video amongst visually impaired individuals by recording television segments onto a computer with user-selectable software overlaid coloured edges on the original scene [7]. Atabany et al. developed a robust scene edge enhancement algorithm that focused on efficiently blurring the original image with anisotropic smoothing to obtain edge enhancement of only the major image edge boundaries while effectively ignoring minor detail edge occurrences [14]. In the previous literature binary edge detection algorithms, e.g. Sobel and Canny edge detection, are extensively used for spatial frequency enhancement. Gibson et al. demonstrated a significant subjective preference in perceived image quality among simulated low-vision subjects with a gradient-based statistical edge detection algorithm applied to real-world scenes, where major object boundaries have a high magnitude level and more subtle edges, common of fine detail, are highlighted with lower magnitude edges [16].

There are other visual enhancement techniques investigated that do not utilise the standard spatial frequency overlay approach. Atabany et al. utilised the proposed robust scene enhancement algorithm to obtain a tinted reduced outline nature enhancement technique which offers several advantages over typical methods such as maintaining edge chromatic information but at a loss of the overall scene visual information [15]. Other approaches of note are efficiently quantizing image colours to reduce detail and fine texture by techniques such as image abstraction [17] for visual enhancement purposes demonstrated by Atabany et al. [15].

2.2 Embedded device implementation

Peli et al. further developed the proposed visual multiplexing techniques for a practical tunnel vision HMD visual aid device where the detected edges from a normal field of view are downscaled onto the patient's limited field of view and are presented on a see-through display. This approach allows patients suffering from tunnel vision to view objects outside their inhibited view and as a result decreased patient visual search time by up to 75% [13]. The device limitations allow only the downscaled edge presentation area to be configured to a specified individual's limited field of view, while the algorithm itself contains no configurable options. Saha et al. developed a low-vision aid device implementing Laplacian edge detection and contrast modification on a Texas Instruments 6446 DSP (Dallas, TX, USA) within real-time constraints for augmented television with prospects for use in an indoor assistive HMD [18]. The design implemented configurable options such as modifying edge threshold value and the ability to switch between edge and contrast enhancement algorithm, while the algorithms lacked device optimisation and customisable parameters such as edge magnitude. Ros et al. proposed a platform for an HMD visual aid device implementing contrast adjustment, interpolated digital zoom and Sobel edge detection for rescaling edges onto a limited field of view [19]. These algorithms were implemented independently on separate FPGA models with different algorithm architecture. The device models implemented minimal flexibility, offering edge threshold configuration only; the algorithms were mapped directly to FPGA and would require resynthesising the device for any single parameter change. Gibson et al. optimised the mathematically complex statistical gradient edge detection algorithm obtaining significant improvements for throughput, latency and slice usage for a Virtex-5 implementation meeting real-time constraints with a maximum frequency of 292.8 MHz [20]. The statistical algorithm produced a significant improvement of perceived image quality over binary-based edge detectors among the visually impaired but was a fixed implementation with no controllable parameters.

The current implementations of visual aid devices within the literature are extremely rigid and limited in terms of design, architecture reconfigurability and user customisation and have very little documented algorithm optimisation for hardware constraints. It is important to utilise an image processing technique flexible enough to produce enhancements with various controllable parameters to maximise user configurability of a device. Nonlinear mathematical morphology [6] is capable of implementing differing image processing responses and techniques in systems composed of morphological operations, offering greater flexibility than current algorithms utilised for augmented visual aid.

3. Morphology review

Morphological image operators evaluate binary, grayscale and colour images, performing a wide range of algorithms such as edge detection, noise removal, image segmentation and corner detection [6, 21, 22]. Morphological processing techniques produce signal and image modifications based on evaluating neighbourhood samples over a predetermined matrix referred to as an SE, which can either be of uniform (flat) or non-uniform (non-flat) nature in grayscale morphology.

Current research in morphological image processing has demonstrated various processing techniques, many which have advantages over equivalent typical image processing techniques. Mahmoud and Marshall applied an edge-guided morphological filter to medical images [23], demonstrating a superior response over other sharpening filter methods with respect to noise removal, edge sharpening and restoring fine image detail [24].

To the authors' knowledge, there are no current applications of the morphological gradient in augmented vision, while hardware implementations have reported a directly mapped realisation on FPGA [25] and proposed a morphological Canny edge model [26].

In the current literature on morphological implementation, authors focus on optimising the pipeline delay process for a latency efficient morphology operator implementation [25, 27–31], while recent notable contributions by Bartovsky et al. instead proposed optimising the entire morphological filter operator chain [32]. These implementations contain some degree of flexibility in SE configuration, such as a scalable rectangular SE [30]. However, these implementations are limited with regard to hardware realisation and efficiency as they cannot implement fully customisable shaped SEs and it is common practice for morphology dilation and erosion to occur in separate entities rather than use a single reconfigurable morphological operator [30, 33].

4. Morphology image operations

All morphological processing techniques are composed of two base morphology operations - dilation and erosion. In binary image morphology, dilation increases forefront object boundaries, while erosion increases background object boundaries. Dilation and erosion functions are dual operators, where dilating background objects is the same as eroding foreground objects.

Figure 2 demonstrates morphology operations to convey morphological dilation and erosion functions on a test image consisting of a white letter ‘j’ on a black background (Figure 2a). The dilation operator expands the forefront letter boundary, enlarging the letter (Figure 2b). Similarly erosion increasing the background boundaries: reducing letter size (Figure 2c).

Figure 2

Morphological operations. (a) Original image; (b) dilated image and (c) eroded image.

The dilation function extended to grayscale is shown in Equation 1, where the dilation (δ) of an image (f) changes the pixel value to the maximum value within its neighbourhood, determined by a flat SE defined as B. Similarly, the morphological grayscale erosion (ϵ) operator shown in Equation 2 transforms the current pixel value to the minimum value determined within the current SE neighbourhood.M

$$\left[{\delta }_B\left(f\right)\right]\left(x,y\right)=\underset{\left(s,t\right)\in B}{\max }\left[f\left(x-s,y-t\right)\right]$$

(1)

$$\left[{ϵ}_B\left(f\right)\right]\left(x,y\right)=\underset{\left(s,t\right)\in B}{\min }\left[f\left(x+s,y+t\right)\right]$$

(2)

The SE is a defined shaped structure that performs morphology operations on an input image. The SE scans the input image where the SE centre matches the input image pixel; all pixels within SE shape dimensions are sampled and morphologically evaluated in relation to each other to determine the output pixel.

Morphology dilation and erosion operations are used in cascaded networks to provide various image processing effects. Exploiting the dilation and erosion of an image during an intensity change can produce several edge detection methods. Dilation thickens regions in an image while erosion shrinks regions and when subtracted produce a boundary emphasis. The standard morphological gradient (G _m) is the full difference between the dilated and eroded image demonstrated by Equation 3.

$$G_{\mathrm{m}}\left(f\right)={\delta }_B\left(f\right)-{ϵ}_B\left(f\right)$$

(3)

The difference between the original image to the eroded and dilated components obtains two other half edge types; the internal (G _i) and external (G _e) half edge gradients, as demonstrated in Equations 4 and 5.

$$G_{\mathrm{i}}\left(f\right)=f-{ϵ}_B\left(f\right)$$

(4)

$$G_{\mathrm{e}}\left(f\right)={\delta }_B\left(f\right)-f$$

(5)

The grayscale morphological, external and internal gradient responses for a standard cameraman test image evaluated with a flat 3 × 3 SE are presented in Figure 3. The gradient images presented contain differing extracted edge gradients.

Figure 3

Morphological gradients of cameraman test image. (a) Original image, (f); (b) morphological gradient, (G _m(f)); (c) external gradient, (G _e(f)) and (d) internal gradient, (G _i(f)).

5. Visual edge enhancement system

The image-extracted edges will be overlaid on top of the original image for visual enhancement where parameters such as edge colour are determined by user configuration for subjective preference and task suitability. It is possible with morphology to further enhance extracted edges by transforming them to be more visually dominant or subtle through dilation or erosion with a cascaded morphological operator to thicken or thin the image edges. However, modifying the SE sample size performs the dilation or erosion operations over a changed pixel neighbourhood, i.e. implementing larger SE dimensions for the dilation and erosion block prior to edge calculation will produce a gradient difference over a larger neighbourhood without the requirement for a further morphological operation to dilate the extracted gradient. Figure 4 demonstrates the effect of SE dimension change has on the extracted morphological gradient (Figure 4b,c,d) and its associated image overlay for various SE mask sizes (Figure 4e,f,g) on a zoomed in standard Lena test image (Figure 4a); it can be observed that SE size increase directly results in a thicker extracted image edge component.

Figure 4

Morphological edge detection and image overlay for various structuring element sizes. (a) Original image; (b) 3 × 3, (c) 5 × 5 and (d) 7 × 7 edge gradient images; (e) 3 × 3, (f) 5 × 5 and (g) 7 × 7 edge overlay images.

Vision is a highly subjective parameter, where augmented vision systems must be flexible to produce various responses with flexible parameters. A hardware system capable of implementing reconfigurable morphological SE operations with a flexible gradient calculation can obtain various morphological gradients each of differing thickness and magnitude, which would be advantageous over current fixed algorithm designs implemented on embedded devices [11–13, 18–20].

6. Morphological architecture

The proposed reconfigurable morphological edge detection architecture can be split into various sections - pixel sample processing, morphological and mathematical operations. Delay lines are used to buffer the input pixels and provide sample points for the SE, where each sample point is evaluated by a pixel sample processing structure and all resulting pixels are evaluated using a morphological operator chain to obtain the morphology obtained pixel value. Delay line architecture implementations have been extensively optimised for improved latency within current image processing literature [25, 27–31], the architecture presented here in contrast implements standard delay line architecture while instead opting for maximal flexibility and reconfigurable morphological functions. The architecture's flexibility is controlled using configurable registers programmed using a soft microprocessor, allowing design reconfiguration during system operation.

6.1. Pixel sample processing architecture

To implement reconfigurable flat and shape-based SEs, the pixel samples undergo pre-processing before the morphological process occurs, as illustrated in Figure 5. The pixel sample register determines if the pixel is directly sampled or ignored by producing a false value to be removed within the later occurring morphological operation. The pixel sample register is configurable to 1 or 0 for producing the sampled or false pixel value. The morphological operation register determines the overall SE morphology function and is used within the pixel processing hardware to obtain a suitable false value to be removed during the morphology process. If the SE sample is to be ignored, then a pixel value of either 0 or 255 will be produced depending on whether dilation or erosion occurs, assuming unsigned 8-bit integers used for luminance channel pixel representation, i.e. pixel range of 0 to 255. Dilation compares the SE samples and selects the largest value; hence, a false value of 0 will be selected for SE sample points that are not sampled, as the produced false value 0 will be removed during comparison statements with any value greater than 0. Similarly, erosion compares the sampled pixels within the SE and selects the smallest value; a produced false value of 255 will be removed during erosion.

Figure 5

Pixel point sample processing. Sampled pixel is passed through when pixel sample register is set to 1, set to 0 outputs a suitable false value determined using the morphological operation register.

6.2. Morphology operator architecture

The proposed architecture for the morphology operator is demonstrated in Figure 6. The morphological operator architecture exploits the duality property of morphology, where erosion and dilation processes are the dual of each other; hence, it is possible to obtain the eroded result from a dilation process. Binary morphology simply inverts the output value, while extension to grayscale requires switching the morphologically selected pixel with the other pixel evaluated. The morphology dilation function represented in hardware determines the largest pixel value of two input pixels (pixel samples A and B) for output at M3. The morphological operation register is a global SE register, configuring all morphology operators within an SE to perform dilation or erosion. The register controlled bus multiplexers (M1 and M2) when set to 1, routes pixels A and B into port 0 and 1 of M3, obtaining a dilated output pixel. When the register is set to 0, pixels B and A are switched into port 0 and 1 of M3, obtaining an eroded output pixel, i.e. the dual of dilation.

Figure 6

Morphology operator. Dilation output is obtained when the morphological operation register is set to 1, routing pixels A and B to ports 0 and 1 of M3, pixels A and B are re-routed producing an erosion output when the register is set to 0.

6.3. Morphology structuring element architecture

The proposed morphological SE consists of pixel sample processors, morphological operators, pixel sample delay (z ⁻¹) and line buffer structures as demonstrated for a 3 × 3 SE boundary in Figure 7a. The input pixel stream is buffered and delayed to provide the correct sample points for the 3 × 3 mask; each sample undergoes a pixel processing operation where it is enabled or disabled using the specified pixel sample register; the resulting output pixels, labelled P# in Figure 7a, are processed by a morphological chain operation. The previously described morphological operation register is a global register set and used within all pixel sample processing and morphological operator structures within independent SE morphology functions. The pixel sample registers determine the real sampled pixels when set to 1, while configured to 0 produces a false value to be removed as previously described. Figure 7b presents an example 3 × 3 ‘x’-shaped SE, and the required pixel sample register settings for the SE structure presented in Figure 7a.

Figure 7

Architecture of a 3 × 3 morphological ‘x’-shaped structuring element. (a) A 3 × 3 morphology architecture and (b) pixel sample register settings for ‘x’-shaped structuring element.

There are several advantages of the proposed architecture including implementation of flat reconfigurable SEs with any shape that fits within the maximum hardware realised SE boundaries. The SE flexibility can be used to avoid processing data outside the image borders, which can introduce inaccuracies and corrupt the output response. The SE can be explicitly controlled using a soft microprocessor to ignore pixels outside the image dimension boundaries or form any SE shape through generating false signals at specific SE points with the previously described pixel sample processing system.

6.4. Mathematical operator architecture

The mathematical operator evaluates the dilated, eroded and delayed original image signals to determine the morphological, external and internal edge gradients, as demonstrated in Figure 8.

Figure 8

Mathematical operator.

The various edge gradient calculations are determined by multiple subtraction functions operating in parallel, where the configurable gradient select register determines the selected edge gradient output from M1. The resultant edge gradient can then be scaled through multiplying with the user-controllable gradient magnitude register.

7. Hardware implementation

The developed flexible and reconfigurable architecture for morphological operations of various SE and gradient function including the associated pixel sample processing points were implemented on a Spartan-3 (XC3SD3400A) device (Xilinx Inc, San Jose, CA, USA), as shown in Figure 9, demonstrating the experimental set-up configuration. The Spartan-3 FPGA device utilises the proposed morphology architecture to extract the morphological gradient from the camera-captured VGA video frames and presents the result on a monitor, as shown in Figure 9a, where the camera captures a Lena test image print out and displays the associated morphological gradient. The architecture can be used to overlay images with the morphological gradient and display the augmented real-time video on an Oculus Rift Virtual Reality developer headset (Oculus VR Inc, Irvine, CA, USA), shown in Figure 9b, to provide a reconfigurable morphological gradient for an experimental real-time augmented vision system. The morphology architecture was developed within Xilinx System Generator and exported to Xilinx Embedded Developer Kit (XEDK), where the design was synthesised and realised onto the FPGA device for evaluation. The proposed architectures' reconfigurable registers are controlled using a Microblaze soft processor (Xilinx Inc, San Jose) implemented in XEDK, which contains various configurable pre-set options for morphology gradient enhancements and controls the camera-video link interface. The models were evaluated and synthesised for a Virtex-5 (XC5VSX95T) (Xilinx Inc, San Jose) device to allow valid comparison with other current morphological operator methods within literature.

Figure 9

FPGA experimental configuration. (a) The Spartan-3 presents the morphological gradient of the camera video frames onto the display monitor in real-time and (b) the augmented video can be displayed onto an Oculus Rift headset providing a real-time augmented vision experimental configuration.

7.1 Morphology operator implementation

The flexible morphology operator units are combined to form reconfigurable morphological operation-based SEs as previously described. The FPGA systems' dynamic power, latency and occupied slice resources increase, while maximum throughput in mega-pixels per second (MPx/s) decreases as SE dimension size increases with fixed image size, as shown in Figure 10. The proposed system obtains one pixel per clock, i.e. pixel throughput is equal to the systems frequency. Smaller SE is more efficient in device resources; however, it has limited capabilities of configuring various SE shapes. Increasing SE dimensions introduces the possibilities of more realisable shapes of various sizes, including the option to implement all realisable shapes of smaller SE within the larger SE, offering a larger degree of configuration options. All parameters follow an approximate linear trend, where the required occupied slice count and system latency have a significantly sharper gradient increase associated due to the increasing quantity of required pixel line buffers and operations for SE reconfiguration. Increasing SE size requires more morphological operators, pixel sample processors and mathematical operation modules resulting in a large increase of occupied FPGA slices, occupying 0.7%, 1.8%, 3.6% and 4.7% of overall FPGA area for 3 × 3, 5 × 5, 7 × 7 and 9 × 9 SE dimension sizes, respectively. The increasing occupied area requires more dynamic power to operate and system latency increases resulting in a throughput decrease.

Figure 10

Virtex-5 morphology operator throughput, dynamic power and occupied area versus degree of structuring element configurability. Image size 256 × 256.

Table 1 compares the proposed reconfigurable morphological operator with equivalent standard morphology operations and other current Virtex-5 morphological operator designs by Holzer et al. [25] and Bartovsky et al. [32] for 1,024 × 768 (XGA) frame resolution. Holzer et al. implemented morphology operations for a fixed diamond SE within a 7 × 7 mask, Bartovsy et al. implemented a morphology operator utilising SE composition for any realisable rectangle within a 31 × 31 mask, and the standard morphology dilation and erosion operators have been implemented with a 7 × 7 fixed square SE. The implemented standard morphological dilation and erosion operators obtained approximately the same resources and results due to the infimum and supremum functions switching as the only difference. The proposed reconfigurable SE morphology operation obtained approximately the same throughput pixel frequency as Holzer et al., while obtaining a 16% decrease and 113% increase in comparison with standard morphological operators and Bartovsky et al. The reconfigurable and fixed morphological operators obtained approximately the same execution latency, while the method proposed by Bartovsky et al. required a significant 709% more clock cycles. The proposed reconfigurable SE morphology operator produced approximately the same frame rate as Holzer et al., in addition to obtaining 429% faster and 16% slower frame rates than the morphology operator presented by Bartovsky et al. and our standard morphology implementations. The proposed reconfigurable implementation requires significantly more slice resources than Bartovsky et al. and Holzer et al., while utilising approximately the same number of slice registers as the standard morphology design; however, the reconfigurable design utilised more look-up tables (LUTs) due to the amount of signal routing and configurable parameters implemented.

Table 1 Morphological operator comparison of Virtex-5 resources for XGA frame size

Table 2 compares important morphology design defining characteristics of the proposed architecture on Virtex-5 technology with other proposed relevant morphology methods by Deforges et al. [29], Holzer et al. [25], Genovese et al. [34] and Bartovsky et al. [32]. The morphology designs are evaluated for flexibility in supporting reconfigurable shaped SE, addressing image boundary values, implementing efficient morphology operators capable of obtaining dilation and erosion from the same architecture through duality and the ability to process grayscale images.

Table 2 Morphological comparison of architecture design characteristics

Holzer et al. processed grayscale images with a diamond-shaped SE, while Genovese et al. processed binary images with a rectangular SE, where both implementations directly mapped a fixed morphological operation with no reconfigurable SE options. Deforge et al. and Bartovsky et al. demonstrate rectangular- and convex-shaped SE of arbitrary size, while Deforge et al. provide no documentation on the designs' programmable options after synthesis, and Bartovsky et al. support reprogrammable rectangular dimensions, however were limited to rectangular-shaped SE. The binary morphology proposed by Genovese et al. maps directly to digital logic gates implementing the dilation and a selectable bit inversion to obtain both dilation and erosion results from the same architecture, while in contrast, efficient grayscale morphology duality architecture has not been attempted. Only Deforge et al. addressed sampling outside the image boundary with introduction of an extra memory line to flush the pipeline when required.

The proposed architecture can process grayscale images with highly reconfigurable SE capable of realising any shape obtainable within the maximum synthesised SE boundaries, which can be configured to ignore values outside the image boundary, where the design exploits morphology duality to obtain dilation and erosion from the same architecture. In addition, the proposed architecture utilises more resources to implement shape reconfigurable SE while obtaining lower latency and higher frame rates than other configurable SE morphological methods.

7.2 Morphological gradient implementation

The proposed reconfigurable morphological gradient architecture is presented in Figure 11. A standard block random access memory (BRAM) delay line architecture produces the 7 × 7 pixel sample points to drive two previously described morphological operators implemented in parallel. The morphology modified and delayed original signals are cascaded into a mathematical operator to determine the edge type. The morphological operators are set to perform dilation and erosion in parallel with a maximum SE boundary of 7 × 7, allowing morphological gradient types to be obtained for any combination of flat programmable SE shape or size that can be realised within the SE boundaries, where both SE structures are defined independently. The mathematical operator performs the required calculations for the specified edge type and extracted edge magnitude which are determined using user-controllable registers. The Virtex-5 hardware implementation resources are presented in Table 3 for VGA, HD resolutions and a specified XGA resolution for comparison with Holzer et al. [25]. The FPGA implementation meets real-time constraints for VGA, XGA and HD resolutions obtaining 705, 266 and 93 fps, respectively. The majority of hardware resources utilised by the proposed design are quasi-constant for all image resolutions with 4%, 2.5% and 7.7% of available slice registers, BRAM and occupied slices used, respectively. FPGA slice LUTs increase as image size increases utilising 5%, 6% and 6% of available slice LUTs for VGA, XGA and HD resolutions, respectively. Pixel throughput frequency decreases as latency and dynamic power increase with increasing frame size due to the associated processing time and power required with slice LUTs increasing. Holzer et al. directly mapped a morphological gradient with a fixed 7 × 7 diamond-shape SE evaluating XGA images on a Virtex-5 device, achieving a maximum throughput of 214 MHz. Comparison with the presented reconfigurable design demonstrates a throughput loss of 2% and larger FPGA resource usage. However, the proposed design in this paper offers a highly flexible method that can be reconfigured to obtain the morphological, internal and external gradients of various SE shapes and sizes within the maximum 7 × 7 SE boundary. The implemented flexibility utilises programmable registers to control each SE shape, increasing FPGA resource usage while making use of operations with minimal latency such as signal routing to obtain a low throughput loss of 2% compared to the fixed approach by Holzer et al. which would require resynthesis to change any single parameter.

Figure 11

Reconfigurable morphological gradient architecture.

Table 3 Virtex-5 resources of reconfigurable morphology gradient

8. Conclusions

Visually impaired individuals often suffer a loss from high spatial frequency information, while current traditional approaches are limited in effectiveness, practicality and application. A video processing HMD visual aid embedded device can enhance and provide augmented vision for the visually impaired, offering a degree of visual aid and practical use currently unobtainable with traditional visual aid technology. Morphological operations were applied to extract image edges for augmenting high spatial frequencies for the visually impaired, resulting in three possible edge gradient types. The proposed flexible design allows edge detection of morphological, internal and external gradient techniques to be obtained from within the same architecture, where the edge thickness can be modified by adjusting SE size and shape. While current trends in augmented vision for the visually impaired have extensively utilised binary-based edge overlaying systems, e.g. Sobel edge detection, with a rigid and fixed implementation [7–10, 13–15, 18–20], the proposed method is highly flexible, offering a high degree of real-time user customisation, where the user can switch between various morphology edge gradient techniques of selectable magnitude and edge thickness. The proposed grayscale morphological architecture exploits duality, capable of producing erosion and dilation functions from the same morphological operation architecture. The morphology operators allow current configurations to be changed during implementation, where SE can be reconfigured to any shape or size within the maximum synthesised SE boundaries, allowing pixels outside the image boundaries to be effectively ignored and obtain a large degree of system flexibility. Realisation of this level of reconfiguration requires more FPGA chip space and obtains a 2% decrease in throughput compared to a directly mapped morphological gradient on FPGA with fixed parameters [25]. The gradient edge detection system on a Virtex-5 device is capable of meeting real-time constraints for HD image and video applications, obtaining 705, 266 and 93 fps for VGA, XGA and HD image dimensions, respectively. The proposed edge enhancement device is capable of processing standard HMD resolution of VGA dimensions in real-time, offering a degree of user algorithm customisation and flexibility that has not been previously implemented in visual aid devices or morphological architecture.