Variational Computing Ba sed Segmentation Methods [602225]

Variational Computing Ba sed Segmentation Methods
for Medical Imaging by using CNN

Alexandru Gacsádi
Electronics Department
University of Oradea
Oradea, Romania
[anonimizat]

Péter Szolgay
Cellular Sensory and Wave Computing Laboratory
Computer and Automation Institute,
Hungarian Academy of Sciences
Affiliated also to
Pázmány Péter Catholic University
Budapest, Hungary
[anonimizat]

Abstract —The paper presents a new variational computing based
medical image segmentation method by using Cellular Neural
Networks (CNN). By implementing the proposed algorithm on FPGA (Field Programmable Gate Array) with an emulated digital CNN-UM (CNN-Universal Machine) there is the
possibility to meet the requirements for medical image
segmentation.
Keywords-medical imaging; segmentation; variational
computing; cellular neural networks;
I. INTRODUCTION
Segmentation of images by defining anatomical structures
and regions of interest have a crucial role in most medical
imaging applications, both in the phase of establishing the
diagnosis by locating pathology, and in planning and carrying
out appropriate treatment, such as for example, biopsy,
radiation therapy, and minimally invasive surgery. In this
respect automatic segmentation is a set of methods to create
using relevant images the specific anatomical model of the
patient. A typical situation for semiautomatic systems for aided
diagnosis involving labeling occurs when the image is
segmented into different regions and regions are subsequently
labeled as healthy tissue or a tumor. For this purpose may be
used, for example, Magnetic Resonance Imaging (MRI),
Computer Tomography (CT) or Positron Emission
Tomography (PET).
A variety of approaches have been developed to solve the
problem of images segmentation which is an important stage in
an automatic diagnosis system. First, for grey scale images, one may classify the segmentation methods into edge-based methods and region-based techniques. Region-growing methods can be made less sensitive to noise than simple edge-based or morphological methods, but they may become extremely computationally complex for even simple rules. On the other hand, curve evolution, active surfaces, statistical approaches, and variational energy methods have become popular approaches in this field. The majority of these methods
prove remarkable performances when the processed image
corresponds to the model of the algorithm but fails or gives significant artifacts otherwise [1]. The process used to perform image segmentation varies
greatly depending on specific application, imaging modality
and other factors. In practical terms it seems that a model should be selected according to its specific application. The optimal method of processing may depend on how was the input image generated namely, it is a CT or a MRI image.
Moreover, in CT images brain tissue segmentation has different
requirements from the segmentation of the liver. Overall image
artifacts such as noise and movement can also have significant
consequences on the choice of appropriate segmentation
algorithm. In addition, each imaging modality introduces its
own difficulties, which hinder the election and execution of the
optimal segmentation method.
The performances of segmentation techniques are difficult
to evaluate. Currently there isn’t a specific general method of segmentation to produce acceptable results for all types of medical images. Each of these methods have their advantages and disadvantages, as some algorithms optimized for a particular hardware structure can no longer work as well on another structure. However, some methods available for relatively large areas, which are optimized for specific applications, can often produce better results by taking into account previous knowledge. Therefore, selecting an optimal segmentation method for a concrete application can be a difficult issue, being a continuous dilemma, and by far it is not a classic algorithm and a classical filtering application, e.g. min/max algorithm.
Mathematical models are the foundation of biomedical
computing [2]. Based on those models data extracted from images continues to be a fundamental technique for achieving
scientific progress in biomedical research. It is extremely
important to notice that regardless of the mathematical
algorithm used for segmentation and the method of
implementation, assessing in whole their efficiency and utility,
that is validating the algorithm for daily medical practice,
results from an iterative loop process, where the radiologists
play very important role. However, there is a major need for
new mathematical techniques and possibilities of
implementation that will lead to more efficient methods that
can be integrated into the semiautomatic systems [2]. Certainly
2010 12th International Workshop on Cellular Nanoscale
Networks and their Applications (CNNA)
978-1-4244-6678-8/10/$26.00 ©2010 IEEE

the process of updating the medical imaging techniques should
occur as soon as new results are obtained in terms of theory and technological conditions (hardware and software) allow their implementation in practice. This objective, which is centered on human health, is achievable only in the conditions of dialogue and effective and permanent interdisciplinary collaboration of specialists from all fields of science involved.
This paper is organized as follows. In Section II are
summarized some current CNN based (Cellular Neural Networks) approaches that were used in the segmentation of particular classes of medical images. In Section III a new algorithm is proposed and described for segmentation of medical images based on variational computing. The results obtained by simulating the new method of image segmentation are presented in Section IV. Finally in section V the
conclusions are drawn on the proposed CNN procedure for CT
image segmentation.
II. CNN
BASED IMAGE SEGMENTATION
Cellular Neural Networks (CNN) proved to be very useful
regarding real-time image processing [3,4]. The reduction of computing time, due to parallel processing, can be obtained
only if the algorithm can be implemented on a processor array
[5,6,7,8]. In general, the CNN based implementation of medical imaging methods is not a purpose in itself. In this way computing time can be reduced because of full parallel processing. For noise extraction, image segmentation, contour determination, various mathematical models were examined and proposed on the possibility of implementation on hardware structures based on cellular neural networks. Unfortunately, comparison the efficiency of CNN implementations of these methods is practically impossible because some of the methods use a dedicated (designed) hardware structure optimized for the concrete application and are available in a limited way. For comparison of methods, computing time results as an objective quantitative parameter, while the precision of the processing can be judged qualitatively only based on the opinion of radiologists.
For image segmentation the latest approach uses the active
contours method, which is usually classified as either energy-
based or level-set based, as well as their variants. These techniques are generally characterized by much better performances, but high running time even if using a very high computing power. Thus by using CNNs in the case of conventional technique of active contours resulted the active cellular contours method (CAC), for which there are currently two approaches. One approach is based on PDE (Partial Differential Equations) [9,10], which implements active contours via a non-iterative region propagation technique, where the contours are defined as the fronts of the propagating
trigger-waves. Although the method ensures a high computing
speed, the weakness of this method comes from the difficulty
of monitoring the evolution of the contour and does not allow the simultaneous expansion and contraction of different parts of the active regions. In real applications the elimination of those deficiencies inevitably increases the complexity of the algorithm. The other method is iterative and is based on a technique called Pixel-Level Snakes (PLS) [11]. The approach is based on deformable contours which evolve pixel by pixel from their initial shapes. The contour shift is guided by external
local information from the image under consideration which attracts them towards edges and by internal forces (regularizing terms) which try to maintain the smoothness of the contour curve.
In the paper [12] the Discrete Time Cellular Neural
Networks (DTCNN) are applied to image segmentation based
on active contour techniques. By these iterative algorithms
parallel processing is only partially assured in certain steps.
The paper [13] presented a special kind of cellular neural
networks based on multiple valued threshold logic in the complex plane. By using CNNs based on Universal Binary Neurons (UBNs), it makes the computation very efficient, because this requires only Boolean operations.
In the paper [11] a new algorithm for the cellular active
contour technique is proposed, which has in view, on the one
hand a greater efficiency and flexibility in the evolution of contour in relation to the limits of interest pursued, on the other hand, the actual implementation on the CNN chip set architecture available at the time. The obtained results are evaluated from the point of view of control capacity, flexibility, processing speed, precision, convergence and robustness.
The [7] paper presented some topographic cellular
computational approaches proposed for contour localization and tracking. The authors described and compared along with their associated hardware and software architectures three specific methods (Pixel Level Snakes, Constrained Wave, Computing and Moving and Patch Method). It is suggested the elaboration of some algorithms that uses (exploits) complementary and optimally the CNN or the DSP platform in the ACE-BOX computational infrastructures.
From the above results that cellular active contour
technique based on pixel level snakes allow rigorous control of contour evolution, but may be of higher complexity and time-consuming compared to the active wave based approaches.
Therefore a new algorithm is proposed in [8] for implementing cellular active contour technique based
on PLS. This new approach optimizes the computational performance, especially when PLS are implemented on pixel-parallel Single Instruction Multiple Data (SIMD) processor arrays.
III. V
ARIATIONAL COMPUTING BASED CNN
SEGMENTATION METHODS
A. Variational Based Images Processing
Over the last decade, variational methods and partial
differential equations [14,15] based techniques have been introduced for a variety of purposes including but not limited to image denoising, curve evolution, mathematical morphology, and image segmentation. Comparing with other approaches image processing based on variational computing or PDE have significant benefits both theoretically (precision, flexibility in modeling) and in terms of numerical implementation facility. The major limiting factor of the algorithms is the huge computing power requirements.

Consider a grey-scale image Φ(p,q), where Φ: R2→R, and
Ω={(p,q): p∈[1,M], q ∈[1,N], M and N ∈ R+}, by using a
variational formulation, an image processing problem can be
obtained as the minimization of a cost function:
() {} ΦΦE Minarg, (1)
where E is a given energy function, and F the first order
derivative of E. Through minimizing E, Φ results from
condition: F(Φ)=0, which is a steady state solution of the
()Φ =∂Φ∂Ft . (2)
Regardless of the chosen formulation for modeling image
processing, some solutions allow us to make combinations of them, resulting in another complex processes. If, e.g., two distinct processing are described by energy functions E1 and E2, another complex image processing can be formulated minimizing the energy:
α
Ε1+βΕ2 . (3)
Weighting the terms E1 and E2, with scalar parameters α
and β (α and β ∈R+), lets us balance the complex processing
between the limits described by the initial results. It is desirable
that the image processing method should be based on a smaller number of imposed parameters at the beginning of the algorithm and the elements deduced from the processed image content should be dominant.
B. Variational Computing Based Image Segmentation
Representing the starting point for many image
segmentation methods based on variational calculations, a special importance in this field is the energy function introduced by Mumford and Shah [16,17]:

() () Γ+ Φ−Φ β+ Φ∇ α=ΓΦ ∫∫∫∫
Γ dxdy dxdy E
R RMS2
02,
\ (4)
where R is a connected, bounded, open subset of R2, Φ0 is
the original image (the feature intensity), Γ is a curve
segmenting R, Φ is the smoothed image ⊂R2\Γ, |Γ| is the length
of Γ and α and β are the weights, scalar parameters, ( α and β
∈R+). Minimizing this classical functional requires estimating
two processes, the continuous segmented field, Φ, and a binary
edge process, Γ.
Actually it can be shown that the deterministic edge
detection based, region based, active contour based and stochastic methods are subsets of the more general problem of variational functional minimization [15].
Since it is difficult to apply gradient descent with respect to
Γ, Ambrosio and Tortorelli [17] replace Γ by a continuous
variable K and obtain:
() () () { + Φ−Φβ+Φ∇ −α =ΚΦ∫∫
RATE2
02 2K1 ,
dxdy
⎭⎬⎫
ρΚ+ ∇ρ+2K222 (5)
Solving for the minimum of equation (5) simultaneously
produces a segmented image estimate ∼
Φ and edge process
estimate∼
K. Minimizing the segmentation energy functional in
equation (5) has been used by a number of researchers for image segmentation even if the results obtained have proved to be more modest in relation to subsequent methods based on other mathematical models.
It is very important to note that the minimization of (5) is
equivalent to the joint minimization of the following pair of subfunctionals, which is the main advantage of this method.
Keeping K fixed, the first equation minimizes:

() ( ) () {}∫∫Φ−Φβ+Φ∇ −α =Φ
Rdxdy E2
02 2
K K1 (6)
Keeping Φ fixed, the second equation minimizes:
()∫∫⎪⎭⎪⎬⎫
⎪⎩⎪⎨⎧
⎟
⎠⎞⎜
⎝⎛
Ψ+Ψ−
ρΨ++ ∇ =ΚΦ
Rdxdy E2
22
1K
21K (7)
where ()22 Φ∇αρ=ΦΨ and ρ is another weight, scalar
parameter ( ρ∈R+).
The corresponding gradient descent equations are [17]:
()()() KK1K1 K2t2−Φ−αβ−Φ∇−+Φ∇⋅∇−=∂Φ∂ (8)
()2
22K12 KKtKΦ∇ −ρα+
ρ− ∇=∂∂ (9)
;0nK;0n=∂∂∂=∂∂Φ∂R R (10)
where R∂ denotes the boundary of R and n denotes the
direction normal to R∂.
In order to implement the image segmentation on CNN
structures, representation (6-7) emphasize the following:
• Solving for the minimum of equations simultaneously
produces a segmented image estimate ∼
Φ and edge
process estimate∼
K.
• In the case of CNN processing a multi-layered
structure is necessary. The two main layers are
necessary to obtain ∼
Φand∼
K. The other layers are
needed to calculate some components of the main layers.

• Each energy function contains weighted smoothing
terms (2Φ∇ from (6) or 2K∇ from (7)) and weighted
fidelity terms, or terms for edge conservation,
(()2
0Φ−Φ from (6) respectively edge calculation
2
1K ⎟
⎠⎞⎜
⎝⎛
Ψ+Ψ−from (7)).
• Solving this system of partial differential equations
includes an number of operations that can be effectively solved by parallel processing structures,
including CNN methods.
• Inclusion in these equations of a significant number of
scalar parameters dependent on image content that need to be specified a priory, makes the task of optimizing the solutions obtained very difficult.
• Even in the case of strict implementation of this model
by numerical methods, resultant accuracy is modest. This is mainly a consequence of smoothing
imperfections with a function of the form
()2⋅∇ .
• In addition, by implementing the image processing
CNN on 8-bit digital structures, solving these equations introduces approximations and additional errors.
Based on the above, as compromise solution, currently is
justified the evaluation of some algorithms that eliminate the interaction between the two main layers, so basically the two partial differential equation can be solved successively.
C. Variational Computing Based CNN Segmentation
Even if variational computing methods are used, the
determination of templates ensuring the desired processing of the grey-scale image remains a difficult problem. For variational computing based template design, using the standard CNN types for gray-scale image processing, all design constrains mentioned in [19] are respected.
For variational computing based CNN image segmentation,
in the following it will be examined the behavior of energy
functions to determine the two images, the filtered image Φ
and the edged image K. The estimate segmented image,
∼
Φ,
will result from the fusion of these two images.
To determine the noise filtered Φ image, the following
energy functions will be used:
• () () {}∫∫Φ−Φβ+Φ∇α =Φ
RL dxdy E2
02 (11)
It results the template:
0 a 0 0 0 0
A = a 1-4⋅a a D = 0 d 0
0 a 0 0 0 0
dn_Laplace.tem (12) where []1,0 ;25.0 a ∈α ⋅α=
( ) []1,0 2;kluijx2d ∈β − β= , with (B=0, z=0).
As it can be observed, actually this energy function
(Laplace ) represents a simplified variant of energy function (6).
• ()∫∫Φ−Φ∇β+Φ∇α =Φ
RVSGN dxdy E0 )( (13)
It results the template:
0 a 0 0 d 0
A = a 1 a D = d 0 d
0 a 0 0 d 0
vsgn.tem (14)
where: ( ) []1,0 ;x xsgn akl ij ∈α − α= and
( ) []1,0 ;u xsgn dkl ij ∈α − β= , with (B=0, z=0).
This function of energy ( VSGN ), is proposed to improve
edge conservation behavior of CNN image filtering [19]. It can be noticed that the image smoothing is obtained using the total variational model proposed by Rudin, Osher and Fatemi [18].
To determine image edges would justify an energy function
dependent on
2Φ∇ :
()∫∫Φ∇ρ =
Rdxdy E2K (15)
It results the nonlinear control template:
0 b 0
B = b 0 b
0 b 0
grad.tem (16)
where: ( ) []1,0 ;u u bkl ij ∈ρ − ρ= , with (A=0, z=0).
We examined the behavior of the template regarding the
production of false contours and we compared the results with those obtained through template avergrad.tem [20]. It was noted that the latter has higher efficiency, because it includes a average process. Therefore, in the segmentation methods analyzed in this paper avegrad.tem template was used for edge detection.
b b b
B = b 0 b
b b b
avegrad.tem (17)

where ( )[]1,0 ;kluiju b ∈ρ − ρ= , with (A=0, z=0).
IV. TESTING VARIATIONAL COMPUTING BASED CNN
SEGMENTATION METHODS
In this section the simulated experimental results obtained
by using the “CadetWin” (CNN Application Development Environment and Toolkit under Windows [20]) and the Matlab Tools and Development Environment are presented.
To test how the proposed algorithms work to obtain CNN
segmentation, synthetic images without noise were used and also images which were artificially added Gaussian white noise, with zero mean and different variance. The obtained results are shown in Figure 1.
In Figure 2 an example of real CT images obtained after
segmenting is presented. Radiologist experts were involved in visual evaluation of results.
V. C
ONCLUSION
In this paper, the authors propose a new method for images
segmentation, in particular segmentation of CT images using CNNs, with their attention focused primarily on the real-time isolation and evaluation of hepatic metastases.
Previous segmentation methods are evaluated to indicate
the advantages and disadvantages of these methods for medical imaging applications, difficulties for each process when it is implemented on different hardware structures. Segmentation methods have to ensure reproducibility, programmability, robustness, sensitivity and high selectivity, but at the same time high immunity to noise and reduced processing time.
A special importance is the evaluation of the methods in
regard to their integration into semiautomatic or automatic medical diagnosis systems, which provide results in real time that can be used in everyday medical practice.
To provide the conditions required in medical imaging the
proposed CNN method for image segmentation have nonlinear
templates, therefore an emulated digital CNN-UM
implemented on an FPGA is necessary [6], which enables integration of the method into an assisted diagnostic system.
Because the diversity and difficulty of currently
existing problems, both theoretically and the way of implementation, segmentation remains to be a main issue in medical imaging.
A
CKNOWLEDGMENT
This work was partially supported by a grant from the
Romanian National University Research Council, PNCDI Program ID-668/2008.

(a) (b)

(c) (d)

(e) (h)

(f) (i)

(g) (j)
Figure 1. Variational computing cased CNN segmentation: a) ideal image
without noise; b) output image after edge detection; c) result of segmentation
of the noiseless image; d) input image with Gaussian white noise, zero mean
and 0.04 variance; e) filtered output image, Φ, by using dn_Laplace.tem;
f) output image, K, after edge detection, by using avegrad.tem;
g) result of image segmentation on the noisy image by using dn_Laplace.tem;
h) filtered output image, Φ, by using vsgn.tem.tem; i) output image, K, after
edge detection by using avegrad.tem; j) result of image segmentation on the
noisy image by using vsgn.tem.

(a)

(b)

(c)

(d)
Figure 2. Variational computing based CNN segmentation of real CT image:
a) input image with noise; b) filtered output image, Φ, by using vsgn.tem; c)
output image, K , after edge detection by using avegrad.tem;
d) segmented output image. REFERENCES
[1] D. L. Pham, C. Xu, J. L. Prince, “A Survey of Current Methods in
Medical Image Segmentation” Annual Review of Biomedical
Engineering, Vol. 2, pp.315-338, 2000.
[2] S. Angenent, E. Pichon, A.Tannenbaum, “Mathematical Methods in
Medical Image Processing”, Bulletin of the American Mathematical
Society, Vol. 43, pp.365-396, 2006.
[3] L.O. Chua, L. Yang: “Cellular Neural Networks: Theory and
Applications”, IEEE Trans. on Circuits and Systems, Vol. 35,
pp.1257-1290, 1988.
[4] T. Roska, L.O. Chua, “The CNN Universal Machine: An Analogic
Array Computer”, IEEE Trans. on Circuits and Systems, Vol. 40,
pp.163-173, 1993.
[5] G.L. Cembrano, A. Rodríguez-Vázquez, S. Espejo-Meana, R.
Domínguez-Castro, “ACE16k: A 128 ×128 Focal Plane Analog
Processor with Digital I/O“, Int. J. Neural Syst. 13(6), pp. 427-434,
2003.
[6] Z. Nagy, Zs. Vörösházi, P. Szolgay, “Emulated Digital CNN-UM
Solution of Partial Diferential” Int. Journal of Circuit Theory and
Applications Vol. 34, Issue 4, pp.445-470, 2006.
[7] D. Hillier, V. Binzberger, D. L. Vilariño, Cs. Rekeczky, “Topographic
cellular active contour techniques: theory, implementations and comparisons”, Int. Journal of Circuit Theory and Applications, Vol. 34
Issue 2, pp.183-216, 2006.
[8] P. Dudek, D. L. Vilariño, “A Cellular Active Contours Algorithm Based
on Region Evolution”, Int. Workshop on Cellular Neural Networks and
Their Applications, Istanbul, Turkey, 2006.
[9] Cs. Rekeczky and L.O. Chua, “Computing with Front Propagation:
Active Contour and Skeleton Models in Continuous-Time”, CNN.
Journal of VLSI Signal Processing Systems, 23(2/3), pp.373-402, 1999.
[10] Cs. Rekeczky, “CNN Architectures for Constrained Diffusion Based
Locally Adaptive Image Processing: Research Articles“, Intern. Journal
of Circuit Theory and Applications Vol.30, pp.313-348, 2002.
[11] D.L. Vilariño and Cs. Rekeczky, “Pixel Level Snakes on the CNNUM:
Algorithm Design, On-chip Implementation and Applications”, Int. Journal of Circuit Theory and Applications, Vol. 33, Issue 1, pp.17–51,
2005.
[12] D.L. Vilariño, D. Cabello, X.M. Pardo, V.M. Brea, “Cellular Neural
Networks and Active Contours: a Tool for Image Segmentation”, Image
and Vision Computing, 21, pp.189–204, 2003.
[13] I. Aizenberg, N. Aizenberg, J. Hiltner, C. Moraga, E. Meyer zu Bexten,
“Cellular Neural Networks and Computational Intelligence in Medical
Image Processing”, Image and Vision Computing 19, pp.177–183, 2001.
[14] V. Casellas, J.M. Morel, G. Sapiro, A. Tannenbaum, “Introduction to the
Special Issue on Partial Differential Equations and Geometry-Driven Diffusion in Image Processing and Analysis” IEEE Trans. on Image
Processing, Vol. 7, pp.269-273, 1998.
[15] D. Mumford and J. Shah, ”Boundary Detection by Minimizing
Functionals”, Proc. IEEE Conf. on Computer Vision and Pattern
Recognition, pp.22-26, 1985.
[16] J. Morel and S. Solimini, “Variational Methods in Image Segmentation”,
Vol. 14 of Progress in Nonlinear Differential Equations and their
Applications, Birkhauser, 1995.
[17] J. Shah, “A Common Framework for Curve Evolution, Segmentation
and Anisotropic Diffusion”, Proc. IEEE Conf. on Computer Vision and
Pattern Recognition, pp.136-142, 1996.
[18] L. Rudin, S. Osher, E. Fatemi, “Nonlinear total variational based noise
removal algorithms,”Physica D, Vol. 60, pp. 259–268, 1992.
[19] A. Gacsádi, P. Szolgay, “Variational Computing Based Images
Denoising Methods by using Cellular Neural Networks”, Proc. of the European Conference on Circuit Theory and Design, pp.85-88, Antalya,
Turkey, 2009.
[20] *** “CadetWin-CNN application development environment and toolkit
under Windows”, Version 3.0, Analogical and Neural Computing Laboratory, Comp. and Aut. Inst. Hungarian Academy of Sciences,
Budapest,1999.