- Research Article
- Open Access
Performance Analysis of Bit-Width Reduced Floating-Point Arithmetic Units in FPGAs: A Case Study of Neural Network-Based Face Detector
EURASIP Journal on Embedded Systemsvolume 2009, Article number: 258921 (2009)
This paper implements a field programmable gate array- (FPGA-) based face detector using a neural network (NN) and the bit-width reduced floating-point arithmetic unit (FPU). The analytical error model, using the maximum relative representation error (MRRE) and the average relative representation error (ARRE), is developed to obtain the maximum and average output errors for the bit-width reduced FPUs. After the development of the analytical error model, the bit-width reduced FPUs and an NN are designed using MATLAB and VHDL. Finally, the analytical (MATLAB) results, along with the experimental (VHDL) results, are compared. The analytical results and the experimental results show conformity of shape. We demonstrate that incremented reductions in the number of bits used can produce significant cost reductions including area, speed, and power.
Neural networks have been studied and applied in various fields requiring learning, classification, fault tolerance, and associate memory since the 1950s. The neural networks are frequently used to model complicated problems which are difficult to make equations by analytical methods. Applications include pattern recognition and function approximation . The most popular neural network is the multilayer perceptron (MLP) trained using the error back propagation (BP) algorithm . Because of the slow training in MLP-BP, however, it is necessary to speed up the training time. The very attractive solution is to implement it on field programmable gate arrays (FPGAs).
For implementing MLP-BP, each processing element must perform multiplication and addition. Another important calculation is an activation function, which is used to calculate the output of the neural network. One of the most important considerations for implementing a neural network on FPGAs is the arithmetic representation format. It is known that floating-point (FP) formats are more area efficient than fixed-point ones to implement artificial neural networks with the combination of addition and multiplication on FPGAs .
The main advantage of the FP format is its wide range. The feature of the wide range is good for neural network systems because the system requires the big range when the learning weight is calculated or changed . Another advantage of the FP format is the ease of use. A personal computer uses the floating-point format for its arithmetic calculation. If the target application uses the FP format, the effort of converting to other arithmetic format is not necessary.
FP hardware offers a wide dynamic range and high computation precision, but it occupies large fractions of total chip area and energy consumption. Therefore, its usage is very limited. Many embedded microprocessors do not even include a floating-point unit (FPU) due to its unacceptable hardware cost.
A bit-width reduced FPU solves this complexity problem [5, 6]. An FP bit-width reduction can provide a significant saving of hardware resources such as area and power. It is useful to understand the loss in accuracy and the reduction in costs as the number of bits in an implementation of floating-point representation is reduced. Incremented reductions in the number of bits used can produce useful cost reductions. In order to determine the required number of bits in the bit-width reduced FPU, analysis of the error caused by a reduced-precision is essential. Precision reduced error analysis for neural network implementations was introduced in . A formula that estimates the standard deviation of the output differences of fixed-point and floating-point networks was developed in . Previous error analyses are useful to estimate possible errors. However, it is necessary to know the maximum and average possible errors caused by a reduced-precision FPU for a practical implementation.
Therefore, in this paper, the error model is developed using the maximum relative representation error (MRRE) and average relative representation error (ARRE) which are representative indices to examine the FPU accuracy.
After the error model for the reduced precision FPU is developed, the bit-width reduced FPUs and the neural network for face detection are designed using MATLAB and Very high speed integrated circuit Hardware Description Language (VHDL). Finally the analytical (MATLAB) results are compared with the experimental (VHDL) results.
Detecting a face in an image means to find its position in the image plane and its size. There has been extensive research in the field, ranging mostly in the software domain [9, 10]. There have been a few researches for hardware face detector implementations on FPGAs [11, 12], but most of the proposed solutions are not very compact and the implementations are not purely on hardware. In our previous work, the FPGA-based stand-alone face detector to support a face recognition system was suggested and showed that an embedded system could be made .
Our central contribution here is to examine how neural network-based face detector can employ the minimal number of bits in an FPU to reduce hardware resources, yet maintain a face detector's overall accuracy.
This paper is outlined as follows. In Section 2, the FPGA implementation of the neural network face detector using the bit-width reduced FPUs is described. Section 3 explains how representation errors theoretically affect a detection rate in order to determine the required number of bits for the bit-width reduced FPUs. In Section 4, the experimental results are presented, and then they are compared to the analytical results to verify if both results match closely. Section 5 draws conclusions.
2. A Neural Network-Based Face Detector Using a Bit-Width Reduced FPU in an FPGA
2.1. General Review on MLP
A neural network model can be categorized into two types: single layer perceptron and multilayer perceptron (MLP). A single layer perceptron has only two layers: the input layer and the output layer. Each layer contains a certain number of neurons. The MLP is a neural network model that contains multiple layers, typically three or more layers including one or more hidden layers. The MLP is a representative method of supervized learning.
Each neuron in one layer receives its input from the neurons in the previous layer and broadcasts its output to the neurons in the next layer. Every processing node in one particular layer is usually connected to every node in the previous layer and the next layer. The connections carry weights, and the weights are adjusted during training. The operation of the network consists of two stages: forward pass and backward pass or back-propagation. In the forward pass, an input pattern vector is presented to the network and the output of the input layer nodes is precisely the components of the input pattern. For successive layers, the input to each node is then the sum of the products of the incoming vector components with their respective weights.
The input to a node is given by simply
where is the weight connecting node to node and out i is the output from node .
The output of a node j is simply
which is then sent to all nodes in the following layer. This continues through all the layers of the network until the output layer is reached and the output vector is computed. The input layer nodes do not perform any of the above calculations. They simply take the corresponding value from the input pattern vector. The function f denotes the activation function of each node, and it will be discussed in the following section.
It is known that 3 layers having 2-hidden layers are better than 2 layers to approximate any given function . However, a 2-layer MLP is used in this paper, as shown in Figure 1. The output error equation of the first layer (15) and the error equation of the second layer (21) are different. However, the error equation of the second layer (21) and the error equation of the other layers (22) are the same form. Therefore, a 2-layer MLP is enough to be examined in this paper. The number of neurons of 400 and 300 were used for input and first layer respectively in this experiment.
After the face data enters the input node, it is calculated by the multiplication-and-accumulation (MAC) with weights. Face or non-face data is determined by comparing output results with the thresholds. For example, if the output is larger than the threshold, it is considered as a face data. Here, on the FPGA, this decision is easily made by checking a sign bit after subtracting the output results and the threshold.
2.2. Estimation of Activation Function
An activation function is used to calculate the output of the neural network. The learning procedure of the neural network requires the differentiation of the activation function to renew the weights value. Therefore, the activation function has to be differentiable. A sigmoid function, having an "'' shape, is used for the activation function, and a logistic or a hyperbolic tangent function is commonly used as the sigmoid function. The hyperbolic tangent function and its antisymmetric feature were better than the logistic function for learning ability in our experiment. Therefore, hyperbolic tangent sigmoid transfer function was used, as shown in (3). The first-order derivative of the hyperbolic tangent sigmoid transfer function can be easily obtained as (4). MATLAB provides the commands, "tansig" and "dtansig":
where x = in (2).
The activation function can be estimated by using different methods. The Taylor and polynomial methods are effective, and guarantee the highest speed and accuracy among these methods.
The polynomial method is used to estimate the activation function in this paper as seen in (5) and (6) because it is simpler than the Taylor approximation.
A first-degree polynomial estimation of an activation function is
A first-order derivative is
Figure 2 shows the estimation (5) of an activation function (3).
2.3. FPU Implementation
Figure 3 shows the simplified block diagram of the implemented neural network in an FPGA. The module consists of control logic and an FPU.
The implemented FPU is IEEE 754 Standard  compliant. The FPU in this system has two modules: multiplication and addition. Figure 4 shows the block diagram of the 5 stage-pipelined FP addition and multiplication unit implemented in this paper. A commercial IP core, an FP adder of the LEON processor  is used and modified to make the bit size adjustable. A bit-width reduced floating-point multiplication unit is designed using a multiplier and a hard intellectual property (IP) core in an FPGA to improve speed. Consequently, the multiplication was performed within 2 cycles of total stages as shown in Figure 4.
2.4. Implementation of the Neural Network-Based FPGA Face Detector Using MATLAB and VHDL
Figure 5 shows the total design flow using MATLAB and VHDL. MATLAB program consists of two parts: learning and detection. After the learning procedure, weights data are fixed and saved to a file. The weights file is saved to a memory model file for FPGA and VHDL simulation. The MATLAB also provides input test data to the VHDL program and analyzes the result from the result file of MODELSIM simulation program. Preprocessing includes mask, resizing, and normalization.
The Olivetti face database  is chosen for this study. The Olivetti face database consists of mono-color face and non-face image so it is easy to use. Some other databases which have large size, color, mixed with other pictures are difficult for this error analysis purpose due to the necessity of more preprocessing like cropping, data classification, and color model change.
Figure 6 shows the classification result of 220 face and non-face data. -axis shows the data number of face data from 1 to 60, and non-face data from 61 to 220. -axis shows the output value of the neural network. The neural network is learned to pursue the desired value "1'' for face and "–1'' for non-face.
3. Error Analysis of the Neural Network Caused by Reduced-Precision FPU
3.1. MRRE and ARRE
The number of bits in the FPU is important for the area, operating speed, and power . Therefore, it is important to decide the minimal number of bits in floating-point hardware to reduce hardware costs, yet maintain an application's overall accuracy. A maximum relative representation error (MRRE)  is used as one of the indices of floating-point arithmetic accuracy, as shown in Table 1. Note that "'' and "'' represent exponent and mantissa, respectively. The MRRE can be obtained as follows:
where ulp is a unit in the last position and is the exponent base.
An average relative representation error (ARRE) can be considered for practical use:
3.2. Output Error Estimation of the Neural Network
The FPU representation error increases with repetitive multiplication and addition in the neural network. The difference in output can be expressed using the following equations with the notation depicted in Figure 7.
The error of the 1st layer is the difference between the output by a finite precision arithmetic () and the ideal output (), and it can be described as
where represents the hidden layer error ( represents total error generated between hidden layer and output layer on Figure 7), W represents the weights, X represents input data, and O represents the output of the hidden layer. is the summation of other possible errors, and defined by
is the nonlinear function error by Taylor estimation; is very small and negligible. Therefore, becomes 0.
Other calculation errors occur when the differential of activation is calculated (i.e., ), and the final face determination is calculated as follows: f(x) = (x) + (-0.5).
The multiplication error, is not considered in this paper. The multiplication unit assigns twice the size of the bits to save the result data. For example, multiplication of 16 bits × 16 bits needs 32 bits. This large size register reduces the error, thus the error is negligible.
However, the summation error, is not negligible and added to the error term, . The multiplication error () and the addition error () are bounded by the MRRE (assuming rounding mode = truncation) as given by (11) and (12):
where negative sign (–) describes the direction.
For example, the of ""::
For example, the of "": .
Note that the maximum error caused by truncation of rounding scheme is bounded as
The error caused by round-to-the-nearest scheme is bounded as
The truncation of rounding scheme creates a negative error and a round-to-nearest scheme creates a positive error. The total error can be reduced by almost 50% by round-to-nearest scheme .
From (9), the terms and are weights data and input data, respectively, including the reduced-precision error. They are described by and . Therefore, the multiplication of weights and input data are denoted by.
From (9), the error of the first layer, is given by
The error of the second layer can also be found as
By replacing the and the with and, (18) becomes
The error (22) can be generalized for the l th layer, l, in a similar way:
3.3. Output Error Estimation by MRRE and ARRE
The error equation can be rewritten using the MRRE in the error term to find the maximum output error caused by reduced-precision. The average error can be estimated in the practical application by replacing the MRRE with ARRE
From (16), the output error of the first layer is described as
The and terms can be defined by and . Thus from (24), the error is bounded so that
Finally, the output error of the second layer is also described from (22) as shown in (28), where the error of weights can also be written as
3.4. Relationship between MRRE and Output Error
In order to observe the relationship between the MRRE and output error, (28) is written as (30) again.
By using (26),
Some properties are derived from (26) and (30) for the output error. The differential of summations affects the output error proportionally like
The output of the well-learned neural network system goes to the desired value as shown in Figure 2. In that case, the differential value goes to 0 as shown in Figure 8. It means the well-learned neural network system has less output error.
One more finding is that the output error is also proportional to the MRRE.
From the (30),
where (assuming "rounding mode = truncation"). Therefore, (33) can be described as
Finally, it is concluded that n-bits reduction in the FPU creates 2n times the error. If one bit is reduced, for example, the output error is doubled (e.g., ). After putting the MRRE between FPU32 and other reduced precision FPU bits into error terms in (26) and (28) using MATLAB and real face data, finally, the total accumulated error of the neural network is obtained as shown in Table 11.
4. Result and Discussion
4.1. FPGA Synthesis Results
The FPGA-based face detector using the neural network and the reduced-precision, FPU, is implemented in this paper. The logic circuits of the neural network-based FPGA face detector are synthesized using the FPGA design tool, Xilinx ISE on a Spartan-3 XC3S4000 . To verify the error model, first of all, the neural network on a PC is designed using MATLAB. Next, the weights and test-bench data are saved as a file to verify the VHDL code.
After simulation, area and operating speed are obtained by synthesizing the logic circuits. The FPU uses the same calculation method, floating-point arithmetic, as the PC so it is easy to verify and easy to change the neural network's structure.
The implemented neural network-based FPGA face detector (FPU16) took only 5.3 milliseconds to process 1 frame at 80 MHz which is 9 times faster than 50 milliseconds (i.e., 40 milliseconds for loading time + 10 milliseconds for calculation time) required for a PC (Pentium 4, 1.4 GHz) as shown in Table 2. As the total FPU representation bits decrease, a maximum clock frequency increases considerably from 21% (FPU24) to 67% (FPU16) compared to FPU32.
The remaining question is to examine if bit-width reduced FPU can still maintain a face detector's overall accuracy. For this purpose, detection rate error for bit-width reduced FPU will be discussed in Section 4.2.2.
As shown in Table 3, only 2% (650/27648) and 4% (1077/27648) of the total available slices (3S4000) are used for FPU16 and FPU32, respectively. Therefore, the stand-alone embedded face detection system including preprocessing, FPU, and other extra functions can be easily implemented on a small inexpensive FPGA.
As the bit-width decreases, the number of slices is decreased from 18.5% (FPU24) to 39.7% (FPU16) compared to FPU32.
Bit reduction of the FP adder leads to an area reduction and a faster operating clock speed. For example, a 50% bit reduction from FP adder 32 to FP adder 16 results in a 49% reduction of the adder area (250/486) and a 50% reduction of the memory (1880/3760) as shown in Table 4. It is possible to use the XILINX FPGA 3S4000 which provides the size of 2160 Kbits memory (block RAM: 1728 Kb, distributed memory: 432 Kb) when the FPU16 is necessary.
The number of slices of the floating-point adder varies from 31% (FP12: 173/556) to 45% (FP32: 486/1077) of the total size of the neural network as shown in Table 4.
As the bit-width decreases, the power consumption decreases. For example, bit reduction from the FPU32 to the FPU16 reduces the total power by 14.7% (FPU32: 306 mW, FPU16: 261 mW) through RAM, multiplier, and I/O as shown in Table 5.
The change of the logic cell does not considerably affect the power as much as hardwired IP such as memory and multiplier spend the power. See the number of configurable logic blocks (CLBs) in Table 5.
4.1.4. Architectures of FP Adder
The neural network system and the FPU hardware performance are greatly affected by the FP addition . The bit-width reduced FP addition is modified for this study from the commercial IP, LEON processor. LEON FPU uses standard adder architecture . The system performance and the clock speed can be further improved by leading-one-prediction (LOP) algorithm and 2-path (close-and-far path) algorithm, respectively .
In our experiment, FP addition based on the LOP algorithm increases the maximum clock frequency by 42.7% compared to the performance of the commercial IP, LEON. The FP addition based on the 2-path algorithm [18, 22] increases the area by 111%, but improves the maximum clock frequency by 180% compared to the performance of the commercial IP, LEON as shown in Table 6.
The specification of the implemented neural network-based FPGA face detector is summarized in Table 7.
4.2. Detection Rate Error
Two factors affect the detection rate error. One is the polynomial estimation error as shown in Figure 2 which is occurred when the activation function is estimated through the polynomial equation. Another possible error caused by the bit-width reduced FPU.
4.2.1. Detection Rate Error by Polynomial Estimation
To reduce the error caused by polynomial estimation, the polynomial equation, (35) can be more elaborately modified as shown in (36). The problem of (36) is not differentiable at and also the error (30) will be identically 0 (i.e., ) for, which will make error analysis difficult:
Therefore, the simplified polynomial (35) is used in this paper. It is confirmed through MATLAB simulation that this polynomial approximation results in an average 4.9% error in the detection rate compared with the result of (3) in our experiment as shown in Table 8.
4.2.2. Detection Rate Error by Reduced-Precision FPU
Table 9 is obtained after the threshold value changed from 0.1 to 1. When the threshold is 0.5, the face detection rate is 83% and the non-face detection rate is 55%. When the threshold is 0.6, face and the non-face detect is almost same as 71.67% and 69.4% respectively. As the threshold value goes to "1" (i.e. as the horizontal red line goes up in Figure 6), face detection rate is decreased. It means input image is difficult to pass, and it is good for security. Therefore, the threshold is needed to be chosen accordingly depending upon applications. The result of Table 9 is used in the second column (FPU64(PC) of the Table 10.
Table 10 shows the detection rate error (i.e., ∣ detection rate of FPU64 (PC software)—detection rate of reduced-precision FPUs∣) caused by reduced-precision FPUs. The detection rate is changed from FPU64(PC) to FPU16 by only 5.91% (i.e., ).
Analytical results are found to be in agreement with simulation results as shown in Figure 10. The analytical MRRE results and the maximum experimental results show conformity of shape. The analytical ARRE results and the minimum experimental results also show conformity of shape.
As the bits in the FPU are reduced within the ranges from 32 bits to 14 bits, the output error is incremented by times. For example, 2-bit reduction from FPU16 to FPU14 makes 4 times () the error.
Due to the small number of fraction bits (e.g., 5 bits in FPU12), no meaningful results are obtained under 14 bits. Therefore, at least 14 bits should be employed to achieve an acceptable face detection rate. See Figures 9 and 10.
In this paper, the analytical error model was developed using the maximum relative representation error (MRRE) and average relative representation error (ARRE) to obtain the maximum and average output errors for the bit-width reduced FPUs.
After the development of the analytical error model, the bit-width reduced FPUs, and the neural network were designed using MATLAB and VHDL. Finally, the analytical (MATLAB) results with the experimental (VHDL) results were compared.
The analytical results and the experimental results showed conformity of shape. According to both results, as the n bits in FPU are reduced within the ranges from 32 bits to 14 bits, the output error is incremented by times.
An operating speed was significantly improved from an FPGA-based face detector implementation using a reduced precision FPU. For example, it took only 5.3 milliseconds in the FPU16 to process one frame which is 9 times faster than 50 milliseconds (40 milliseconds for loading time milliseconds for calculation time) of the PC (Pentium 4, 1.4 GHz). It was found that bit reduction from FPU 32 bits to FPU16 bits reduced the size of memory and arithmetic units by 50% and the total power consumption by 14.7%, while still maintaining 94.1% face detection accuracy. The developed error analysis for bit-width reduced FPUs will be helpful to determine the specification for an embedded neural network hardware system.
Skrbek M: Fast neural network implementation. Neural Network World 1999,9(5):375-391.
Rumelhart DE, McClelland JL: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1. MIT Press, Cambridge, Mass, USA; 1986.
Li X, Moussa M, Areibi S: Arithmetic formats for implementing artificial neural networks on FPGAs. Canadian Journal of Electrical and Computer Engineering 2006,31(1):30-40.
Brown HK, Cross DD, Whittaker AG: Neural network number systems. Proceedings of International Joint Conference on Neural Networks (IJCNN '90), June 1990, San Diego, Calif, USA 3: 903-908.
Kontro J, Kalliojarvi K, Neuvo Y: Use of short floating-point formats in audio applications. IEEE Transactions on Consumer Electronics 1992,38(3):200-207. 10.1109/30.156684
Tong J, Nagle D, Rutenbar R: Reducing power by optimizing the necessary precision/range of floating-point arithmetic. IEEE Transactions on VLSI Systems 2000,8(3):273-286.
Holt JL, Hwang J-N: Finite precision error analysis of neural network hardware implementations. IEEE Transactions on Computers 1993,42(3):281-290. 10.1109/12.210171
Sen S, Robertson W, Phillips WJ: The effects of reduced precision bit lengths on feed forward neural networks for speech recognition. Proceedings of IEEE International Conference on Neural Networks, June 1996, Washington, DC, USA 4: 1986-1991.
Feraud R, Bernier OJ, Viallet J-E, Collobert M: A fast and accurate face detector based on neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 2001,23(1):42-53. 10.1109/34.899945
Rowley HA, Baluja S, Kanade T: Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998,20(1):23-38. 10.1109/34.655647
Theocharides T, Link G, Vijaykrishnan N, Irwin MJ, Wolf W: Embedded hardware face detection. Proceedings of the 17th IEEE International Conference on VLSI Design, January 2004, Mumbai, India 133-138.
Sadri M, Shams N, Rahmaty M, et al.: An FPGA based fast face detector. Global Signal Processing Expo and Conference (GSPX '04), September 2004, Santa Clara, Calif, USA
Lee Y, Ko S-B: FPGA implementation of a face detector using neural networks. Canadian Conference on Electrical and Computer Engineering (CCECE '07), May 2006, Ottawa, Canada 1914-1917.
Chester D: Why two hidden layers are better than one. Proceedings of International Joint Conference on Neural Networks (IJCNN '90), January 1990, Washington, DC, USA 1: 265-268.
IEEE Std 754-1985 : IEEE standard for binary floating-point arithmetic. Standards Committee of the IEEE Computer Society, New York, NY, USA, August 1985
LEON Processor http://www.gaisler.com
Olivetti & Oracle Research Laboratory The Olivetti & Oracle Research Laboratory Face Database of Faces, http://www.cam-orl.co.uk/facedatabase.html The Olivetti & Oracle Research Laboratory Face Database of Faces,
Koren I: Computer Arithmetic Algorithms. 2nd edition. A K Peters, Natick, Mass, USA; 2001.
XILINX : Spartan-3 FPGA Family Complete Data Sheet. Product Specification, April 2008
XILINX Spartan-3 Web Power Tool Version 8.1.01 http://www.xilinx.com/cgi-bin/power_tool/power_Spartan3
Govindu G, Zhuo L, Choi S, Prasanna V: Analysis of high-performance floating-point arithmetic on FPGAs. Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS '04), April 2004, Santa Fe, NM, USA 149-156.
Malik A: Design trade-off analysis of floating-point adder in FPGAs, M.S. thesis. Department of Electrical and Computer Engineering, University of Saskatchewan, Saskatoon, Canada; 2005.
The authors would like to acknowledge the Natural Science and Engineering Research Council of Canada (NSERC) / the University of Saskatchewan's Publications Fund, the Korea Research Foundation, and a Korean Federation of Science and Technology Societies grant funded by the South Korean government (MOEHRD, Basic Research Promotion Fund) for supporting this research and to thank the reviewers for their valuable suggestions.