Accelerating Seismic Computations Using Customized Number Representations on FPGAs

2.1. Number Representation

As mentioned in Section 1, precision and range are key resources to be traded off against the performance of a computation. In this work, we look at two different types of number representation: fixed-point and floating-point.

Fixed-Point Numbers

The fixed-point number has two parts, the integer part and the fractional part. It is in a format as shown in Table 1.

When it uses a sign-magnitude format (the first bit defines the sign of the number), its value is given by . It may also use a two-complement format to indicate the sign.

Integer part:  bits	Fractional part:  bits

Floating-Point Numbers

According to IEEE-754 standard, floating-point numbers can be divided into three parts: the sign bit, the exponent, and the mantissa, shown as in Table 2.

Their values are given by . The sign bit defines the sign of the number. The exponent part uses a biased format. Its value equals to the sum of the original value and the bias, which is defined as . The extreme values of the exponent ( and are used for special cases, such as values of zero and . The mantissa is an unsigned fractional number, with an implied "1" to the left of the radix point.

Sign: 1 bit	Exponent:  bits	Mantissa:  bits

2.2. Hardware Compilation Tool

We use a stream compiler (ASC) [5] as our hardware compilation tool to develop a range of different solutions for seismic applications. ASC was developed following research at Stanford University and Bell Labs, and is now commercialized by Maxeler Technologies. ASC enables the use of FPGAs as highly parallel stream processors. ASC is a C-like programming environment for FPGAs. ASC code makes use of C++ syntax and ASC semantics which allow the user to program on the architecture level, the arithmetic level, and the gate level. ASC provides the productivity of high-level hardware design tools and the performance of low-level optimized hardware design. On the arithmetic level, PAM-Blox II provides an interface for custom arithmetic optimization. On the higher level, ASC provides types and operators to enable research on custom data representation and arithmetic. ASC hardware types are HWint, HWfix, and HWfloat. Utilizing the data-types, we build libraries such as a function evaluation library or develop special circuits to solve particular computational problems such as graph algorithms. Algorithm 1 shows a simple example of an ASC description for a stream architecture that doubles the input and adds "55."

Algorithm 1: A simple ASC example.

// ASC code starts here

STREAM_START;

// Hardware Variable Declarations

HWint in (IN);

HWint out (OUT);

HWint tmp (TMP);

STREAM_LOOP (16);

tmp = (in ) 55;

out = tmp;

// ASC code ends here

STREAM_END;

The ASC code segment shows HWint variables and the familiar C syntax for equations and assignments. Compiling this program with "gcc" and running it creates a netlist which can be transformed into a configuration bitstream for an FPGA.

2.3. Precision Analysis

There exist a number of research projects that focus on precision analysis, most of which are static methods that operate on the computational flow of the design and uses techniques based on range and error propagation to perform the analysis.

Lee et al. [6] present a static precision analysis technique which uses affine arithmetic to derive an error model of the design and applies simulated annealing to find out minimum bit widths to satisfy the given error requirement. A similar approach is shown in a bit-width optimization tool called Prcis [7].

These techniques are able to perform an automated precision analysis of the design and provide optimized bit widths for the variables. However, they are not quite suitable for seismic imaging algorithms. The first reason is that seismic imaging algorithms usually involve numerous iterations, which can lead to overestimation of the error bounds and derive a meaningless error function. Secondly, the computation in the seismic algorithms does not have a clear error requirement. We can only judge the accuracy of the computation from the generated seismic image. Therefore, we choose to use a dynamic simulation method to explore different precisions, detailed in Sections 3.3 and 3.4.

2.4. Computation Bottlenecks in Seismic Applications

Downward-continued migration comes in various flavors including common azimuth migration [8], shot profile migration, source-receiver migration, plane-wave or delayed shot migration, and narrow azimuth migration. Depending on the flavor of the downward continuation algorithm, there are four potential computation bottlenecks.

The primary bottleneck of reverse time migration is applying the finite-different stencil. In addition to the large operation count (5 to 31 samples per cell) the access pattern has poor cache behavior for real size problems. Beyond applying the 3D stencil, the next most dominant cost is implementing damping boundary conditions. Methods such as perfectly matched layers (PMLs) can be costly [11]. Finally, if you want to use reverse time migration for velocity analysis, subsurface offset gathers need to be generated. The same cost profile that exists in downward continued-based migration exists for reverse time migration.

In this paper, we focus on two of the above computation bottlenecks: one is the FK step in forward continued-based migration, which includes a square root function and a complex exponential operation; the other one is the 3D convolution in reverse time migration. We perform automated precision exploration of these two computation cores, so as to figure out the minimum precision that can still generate accurate enough seismic images.

3.1. Range Profiling

In the profiling stage, the major objective is to collect range and distribution information for the variables. The idea of our approach is to instrument every target variable in the code, adding function calls to initialize data structures for recording range information and to modify the recorded information when the variable value changes.

For the range information of the target variables (variables to map into the circuit design), we keep a record of four specific points on the axis, shown in Figure 2. The points and represent the values far away from zero, that is, the maximum absolute values that need to be represented. Based on their values, the integer bit width of fixed-point numbers can be determined. Points and represent the values close to zero, that is, the minimum absolute values that need to be represented. Using both the minimum and maximum values, the exponent bit width of floating-point numbers can be determined.

For the distribution information of each target variable, we keep a number of buckets to store the frequency of values at different intervals. Figure 3 shows the distribution information recorded for the real part of variable "wfld" (a complex variable). In each interval, the frequency of positive and negative values is recorded separately. The results show that, for the real part of variable "wfld," in each interval, the frequencies of positive and negative values are quite similar, and the major distribution of the values falls into the range to .

The distribution information provides a rough metric for the users to make an initial guess about which number representations to use. If the values of the variables cover a wide range, floating-point and LNS number formats are usually more suitable. Otherwise, fixed-point numbers shall be enough to handle the range.

3.2. Circuit Design: Basic Arithmetic and Elementary Function Evaluation

After profiling range information for the variables in the target code, the second step is to map the code into a circuit design described in ASC. As a high-level FPGA programming language, ASC provides hardware data types, such as HWint, HWfix, and HWfloat. Users can specify the bit-width values for hardware variables, and ASC automatically generates corresponding arithmetic units for the specified bit widths. It also provides configurable options to specify different optimization modes, such as AREA, LATENCY, and THROUGHPUT. In the THROUGHPUT optimization mode, ASC automatically generates a fully pipelined circuit. These features make ASC an ideal hardware compilation tool to retarget a piece of software code onto the FPGA hardware platform.

With support for fixed-point and floating-point arithmetic operations, the target Fortran code can be transformed into ASC C++ code in a straightforward manner. We also have interfaces provided by ASC to modify the internal settings of these arithmetic units.

Besides basic arithmetic operations, evaluation of elementary functions takes a large part in seismic applications. For instance, in the first piece of target code we try to accelerate, the FK step, a large portion of the computation is to evaluate the square root and sine/cosine functions. To map these functions into efficient units on the FPGA board, we use a table-based uniform polynomial approximation approach, based on Dong-U Lee's work on optimizing hardware function evaluation [16]. The evaluation of the two functions can be divided into three different phases [17].

To keep the whole unit small and efficient, we use degree-one polynomial so that only one multiplication and one addition are needed to produce the evaluation result. Meanwhile, to preserve the approximation error at a small scale, the reduced evaluation range is divided into uniform segments. Each segment is approximated with a degree-one polynomial, using the minimax algorithm. In the FK step, the square root function is approximated with 384 segments in the range of [0.25, 1] with a maximum approximation error of , while the sine and cosine functions are approximated with 512 segments in the range of [0, 2] with a maximum approximation error of .

3.3. Bit-Accurate Value Simulator

As discussed in Section 3.1, based on the range information, we are able to determine the integer bit width of fixed-point, and partly determine the exponent bit width of floating-point numbers (as exponent bit width does not only relate to the range but also to the accuracy). The remaining bit widths, such as the fractional bit width of fixed-point, and the mantissa bit width of floating-point numbers, are predominantly related to the precision of the calculation. In order to find out the minimum acceptable values for these precision bit widths, we need a mechanism to determine whether a given set of bit-width values produce satisfactory results for the application.

In our previous work on function evaluation or other arithmetic designs, we set a requirement of the absolute error of the whole calculation, and use a conservative error model to determine whether the current bit-width values meet the requirement or not [6]. However, a specified requirement for absolute error does not work for seismic processing. To find out whether the current configuration of precision bit width is accurate enough, we need to run the whole program to produce the seismic image, and find out whether the image contains the correct pattern information. Thus, to enable exploration of different bit-width values, a value simulator for different number representations is needed to provide bit-accurate simulation results for the hardware designs.

With the requirement to produce bit-accurate results as the corresponding hardware design, the simulator also needs to be efficiently implemented, as we need to run the whole application (which takes days using the whole input dataset) to produce the image.

In our approach, the simulator works with ASC format C++ code. It reimplements the ASC hardware data types, such as HWfix and HWfloat, and overloads their arithmetic operators with the corresponding simulation code. For HWfix variables, the value is stored in a 64-bit signed integer, while another integer is used to record the fractional point. The basic arithmetic operations are mapped into shifts and arithmetic operations of the 64-bit integers. For HWfloat variables, the value is stored in a 80-bit extended-precision floating-point number, with two other integers used to record the exponent and mantissa bit width. To keep the simulation simple and fast, the arithmetic operations are processed using floating-point values. However, to keep the result bit accurate, during each assignment, by performing corresponding bit operations, we decompose the floating-point value into mantissa and exponent, truncate according to the exponent and mantissa bit widths, and combine them back into the floating-point value.

3.4. Accuracy Evaluation of Generated Seismic Images

As mentioned above, the accuracy of a generated seismic image depends on the pattern contained inside, which estimates the geophysical status of the investigated area. To judge whether the image is accurate enough, we compare it to a "target" image, which is processed using single-precision floating-point and assumed to contain the correct pattern.

To perform this pattern comparison automatically, we use techniques based on prediction error filters (PEFs) [18] to highlight differences between two images. The basic work flow of comparing image to image (assume image is the "target" image) is as follows.

By the end of the above work flow, we achieve a single value which describes the difference from the generated image to the "target image." For convenience of discussion afterwards, we call this value as "difference indicator" (DI).

Figure 4 shows a set of different seismic images calculated from the same dataset, and their DI values compared to the image with correct pattern. The image showing correct pattern is calculated using single-precision floating-point, while the other images are calculated using fixed-point designs with different bit-width settings. All these images are results of the bit-accurate value simulator mentioned above.

If the generated image contains no information at all (as shown in Figure 4(a)), the comparison does not return a finite value. This is mostly because a very low precision is used for the calculation. The information is lost during numerous iterations and the result only contains zeros or infinities. If the comparison result is in the range of to (Figures 4(b) and 4(c)), the image contains random pattern which is far different from the correct one. With a comparison result in the range of (Figure 4(d)), the image contains similar pattern to the correct one, but information in some parts is lost. With a comparison result in the range of or smaller, the generated image contains almost the same pattern as the correct one.

Note that the DI value is calculated from algebraic operations on the two images you compare with. The magnitude of DI value is only a relative indication of the difference between the two images. The actual usage of the DI value is to figure out the boundary between the images that contains mostly noises and the images that provide useful patterns of the earth model. From the samples shown in Figure 7, in this specific case, the DI value of is a good guidance value for acceptable accuracy of the design. From the bit-width exploration results shown in Section 4, we can see that the DI value of also happens to be a precision threshold, where the image turns from noise into accurate pattern with the increase of bit width.

3.5. Number Representation Exploration

Based on all the above modules, we can now perform exploration of different number representations for the FPGA implementation of a specific piece of Fortran code.

The current tools support two different number representations, fixed-point, and floating-point numbers (the value simulator for LNS is still in progress). For all the different number formats, the users can also specify arbitrary bit widths for each different variable.

There are usually a large number of different variables involved in one circuit design. In our previous work, we usually apply heuristic algorithms, such as ASA [19], to find out a close-to-optimal set of bit-width values for different variables. The heuristic algorithms may require millions of test runs to check whether a specific set of values meet the constraints or not. This is acceptable when the test run is only a simple error function and can be processed in nanoseconds. In our seismic processing application, depending on the problem size, it takes half an hour to several days to run one test set and achieve the result image. Thus, heuristic algorithms become impractical.

A simple and straightforward method to solve the problem is to use uniform bit width over all the different variables, and either iterate over a set of possible values or use a binary search algorithm to jump to an appropriate bit-width value. Based on the range information and the internal behavior of the program, we can also try to divide the variables in the target Fortran code into several different groups, and assign a different uniform bit width for each different group. For instance, in the FK step, there is a clear boundary that the first half performs square, square root, and division operations to calculate an integer value, and the second half uses the integer value as a table index, and performs sine, cosine, and complex multiplications to get the final result. Thus, in the hardware circuit design, we divide the variables into two groups based on which half it belongs to. Furthermore, in the second half of the function, some of the variables are trigonometric values in the range of , while the other variables represent the seismic image data and scale up to . Thus, they can be further divided into two parts and assigned bit widths separately.

4.1. Brief Introduction

The code shown in Algorithm 2 is the computationally intensive portion of the FK step in a downward continued-based migration. The governing equation for the FK step is the double square root equation (DSR) [20]. The DSR equation describes how to downward continue a wave-field one depth step. The equation is valid for a constant velocity medium and is based on the wave number of the source and receiver . The DSR equation can be written as (1), where is the frequency. The code takes the approach of building a priori a relatively small table of the possible values of . The code then performs a table lookup that converts a given value to an approximate value of the square root.

Algorithm 2: The code for the major computations of the FK step.

! generation of table step%ctable

do i = 1, size (step%ctable)

∗ step%dstep ∗ dsr%phase (i)

step%ctable (i) = dsr%amp (i) ∗ cmplx (cos ( ), sin ( ))

end do

! the core part of function wei_wem

do i4 = 1, size (wfld, 4)

do i3 = 1, size (wfld, 3)

do i2 = 1, size (wfld, 2)

do i1 = 1, size (wfld, 1)

= sqrt (step% (i1, i3) ∗ ∗ 2 + step% (i2, i4)∗ ∗ 2)

itable = max (1, min (int (1 + k / ko /dsr%d), dsr%n))

wfld (i1, i2, i3, i4, i5) = wfld (i1, i2, i3, i4, i5) ∗ step%ctable (itable)

end do

end do

end do

end do

In practical applications, "wfld" contains millions of data items. The computation pattern of this function makes it an ideal target to map to a streaming hardware circuit on an FPGA.

4.2. Circuit Design

The mapping from the software code to a hardware circuit design is straightforward for most parts. Figure 5 shows the general structure of the circuit design. Compared with the software Fortran code shown in Algorithm 2, one big difference is the handling of the sine and cosine functions. In the software code, the trigonometric functions are calculated outside the five-level loop, and stored as a lookup table. In the hardware design, to take advantage of the parallel calculation capability provided by the numerous logic units on the FPGA, the calculation of the sine/cosine functions is merged into the processing core of the inner loop. Three function evaluation units are included in this design to produce values for the square root, cosine and sine functions separately. As mentioned in Section 3.2, all three functions are evaluated using degree-one polynomial approximation with 386 or 512 uniform segments:

(1)

The other task in the hardware circuit design is to map the calculation into arithmetic operations of certain number representations. Table 3 shows the value range of some typical variables in the FK step. Some of the variables (in the part of square root and sine/cosine function evaluations) have a small range within [0, 1], while other values (especially "wfld" data) have a wide range from to . If we use floating-point or LNS number representations, their wide representation ranges are enough to handle these variables. However, if we use fixed-point number representations in the design, special handling is needed to achieve acceptable accuracy over wide ranges.

Variable	Step%	ko	wfld_real	wfld_img
Max	0.377	0.147	3.918e6	3.752e6
Min	0	7.658	4.168

The first issue to consider in fixed-point designs is the enlarged error caused by the division after the evaluation of the square root ( ). The values of , and come from the software program as input values to the hardware circuit, and contain errors propagated from previous calculations or caused by the truncation/rounding into the specified bit width on hardware. Suppose the error in the square root result is , and the error in variable is , assuming that the division unit itself does not bring extra error, the error in the division result is given by . According to the profiling results, holds a dynamic range from to , and has a maximum value of (variables step% and step% have similar ranges). In the worst case, the error from can be magnified by 70 times, and the error from magnified by approximately 9000 times.

To solve the problem of enlarged errors, we perform shifts at the input side to keep the three values , and in a similar range. The variable is shifted by the distance so that the value after shifting falls in the range of . The variables and are shifted by another distance so that the larger value of the two also falls in the range of . The difference between and is recorded so that after the division, the result can be shifted back into the correct scale. In this way, the has a range of and has a range of . Thus, the division only magnifies the errors by an order of 3 to 6. Meanwhile, as the three variables , and are originally in single-precision floating-point representation in software, when we pass their values after shifts, a large part of the information stored in the mantissa part can be preserved. Thus, a better accuracy is achieved through the shifting mechanism for fixed-point designs.

Figure 6 shows experimental results about the accuracy of the table index calculation when using shifting compared to not using shifting, with different uniform bit widths. The possible range of the table index result is from 1 to 2001. As it is the index for tables of smooth sequential values, an error within five indices is generally acceptable. We use the table index results calculated with single-precision floating-point as the true values for error calculation. When the uniform bit width of the design changes from 10 to 20, designs using the shifting mechanism show a stable maximum error of 3, and an average error around 0.11. On the other hand, the maximum error of designs without shifting vary from 2000 to 75, and the average errors vary from approximately 148 to 0.5. These results show that the shifting mechanism provides much better accuracy for the part of the table index calculation in fixed-point designs.

The other issue to consider is the representation of "wfld" data variables. As shown in Table 3, both the real and imaginary parts of "wfld" data have a wide range from to . Generally, fixed-point numbers are not suitable to represent such wide ranges. However, in this seismic application, the "wfld" data is used to store the processed image information. It is more important to preserve the pattern information shown in the data values rather than the data values themselves. Thus, by omitting the small values and using the limited bit width to store the information contained in large values, fixed-point representations still have a big chance to achieve accurate image in the final step. In our design, for convenience of bit-width exploration, we scale down all the "wfld" data values by a ratio of so that they fall into the range of [0,1).

4.3. Bit-Width Exploration Results

The original software Fortran code of the FK step performs the whole computation using single-precision floating-point. We firstly replace the original Fortran code of the FK step with a piece of C++ code using double-precision floating-point to generate a full-precision image to compare with. After that, to investigate the effect of different number representations for variables in the FK step on the accuracy of the whole application, we replace the code of the FK step with our simulation code that can be configured with different number representations and different bit widths and generate results for different settings. The approach for accuracy evaluation, introduced in Section 3.4, is used to provide DI values that indicate the differences in the patterns of the resulted seismic images from the pattern in full-precision image.

4.4. Hardware Acceleration Results

The hardware acceleration tool used in this project is the FPGA computing platform MAX-1, provided by Maxeler Technologies [21]. It contains a high-performance Xilinx Virtex IV FX100 FPGA, which consists of 42176 slices, 376 BRAMs, and 192 embedded multipliers. Meanwhile, it provides a high-bandwidth interface of PCI Express X8 (2 G bytes per second) to the software side residing in CPUs.

Based on the exploration results of different number representations, the fixed-point design with bit widths of 12, 16, and 16 for three different parts is selected in our hardware implementation. The design produces images containing the same pattern as the double-precision floating-point implementation, and has the smallest bit-width values, that is, the lowest resource cost among all the different number representations.