Skip to main content

Advertisement

EURASIP Journal on Embedded Systems
EURASIP Journal on Embedded Systems Cover Image

Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs

EURASIP Journal on Embedded Systemsvolume 2007, Article number: 049236 (2007) | Download Citation

Article metrics

  • 1593 Accesses

  • 5 Citations

Abstract

We develop a design methodology for mapping computer vision algorithms onto an FPGA through the use of coarse-grain reconfigurable dataflow graphs as a representation to guide the designer. We first describe a new dataflow modeling technique called homogeneous parameterized dataflow (HPDF), which effectively captures the structure of an important class of computer vision applications. This form of dynamic dataflow takes advantage of the property that in a large number of image processing applications, data production and consumption rates can vary, but are equal across dataflow graph edges for any particular application iteration. After motivating and defining the HPDF model of computation, we develop an HPDF-based design methodology that offers useful properties in terms of verifying correctness and exposing performance-enhancing transformations; we discuss and address various challenges in efficiently mapping an HPDF-based application representation into target-specific HDL code; and we present experimental results pertaining to the mapping of a gesture recognition application onto the Xilinx Virtex II FPGA.

[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]

References

  1. 1.

    Sriram S, Bhattacharyya SS: Embedded Multiprocessors: Scheduling and Synchronization. Marcel Dekker, New York, NY, USA; 2000.

  2. 2.

    Sen M, Bhattacharyya SS, Lv T, Wolf W: Modeling image processing systems with homogeneous parameterized dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '05), March 2005, Philadelphia, Pa, USA 5: 133-136.

  3. 3.

    Bhattacharyya SS, Leupers R, Marwedel P: Software synthesis and code generation for signal processing systems. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 2000,47(9):849-875. 10.1109/82.868454

  4. 4.

    Lee E, Messerschmitt D: Synchronous data flow. Proceedings of the IEEE 1987,75(9):55-64. 10.1109/PROC.1987.13876

  5. 5.

    Bilsen G, Engels M, Lauwereins R, Peperstraete J: Cyclo-static dataflow. IEEE Transactions on Signal Processing 1996,44(2):397-408. 10.1109/78.485935

  6. 6.

    Buck JT: A dynamic dataflow model suitable for efficient mixed hardware and software implementations of DSP applications. Proceedings of the 3rd International Workshop on Hardware/Software Codesign (CODES '94), September 1994, Grenoble, France 165-172.

  7. 7.

    Bhattacharya B, Bhattacharyya SS: Parameterized dataflow modeling for DSP systems. IEEE Transactions on Signal Processing 2001,49(10):2408-2421. 10.1109/78.950795

  8. 8.

    Gokhale M, Stone J, Arnold J, Kalinowski M: Stream-oriented FPGA computing in the Streams-C high level language. Proceedings of IEEE Symposium on Field-Programmable Custom Computing Machines, April 2000, Napa Valley, Calif, USA 49-56.

  9. 9.

    Hoare CAR: Communicating Sequential Processes. Prentice-Hall, New York, NY, USA; 1985.

  10. 10.

    Chappell S, Sullivan C: Handel-C for co-processing & co-design of Field Programmable System on Chip. In White Paper. Celoxica, Oxford, UK; 2002.

  11. 11.

    Banerjee P, Bagchi D, Haldar M, Nayak A, Kim V, Uribe R: Automatic conversion of floating-point MATLAB programs into fixed-point FPGA based hardware design. Proceedings of the 41st Annual Design Automation Conference (DAC '04), June 2004, San Diego, Calif, USA 484-487.

  12. 12.

    Kienhuis B, Rijpkema E, Deprettere E: Compaan: deriving process networks from Matlab for embedded signal processing architectures. Proceedings of the 18th International Workshop on Hardware/Software Codesign (CODES '00), May 2000, San Diego, Calif, USA 13-17.

  13. 13.

    Kahn G: The semantics of simple language for parallel programming. Proceedings of IFIP Congress, August 1974, Stockholm, Sweden 471-475.

  14. 14.

    Lee EA: Multidimensional streams rooted in dataflow. Proceedings of the IFIP Working Conference on Architectures and Compilation Techniques for Fine and Medium Grain Parallelism, January 1993, Orlando, Fla, USA (23):295-306.

  15. 15.

    Wolf W, Ozer B, Lv T: Smart cameras as embedded systems. Computer 2002,35(9):48-53. 10.1109/MC.2002.1033027

  16. 16.

    Eker J, Janneck JW, Lee EA, et al.: Taming heterogeneity—the ptolemy approach. Proceedings of the IEEE 2003,91(1):127-144. 10.1109/JPROC.2002.805829

  17. 17.

    Haim F, Sen M, Ko D-I, Bhattacharyya SS, Wolf W: Mapping multimedia applications onto configurable hardware with parameterized cyclo-static dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), May 2006, Toulouse, France 3: 1052-1055.

  18. 18.

    Horstmannshoff J, Grötker T, Meyr H: Mapping multirate dataflow to complex RT level hardware models. Proceedings of the International Conference on Application-Specific Systems, Architectures and Processors (ASAP '97), July 1997, Zurich, Switzerland 283-292.

  19. 19.

    Sen M, Bhattacharyya SS: Systematic exploitation of data parallelism in hardware synthesis of DSP applications. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 5: 229-232.

  20. 20.

    Williamson MC, Lee EA: Synthesis of parallel hardware implementations from synchronous dataflow graph specifications. Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers, November 1996, Grove, Calif, USA 2: 1340-1343.

  21. 21.

    Data-sheet for ZBT memory, http://www.samsung.com/Products/

  22. 22.

    OpenCores Organization : WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores. revision B.3, September 2002, http://www.opencores.org

  23. 23.

    Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, New York, NY, USA; 1989.

  24. 24.

    IEEE Working Group http://www.eda.org/vhdl-200x/vhdl-200x-ft/packages/files.html

Download references

Author information

Affiliations

  1. Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, 20742, USA

    • Mainak Sen
    • , Ivan Corretjer
    • , Fiorella Haim
    • , Sankalita Saha
    •  & Shuvra S Bhattacharyya

  2. Department of Electrical Engineering, Princeton University, Princeton, NJ, 08544, USA

    • Jason Schlessman
    • , Tiehan Lv
    •  & Wayne Wolf

Authors

  1. Search for Mainak Sen in:

  2. Search for Ivan Corretjer in:

  3. Search for Fiorella Haim in:

  4. Search for Sankalita Saha in:

  5. Search for Jason Schlessman in:

  6. Search for Tiehan Lv in:

  7. Search for Shuvra S Bhattacharyya in:

  8. Search for Wayne Wolf in:

Corresponding author

Correspondence to Mainak Sen.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Keywords

  • Design Methodology
  • Processing Application
  • Electronic Circuit
  • Gesture Recognition
  • Mapping Computer