Skip to main content
EURASIP Journal on Embedded Systems
Download PDF

Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs

EURASIP Journal on Embedded Systems volume 2007, Article number: 049236 (2007) Cite this article

Abstract

We develop a design methodology for mapping computer vision algorithms onto an FPGA through the use of coarse-grain reconfigurable dataflow graphs as a representation to guide the designer. We first describe a new dataflow modeling technique called homogeneous parameterized dataflow (HPDF), which effectively captures the structure of an important class of computer vision applications. This form of dynamic dataflow takes advantage of the property that in a large number of image processing applications, data production and consumption rates can vary, but are equal across dataflow graph edges for any particular application iteration. After motivating and defining the HPDF model of computation, we develop an HPDF-based design methodology that offers useful properties in terms of verifying correctness and exposing performance-enhancing transformations; we discuss and address various challenges in efficiently mapping an HPDF-based application representation into target-specific HDL code; and we present experimental results pertaining to the mapping of a gesture recognition application onto the Xilinx Virtex II FPGA.

[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]

References

  1. Sriram S, Bhattacharyya SS: Embedded Multiprocessors: Scheduling and Synchronization. Marcel Dekker, New York, NY, USA; 2000.

    Google Scholar 

  2. Sen M, Bhattacharyya SS, Lv T, Wolf W: Modeling image processing systems with homogeneous parameterized dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '05), March 2005, Philadelphia, Pa, USA 5: 133-136.

    Google Scholar 

  3. Bhattacharyya SS, Leupers R, Marwedel P: Software synthesis and code generation for signal processing systems. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 2000,47(9):849-875. 10.1109/82.868454

    Article  Google Scholar 

  4. Lee E, Messerschmitt D: Synchronous data flow. Proceedings of the IEEE 1987,75(9):55-64. 10.1109/PROC.1987.13876

    Article  Google Scholar 

  5. Bilsen G, Engels M, Lauwereins R, Peperstraete J: Cyclo-static dataflow. IEEE Transactions on Signal Processing 1996,44(2):397-408. 10.1109/78.485935

    Article  Google Scholar 

  6. Buck JT: A dynamic dataflow model suitable for efficient mixed hardware and software implementations of DSP applications. Proceedings of the 3rd International Workshop on Hardware/Software Codesign (CODES '94), September 1994, Grenoble, France 165-172.

    Chapter  Google Scholar 

  7. Bhattacharya B, Bhattacharyya SS: Parameterized dataflow modeling for DSP systems. IEEE Transactions on Signal Processing 2001,49(10):2408-2421. 10.1109/78.950795

    MathSciNet  Article  Google Scholar 

  8. Gokhale M, Stone J, Arnold J, Kalinowski M: Stream-oriented FPGA computing in the Streams-C high level language. Proceedings of IEEE Symposium on Field-Programmable Custom Computing Machines, April 2000, Napa Valley, Calif, USA 49-56.

    Google Scholar 

  9. Hoare CAR: Communicating Sequential Processes. Prentice-Hall, New York, NY, USA; 1985.

    MATH  Google Scholar 

  10. Chappell S, Sullivan C: Handel-C for co-processing & co-design of Field Programmable System on Chip. In White Paper. Celoxica, Oxford, UK; 2002.

    Google Scholar 

  11. Banerjee P, Bagchi D, Haldar M, Nayak A, Kim V, Uribe R: Automatic conversion of floating-point MATLAB programs into fixed-point FPGA based hardware design. Proceedings of the 41st Annual Design Automation Conference (DAC '04), June 2004, San Diego, Calif, USA 484-487.

    Google Scholar 

  12. Kienhuis B, Rijpkema E, Deprettere E: Compaan: deriving process networks from Matlab for embedded signal processing architectures. Proceedings of the 18th International Workshop on Hardware/Software Codesign (CODES '00), May 2000, San Diego, Calif, USA 13-17.

    Google Scholar 

  13. Kahn G: The semantics of simple language for parallel programming. Proceedings of IFIP Congress, August 1974, Stockholm, Sweden 471-475.

    Google Scholar 

  14. Lee EA: Multidimensional streams rooted in dataflow. Proceedings of the IFIP Working Conference on Architectures and Compilation Techniques for Fine and Medium Grain Parallelism, January 1993, Orlando, Fla, USA (23):295-306.

  15. Wolf W, Ozer B, Lv T: Smart cameras as embedded systems. Computer 2002,35(9):48-53. 10.1109/MC.2002.1033027

    Article  Google Scholar 

  16. Eker J, Janneck JW, Lee EA, et al.: Taming heterogeneity—the ptolemy approach. Proceedings of the IEEE 2003,91(1):127-144. 10.1109/JPROC.2002.805829

    Article  Google Scholar 

  17. Haim F, Sen M, Ko D-I, Bhattacharyya SS, Wolf W: Mapping multimedia applications onto configurable hardware with parameterized cyclo-static dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), May 2006, Toulouse, France 3: 1052-1055.

    Google Scholar 

  18. Horstmannshoff J, Grötker T, Meyr H: Mapping multirate dataflow to complex RT level hardware models. Proceedings of the International Conference on Application-Specific Systems, Architectures and Processors (ASAP '97), July 1997, Zurich, Switzerland 283-292.

    Chapter  Google Scholar 

  19. Sen M, Bhattacharyya SS: Systematic exploitation of data parallelism in hardware synthesis of DSP applications. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 5: 229-232.

    Google Scholar 

  20. Williamson MC, Lee EA: Synthesis of parallel hardware implementations from synchronous dataflow graph specifications. Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers, November 1996, Grove, Calif, USA 2: 1340-1343.

    Google Scholar 

  21. Data-sheet for ZBT memory, http://www.samsung.com/Products/

  22. OpenCores Organization : WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores. revision B.3, September 2002, http://www.opencores.org

  23. Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, New York, NY, USA; 1989.

    MATH  Google Scholar 

  24. IEEE Working Group http://www.eda.org/vhdl-200x/vhdl-200x-ft/packages/files.html

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, 20742, USA

    Mainak Sen, Ivan Corretjer, Fiorella Haim, Sankalita Saha & Shuvra S Bhattacharyya

  2. Department of Electrical Engineering, Princeton University, Princeton, NJ, 08544, USA

    Jason Schlessman, Tiehan Lv & Wayne Wolf

Authors
  1. Mainak Sen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ivan Corretjer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fiorella Haim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sankalita Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jason Schlessman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tiehan Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shuvra S Bhattacharyya
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wayne Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mainak Sen.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Sen, M., Corretjer, I., Haim, F. et al. Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs. J Embedded Systems 2007, 049236 (2007). https://doi.org/10.1155/2007/49236

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/49236

Keywords

  • Design Methodology
  • Processing Application
  • Electronic Circuit
  • Gesture Recognition
  • Mapping Computer