EURASIP Journal on Embedded Systems

EURASIP Journal on Embedded Systems Cover Image
Research Article
Open Access

Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs

  • Mainak Sen1Email author,
  • Ivan Corretjer1,
  • Fiorella Haim1,
  • Sankalita Saha1,
  • Jason Schlessman2,
  • Tiehan Lv2,
  • Shuvra S Bhattacharyya1 and
  • Wayne Wolf2
EURASIP Journal on Embedded Systems20072007:049236

https://doi.org/10.1155/2007/49236

©  Mainak Sen et al. 2007

Received: 1 May 2006

Accepted: 9 October 2006

Published: 28 January 2007

Abstract

We develop a design methodology for mapping computer vision algorithms onto an FPGA through the use of coarse-grain reconfigurable dataflow graphs as a representation to guide the designer. We first describe a new dataflow modeling technique called homogeneous parameterized dataflow (HPDF), which effectively captures the structure of an important class of computer vision applications. This form of dynamic dataflow takes advantage of the property that in a large number of image processing applications, data production and consumption rates can vary, but are equal across dataflow graph edges for any particular application iteration. After motivating and defining the HPDF model of computation, we develop an HPDF-based design methodology that offers useful properties in terms of verifying correctness and exposing performance-enhancing transformations; we discuss and address various challenges in efficiently mapping an HPDF-based application representation into target-specific HDL code; and we present experimental results pertaining to the mapping of a gesture recognition application onto the Xilinx Virtex II FPGA.

[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]

Authors’ Affiliations

(1)
Department of Electrical and Computer Engineering, University of Maryland
(2)
Department of Electrical Engineering, Princeton University

References

  1. Sriram S, Bhattacharyya SS: Embedded Multiprocessors: Scheduling and Synchronization. Marcel Dekker, New York, NY, USA; 2000.Google Scholar
  2. Sen M, Bhattacharyya SS, Lv T, Wolf W: Modeling image processing systems with homogeneous parameterized dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '05), March 2005, Philadelphia, Pa, USA 5: 133-136.Google Scholar
  3. Bhattacharyya SS, Leupers R, Marwedel P: Software synthesis and code generation for signal processing systems. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 2000,47(9):849-875. 10.1109/82.868454View ArticleGoogle Scholar
  4. Lee E, Messerschmitt D: Synchronous data flow. Proceedings of the IEEE 1987,75(9):55-64. 10.1109/PROC.1987.13876View ArticleGoogle Scholar
  5. Bilsen G, Engels M, Lauwereins R, Peperstraete J: Cyclo-static dataflow. IEEE Transactions on Signal Processing 1996,44(2):397-408. 10.1109/78.485935View ArticleGoogle Scholar
  6. Buck JT: A dynamic dataflow model suitable for efficient mixed hardware and software implementations of DSP applications. Proceedings of the 3rd International Workshop on Hardware/Software Codesign (CODES '94), September 1994, Grenoble, France 165-172.View ArticleGoogle Scholar
  7. Bhattacharya B, Bhattacharyya SS: Parameterized dataflow modeling for DSP systems. IEEE Transactions on Signal Processing 2001,49(10):2408-2421. 10.1109/78.950795MathSciNetView ArticleGoogle Scholar
  8. Gokhale M, Stone J, Arnold J, Kalinowski M: Stream-oriented FPGA computing in the Streams-C high level language. Proceedings of IEEE Symposium on Field-Programmable Custom Computing Machines, April 2000, Napa Valley, Calif, USA 49-56.Google Scholar
  9. Hoare CAR: Communicating Sequential Processes. Prentice-Hall, New York, NY, USA; 1985.MATHGoogle Scholar
  10. Chappell S, Sullivan C: Handel-C for co-processing & co-design of Field Programmable System on Chip. In White Paper. Celoxica, Oxford, UK; 2002.Google Scholar
  11. Banerjee P, Bagchi D, Haldar M, Nayak A, Kim V, Uribe R: Automatic conversion of floating-point MATLAB programs into fixed-point FPGA based hardware design. Proceedings of the 41st Annual Design Automation Conference (DAC '04), June 2004, San Diego, Calif, USA 484-487.Google Scholar
  12. Kienhuis B, Rijpkema E, Deprettere E: Compaan: deriving process networks from Matlab for embedded signal processing architectures. Proceedings of the 18th International Workshop on Hardware/Software Codesign (CODES '00), May 2000, San Diego, Calif, USA 13-17.Google Scholar
  13. Kahn G: The semantics of simple language for parallel programming. Proceedings of IFIP Congress, August 1974, Stockholm, Sweden 471-475.Google Scholar
  14. Lee EA: Multidimensional streams rooted in dataflow. Proceedings of the IFIP Working Conference on Architectures and Compilation Techniques for Fine and Medium Grain Parallelism, January 1993, Orlando, Fla, USA (23):295-306.Google Scholar
  15. Wolf W, Ozer B, Lv T: Smart cameras as embedded systems. Computer 2002,35(9):48-53. 10.1109/MC.2002.1033027View ArticleGoogle Scholar
  16. Eker J, Janneck JW, Lee EA, et al.: Taming heterogeneity—the ptolemy approach. Proceedings of the IEEE 2003,91(1):127-144. 10.1109/JPROC.2002.805829View ArticleGoogle Scholar
  17. Haim F, Sen M, Ko D-I, Bhattacharyya SS, Wolf W: Mapping multimedia applications onto configurable hardware with parameterized cyclo-static dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), May 2006, Toulouse, France 3: 1052-1055.Google Scholar
  18. Horstmannshoff J, Grötker T, Meyr H: Mapping multirate dataflow to complex RT level hardware models. Proceedings of the International Conference on Application-Specific Systems, Architectures and Processors (ASAP '97), July 1997, Zurich, Switzerland 283-292.View ArticleGoogle Scholar
  19. Sen M, Bhattacharyya SS: Systematic exploitation of data parallelism in hardware synthesis of DSP applications. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 5: 229-232.Google Scholar
  20. Williamson MC, Lee EA: Synthesis of parallel hardware implementations from synchronous dataflow graph specifications. Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers, November 1996, Grove, Calif, USA 2: 1340-1343.Google Scholar
  21. Data-sheet for ZBT memory, http://www.samsung.com/Products/
  22. OpenCores Organization : WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores. revision B.3, September 2002, http://www.opencores.org
  23. Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, New York, NY, USA; 1989.MATHGoogle Scholar
  24. IEEE Working Group http://www.eda.org/vhdl-200x/vhdl-200x-ft/packages/files.html

Copyright

© Mainak Sen et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.