Bandwidth Challenge

Abstract

With the high powered rendering systems and recording equipment available today, 4k video streams are now within the reach of several scientific communities. A resolution of 4096x3072, with 24-bit color, running at 30 frames per second, produces just under 10Gbs of network traffic. Existing tiled displays are being used to show such media, and these high resolution streams are being sent across the country. Because of the data throughput required for 4k video streams, pulling stored video off storage systems in real time is not possible. So current systems providing 4k video streams either use a live source, such as a high-definition camera, or use lossy compression to store the video and transmit it using 1Gb networks. Compression requires specialized high performing equipment on the receiving end and can introduce artifacts in the appearance of the data. We propose the construction of a system to stream stored media (either recorded video or rendered graphics) at full non-compressed resolution to tiled displays.

Laura Arns ,Ryan Pedela, Michael Shuey, Preston Smith, Jenett Tillotson,
Tom Hacker, Dave Braun
Envision Center & Rosen Center for Advanced Computing - Purdue University

Background

There are currently existing movie players for tiled displays, such as Blockbuster and PixelBlaster [Blockbuster06, PixelBlaster06]. These generally do not stream the video but instead store smaller individual movies on the PC for each tile so that each PC only needs to display its own small local file. Such an approach does not scale well to movies of great length or resolution, unless each PC has a great deal of local storage space. Also, if the video is to be ported to another display with a different number of tiles or different resolution, the individual smaller movies must be recreated based on these new sizes and restored on the new system. Another approach is to store only a single high resolution version of the movie, then stream it to each playback node. The nodes then break up the incoming stream into smaller sized units, with each node storing only the portion of each image that it should display. Because of bandwidth limitations of the network and from storage, this results in a much slower than realtime framerate. However, since the nodes store each of the smaller images as they are received, the movie can then be replayed at real speed by using this local data.

One possibility to get around the network and storage limitations is to stream compressed video streams. This decreases the amount of data that must be read from storage and sent over the network. Approaches such as this [AT &T02] can stream higher resolution data at a higher framerate. However, compression many be lossy, resulting in visual artifacts in the image. Or, the compression algorithm may require expensive specialized hardware at the server or client, or both [Yamaguchi05].

System

Future Work

The current system uses our own custom built streaming server and player. We would eventually like to integrate our system with open source video streaming projects in order to support a much wider range of possible formats and to make the system more accessible to other sites. We would also like to add a permanent streaming server resource available to remote sites via TeraGrid or similar systems. Because each site may have a different sized tiled display (3x4, 4x6, etc), it will be necessary to create software to automatically subdivide the stored 4K frames into frames of the correct size for each tile. A simple front-end web portal would be created to allow users to pull the media from the archive system and prepare it for their particular tiled wall and schedule the system for subsequent broadcasts. We would also develop a control system allowing users to pause, rewind, and fast-forward the video streams.

This system could also be used to stream compressed video to sites that have the specialized hardware required to decompress the streams in real time. The stream servers could be used as a cluster to handle the compression and currently cluster-based decoders and special purpose boards are being developed that can handle the decompression in real time. Such a system could explore the relationship between data integrity, data throughput, compression, and bandwidth.

If this initiative results in a popular service, this system may be required to store many different stored medias preprocessed in may different ways. Managing these copies would incur substantial overhead, especially if the broadcast system continues to employ directly-connected storage elements. Instead, a large-scale system should employ a high-performance cluster file system attached to a cluster of stream servers with equal access to all files. This would eliminate the data management overhead, and should provide the basis for a broadcast system that can be extended linearly with only the addition of more stream servers.

Because of the bandwidth limitations of a single storage device, the key to maintaining the data throughput needed for high resolution at the frame rates required for video is to split the recorded video across several storage elements each connected to a stream server. One storage element and one 1Gb network connection is required for each tile of the remote system. Each storage element then supplies only a fraction of the total necessary data throughput and each network connection supplies a fraction of the network bandwidth. Although a cluster file system would offer the necessary bandwidth to storage, such a solution requires a larger investment in time and resources than our system. Although such an investment could be worthwhile for a large user base, this initial work is intended to create a resource for a reasonably small number of users, which could be easily recreated at other sites.

This setup takes advantage of the tiled walls already available at many sites, where each tile of the display is connected to a single PC with a 1Gb network connection. It requires only that the receiving sites add a 10Gb switch and network connection to their existing infrastructure. We have developed both the server and client applications for sending and receiving video frames, and synchronizing their playback across a tiled display.

Our current server hardware configuration consists of 6 dual Opteron servers with PCI-x buses, connected to 6 Apple Xserve RAIDs via 6 Apple FibreChannel cards. Each server machine also has two 1Gb network connections to a 10Gb switch. This allows the system to pull the video frames off the Xserve RAIDs at a high enough rate for playback. The frames are then sent over the 10Gb network to our tiled display wall. The tiled wall consists of 12 tiles, each driven by a single PC with a single 1Gb network card, connected to a 10Gb switch. These PCs have video cards that incorporate hardware genlock (nVidia Quadro FX 4500Gs), which allows us to synchronize the playback of all the individual streams. Two receiving tiles connect to each of the streaming servers. ?The streaming servers transmit video frames as quickly as possible, and the receiving PCs buffer the frames until all machines are ready to display the next frame.

References

[AT&T02] AT&T Labs Research, white paper, Han Chen, Kai Li, Bin Wei: A Parallel Ultra-High Resolution MPEG-2 Video Decoder for PC Cluster Based Tiled Display Systems. IPDPS 2002.
[Blockbuster06] Blockbuster Verified August 2006.
[PixelBlaster06] PixelBlaster Verified August 2006.
[Yamaguchi05] Yamaguchi, T.; Nomura, M.; Shirakawa, K.; Fujii, T. SHD movie distribution system using image container with 4096x2160 pixel resolution and 36 bit color., 2005. ISCAS 2005. IEEE International Symposium on Circuits and Systems. 23-26 May 2005 Page(s):5918 - 5921 Vol. 6.