The Next Generation of In-home Streaming: Light Fields, 5K, 10 GbE, and Foveated Compression

Abstract

Interacting with real-time rendered 3D content from powerful machines on smaller devices is becoming ubiquitous through commercial products that enable in-home streaming within the same local network. However, support for high resolution, low latency in-home streaming at high image quality is still a challenging problem. To enable this, we enhance an existing open source framework for in-home streaming. We add highly optimized DXT1 (DirectX Texture Compression) support for thin desktop and notebook clients. For rendered light fields, we improve the encoding algorithms for higher image quality. Within a 10 Gigabit Ethernet (10 GbE) network, we achieve streaming up to 5K resolution at 55 frames per second. Through new low-level algorithmic improvements, we increase the compression speed of ETC1 (Ericsson Texture Compression) by a factor of 5. We are the first to bring ETC2 compression to real-time speed, which increases the streamed image quality. Last, we reduce the required data rate by more than a factor of 2 through foveated compression with real-time eye tracking.