Video rental giants like Blockerbuster and Hollywood were swept away in a streaming flood of on-demand cable offerings. In a second wave, cable has become inundated by Netflix, Hulu and other over-the-top entertainment streams.
On-demand video streaming on a massive scale has unquestionably transformed the way we experience movies and other video offerings. At the same time, despite considerable investment and effort, video streaming has failed to give us practical video gaming as-a-service.
The limited market penetration of video-based cloud gaming is in part a direct consequence of the high cost of running the games on specialized hardware in the data center. Cloud gaming providers must pass these operating costs on to gamers, who can often realize a much better price-performance by running their game locally, e.g. on a game console.
In addition, certain intrinsic characteristics of video streaming limit the transmission efficiency and real-time quality of experience that can be achieved using video for cloud gaming.
As a result of these factors, the potential benefits of ‘game streaming’: instant access, no client patching, no load times, and no piracy, have not been realized by video-based cloud gaming services.
In this blog, we look at the history of cloud gaming and examine in some detail why a video stream is not the best way to convey a real-time gaming experience.
We suggest how a new type of content stream is better suited to delivering the high quality of experience that gamers expect, at a cost that will make games-as-a-service practical.
Video-Based Cloud Gaming: Barely a Trickle.
In 2009, OnLive announced the first-ever cloud gaming service. OnLive used special game server hardware to run the entire game in centralized data centers and stream the live, compressed video to the user. At launch in 2010, OnLive suggested that its video-based game streaming service would represent the “future of gaming”. However, the high cost of specialized game servers, together with limited image quality and input lag did not allow the cloud gaming service to scale. After earlier investments in OnLive by Time Warner, Autodesk, and others, a $40M investment by HTC indicated a $1.8B valuation in February 2011. But in August 2012, OnLive filed for Assignment for Benefit of Creditors in California, effectively eliminating virtually all shareholder value.
Early indications were that that the video-based cloud gaming approach did not deliver a good quality of experience or value proposition for users. In an extensive review of OnLive (Digital Foundry vs OnLive July 9, 2010) which included a careful frame-by-frame analysis of performance, Digital Foundry concluded:
“…the bottom line is that [OnLive] gameplay experience is not better than what we already have - by and large it's tangibly worse. The varying quality of the graphics is questionable, and the lag is best described as "better than expected" - nowhere near the claims that have been made for the system, and still measurably inferior to current standards.”
Problems with video-based cloud gaming systems first exposed by OnLive have continued to limit the price-performance of cloud gaming services. Even five years following the initial failure of OnLive, more capable game server hardware and higher connection speeds have not made video-based cloud gaming systems a good value proposition for end users.
Nvidia’s GeForce Now is a video-based cloud gaming service that has been limited to users of its $199 Shield or $299 Shield Pro devices. The service includes access to 60, mostly older, game titles for $7.99 per month subscription. Another 40 or so games are available for purchase through the service. Although Nvidia has signaled intention to make GeForce Now available to PC and Mac users, the service has failed to emerge from beta for these and other platforms since it was launched in 2013.
In early 2017, Nvidia once again announced the possibility of making the GeForce Now video-based game streaming available to PC and Mac users. However, Nvidia indicated that the cost of the service for users would be $1.25-$2.50 per hour charged to users to play games that they already own. The $2.50 per hour fee was anticipated for the highest quality service, which is limited to 1080p at 60fps. Of course, this resolution and frame rate is generally achievable for most games by current-generation game console hardware that typically costs $300-350, making the expensive GeForce Now streaming solution a limited value proposition for mainstream gamers.
Moreover, PC gamers can currently use commodity graphics cards costing less than $300 (from Nvidia and AMD) to run these games at even higher resolutions and frame rates. To receive the GeForce 1080p, 60 fps cloud gaming service, each user must be connected to a dedicated Nvidia Grid Server unit including a GPU and at least 2 CPUs, and costing approximately $3000. These devices also have very high per-user power and cooling costs in the data center. The high proposed hourly cost of the Nvidia GeForce Now service is a direct consequence of the high cost of the server-side equipment required to run and render the game, and compress the video frames in real time.
Nvidia also recommended a minimum connection bandwidth of 50 Megabits/s to support 1080p cloud gaming. This relatively high bandwidth requirement results because the necessary low-latency video encoding is typically much less efficient than higher-latency encoding using for movies or TV broadcasts. This inefficiency results in less compression, and a need for higher connection speed. Such connection speeds can be difficult to maintain continuously, and even small amounts of network jitter or packet delay can result in dropped frames, since unlike pre-encoded Netflix movie streams, a cloud gaming video stream is effectively unbuffered. (See section “Why Video Streams used by Netflix and Twitch Don’t Work Well for Cloud Gaming”)
By May 2017, GeForce Now had still not emerged from closed beta, and NVIDIA CEO Jen-Hsun Huang provided an update on the service during an investor Q&A. Huang stated that he was
“…looking forward to people trying it. But the important thing to realize about that is that’s still years away from really becoming a major gaming service. And it’s still years away from being able to find the right balance between cost and quality of service and the pervasiveness of virtualizing a gaming PC.”
PlayStation Now is a video-based cloud gaming service that offers a lower quality of service than that targeted by GeForce Now. Sony’s PlayStation Now service offers a $19.99 per month subscription to a library of mostly PS3 games playable in 720p at 30 frames per second. These PS3 titles are between 5-11 years old, and run natively at 720p 30fps, meaning that they can be streamed using less expensive game servers and lower connection speeds. (Typically, one-fourth of the resources required for 1080p 60fps cloud gaming). Sony announced the PlayStation Now service prior to the release of the PS4 in 2013. The service was developed to allow PS4 customers to play PS3 games, since PS3 games were otherwise incompatible with the new PS4 system. In early 2017, Sony announced that it was scaling back PlayStation Now service and removing a number of previously supported client devices. With the loss of these devices, PlayStation Now will only be available on PS4 and PC. http://www.gamesradar.com/sonys-scaling-back-playstation-now-so-lets-look-back-at-all-the-stuff-its-quietly-killed-off/
GPEG: A New Stream for Cloud Gaming
GPEG is a game engine middleware protocol that implements a new type of stream for cloud gaming. GPEG specifies a well buffered content stream of pre-encoded Visibility Event packets. These packets are encoded offline, and can be streamed without special game server hardware. The prefetched GPEG packets can be rendered by the client game engine at the highest image quality and frame rate.
GPEG incorporates unique methods of conservative visibility precomputation and visibility cache management to overcome the limitations of video streaming and realize the potential of low-cost, on-demand gaming.
Why Video Streams used by Netflix and Twitch Don’t Work Well for Cloud Gaming
When a game engine is running, everything happens fast. Sixty times a second fast for image generation.
But video encoding works much better when the encoder can take its time.
In normal video encoding, for example of ‘live’ sports, expensive broadcast encoders delay the stream in order to build up a larger group of pictures (GOP) within the encoder. A larger group of pictures allows better compression, since the encoder is better able to identify and exploit the slight differences between base frames (called I-frames) and surrounding interpolated frames (P-frames). Modern H.264 encoders derive much of their efficiency from using groups of forward frames (B-frames) that require additional encoding delay.
Even broadcast quality encoders costing thousands of dollars require encoding latencies of 100-150 ms. While this brief delay would not be noticeable to a someone watching a NFL broadcast or Twitch stream, this amount of latency would make a cloud gaming system unplayable. And, any network latency adds additional lag.
To overcome this encoding latency effect, cloud gaming systems employ video encoding profiles that use small GOPs and avoid B-frames. This results in inefficient encoding which produces high bitrate requirements, compression artifacts, or both.
Another downstream consequence of keeping latency low is that unlike Netflix, Twitch, and most other video streams, cloud gaming video streams have virtually no client-side buffering. For well buffered Netflix video streams, this lag is not experienced, and allows such streams to be buffered against packet loss and network jitter. But for cloud gaming video streams, any attempt to buffer frames on the client prior to display incurs an immediate lag penalty. This lack of buffering makes cloud gaming video streams very susceptible to packet loss and even small amounts of network jitter.
Figure 1.1 shows a typical MPEG video packet trace comprised of approximately 8-10 P-frames for each I-frame (base frame). Without client-side buffering, even a slightly delayed I-frame would typically result in dropping at least two consecutive frames, since a (smaller) P-frame packet cannot generally be displayed until the entire preceeding I-frame packet has been received and processed. This lack of buffering is the Achilles heel of cloud gaming video streams. Trying to reduce the interframe dependency (e.g. by using very few I-frames) only increases bitrate requirements, making packet delay even more likely.
For streaming movies, which are not interactive, video is good medium. For these applications encoding takes advantage of determinism in the sequence to achieve good compression, and the encoding only needs to be done once, amortizing the encoding cost over all users. Server costs for streaming movies are very low, because of the simplicity of the packet streaming process. Unfortunately, few of these strengths of video streaming can be exploited for cloud gaming. In contrast, cloud gaming exposes the underlying weaknesses of video for conveying real-time interactive experiences.