Achieving high quality, low latency rendering of complex visualisations on-demand to mobile devices is a tough challenge for Virtalis Reach. It’s easy to jump to the conclusion that it works like Cloud Gaming systems, but that’s not really true.
To understand the difference between Virtalis Reach and Cloud Gaming some background on Cloud and Mobile Gaming is required first.
Mobile Gaming
Mobile Gaming, in the scope of this discussion, means running games locally on a mobile device's built-in hardware.
In the past decade the processing capabilities of mobile devices has continued to increase exponentially. Although still far behind the capabilities of modern dedicated graphical workstations, mobile devices have the capability to run sophisticated games and they have their own dedicated graphics accelerators.
Cloud Gaming
Cloud Gaming has also been with us for over a decade. The attraction for publishers of having complete control of the distribution, deployment and licensing of their product, as well as full control over the hardware platform itself, is very strong.
For consumers the promise is on-demand access to powerful hardware and feature rich games that far exceed the capabilities of mobile devices.
The challenges of cloud gaming
Over the years solutions have risen and fallen, bitten by different bugs, such as persuading publishers to license compelling content to their platform, the ever falling cost and rising feature set of home hardware, and the surge in compelling mobile gaming content and affordable devices which can support them.
Those impediments are dwarfed however by Cloud Gaming's great Achilles' heel, the ability of the internet itself to deliver the content. The bandwidth and crucially the latency requirements for Cloud Gaming are still much too high for it deliver a compelling experience for the majority of modern games to the majority of consumers.
Ironically for games where compromises can be made they are often more effectively delivered by mobile gaming technology instead.
How Cloud Gaming Works
In simple terms the gaming PC that the player would have had at home is now located in a data center (the “cloud”) but the inputs and outputs are routed over the internet. The graphical output is sent as a video stream over the internet to the users device where it is decoded and displayed, input from the users device feeds back to the game running in the cloud.
This loop of input and output via the internet is the Achilles' heel we mentioned previously, on the home users gaming PC these streams are transmitted short distances over dedicated wires, whereas for the Cloud Gaming platforms they must traverse the internet.
Cloud Gaming as a subset of Computer Graphics Clusters
Cloud Gaming solutions can be considered as a specific subset of the more general field of computer graphics clusters which distribute the work of managing and rendering virtual environments over two or more discrete devices.
In the field of military and flight simulators, distributed war gaming systems combine multiple rendering devices with server systems which simulate the virtual battlefield. The rendering devices in such systems are anything but mobile but they are illustrative of a long history in computer graphics of distributing work over discrete hardware.
The practice of off-loading the management of a shared virtual environment to a central server is a common feature of multi-player online games however using cloud computing to accelerate rendering is still an uncommon and emerging technology. Previous solutions have limited attempts to off-loading CPU work for AI path-finding to free local compute resources or background rendering complex textures to supplement the rendering at the client.
Streaming GPU control language instructions
More similar in some regards to Cloud Gaming, some visualisation cluster solutions stream GPU control language instructions. The purpose of these systems is to allow a single application (usually a high-end CAD program) to be rendered simultaneously by several separate high-end graphics workstations thus yielding a very high resolution image and scaling the performance, this supports large scale design review and Virtual Reality use-cases.
Rather than having the remote game or application render the entire picture, instead the low level graphics instructions are captured, transmitted over a fast network, stored and repeated at the client devices. Like Cloud Gaming, these solutions are not able to deliver rich mobile content as they rely on a very fast network, also they require a powerful GPU in the client rendering devices.
Virtalis Reach Hybrid Mobile Renderer
Cloud and Mobile Gaming can be considered as the opposing points on an axis which defines how much of the game is processed on the cloud and how much on the mobile device. The rendering solution for Virtalis Reach is also a point along this axis but much closer to the middle which makes it a hybrid solution.
In our hybrid solution the computational work required to service and compose a virtual environment is handled by a cloud component but a second component on the mobile device renders the final picture using its own graphics capability.
The cloud component does not perform any 3D rendering and does not require an expensive graphics accelerator because the mobile device's own graphics accelerator is used at the very last stage of the rendering.
This is achievable because the cloud component performs the computationally expensive steps of determining what in the virtual environment is visible, the cloud component is supported by a data processing platform which has pre-optimised the data into many levels of detail so it can be optimally rendered. Rather than compressed video the cloud component sends very carefully curated chunks of data and a compact stream of what to render.
This gives the illusion that the mobile device is rendering a hugely complex environment when in fact it only has a very narrow and skilfully managed slice of the data and there is actually nothing on the mobile device beyond that which can be seen.
The latency associated with the video streaming used by Cloud Gaming is also not present, this is crucial for our solution which also supports XR because latency in XR will induce simulator sickness.