Google kindly gave me access to their Google Earth Studio platform a few months back. I recently used it to create a variety of videos including a 2D 360 interactive video tour of my home city of Chester in England amongst other tours including London and Snowdonia National Park.

Familiar Tools And Techniques

Google Earth studio is relatively easy to work with if you are familiar with video editing or animation software. Camera controls including field of view, location, tilt and roll all have similarity to the type of keyframing you would find in Adobe software such as After Effects, Premier Pro and Character Animator. It’s fairly easy to create animated sequences and produce some interesting videos using areas of the map where 3D data is available. Google have only recently released 3D data for Chester…as its a beautiful city and familiar territory, this made a great focal point for some further exploration of the tool. In addition, I added some ‘tilt shift’ effects to Birmingham and Barcelona sequences to create a ‘miniature’ effect.

Exporting Videos

Once you have established the camera path for your sequence using the keyframing tools, it is possible to export traditional video or equirectangular video suitable for upload to YouTube – this process is either completed in the cloud (via Google servers) or by export as individual jpeg frames (which at 50fps is time consuming). Note that the limit per day for processing frames is around 15000. Having uploaded, YouTube then translates the data into a 360 video capable of being controlled with a mouse or TV remote.

Render Quality

Having uploaded 360 videos to YouTube the first things you notice are clarity, resolution and aliasing issues. Essentially, the 3D render contains moire patterns and flicker on buildings with lack of definition on hard edges. This reduced the quality of the viewing experience considerably. However, using Topaz’s video enhancement software it was possible to upscale the rendered video to 8K. When re-uploaded to YouTube the conversion was much sharper with improvements in antialiasing and fine detail.

Notes

• Not all of Google’s coverage features 3D photogrammetry data. The major cities of the world and sites of interest are covered, but there are still many locations with 2D imagery only
• If you’re interested in alternativeS then maybe try Geolayers, a plugin for After Effects. It allows more advanced visual techniques for animating maps with After Effects suite of effects and camera attributes such as depth of field.
• See my previous Google Earth Studio experiments

Here’s a more detailed summary from a LinkedIn post by Arkadiusz Szadkowski, Senior Business Development Manager in Reality Mapping, Imagery & Remote Sensing, Esri:

In volume rendering, you deal with a 3D grid of samples (like the pixels in a 2D image, but in 3D space). For each volume sample, you project it onto the 2D viewing plane. Think of this as determining where on your screen this 3D point should appear. When you splat the projected point onto the 2D viewing plane, instead of simply putting a dot there, you “spread” its value using a Gaussian curve. The center of the splat (where the original projected point was) will have the highest value (the peak of the Gaussian), and the values will taper off as you move away from the center, following the Gaussian curve. As you project more and more points from the 3D volume onto the 2D viewing plane, some of their Gaussian splats will overlap. When this happens, the splats are combined based on their weights (from the Gaussian function) to produce a smooth and blended result.

(See LinkedIn article)

Online Apps

Poly.cam have introduced the option to create 3D Gaussian Splats on their website where up to 200 images can be uploaded to their servers for processing. They also published a link to this guide which provides hints and tips on how to create the best ‘splats’. Each 3D environment can be shared and, as mentioned, is built in real time in front of the viewer. See their gallery and try Gaussian Splatting for yourself by signing up at Poly.cam.

Luma.ai have recently launched an Interactive Scenes section on their website, allowing people to create their gaussian splats in a similar manner to poly.cam. Check out their gallery pages to see examples and sign up to try your own splat!

Cleaning Up The Splats

One key issue with Gaussian Splatting is that errors can, and often do, occur in the final cloud – typically leaving artifacts or ‘floaters’ in the air around your model. These are false depth measurements and are usually a result of elements of an image that are far away such as sky. Once a false measurement is recorded then other ‘splats’ will hit these, snowballing into a group of inaccurate splats. This can be frustrating when moving around the model as ‘floaters’ can disrupt or even obscure the view. In the video on the right Aras Pranckevičius guides us through the process of cleaning up a model within Unity using a Gaussian Splat editor he created which is available on GitHub. A degree of familiarity with Unity and coding is required to process your models in this manner.

Links

If you want to go into much more detail here are some links examining various aspects of Gaussian Splatting, NeRFs and beyond:

Overview of Gaussian Splatting from Medium.com
Aras Pranckevičius’s blog article 1
Aras Pranckevičius’s blog article 2
Differences between NeRFs and Gaussian Splatting by Medium.com
Neural Radience Field – an introduction to NeRFs

Relevant Projects

Simcoemedia created a visual handbook for Matterport during 2018. This involved a variety of experiments with their Pro2 camera designed to capture 3D spaces using infrared measurement and high resolution HDR image capture. See more information about this project.