• Contact Us
  • Login
Upgrade
Tech News Hero
Advertisement
  • Home
  • News
  • Gadgets
  • Social
  • Gaming
  • Mobile
  • PC
  • Internet
  • Security
  • Apps
No Result
View All Result
  • Home
  • News
  • Gadgets
  • Social
  • Gaming
  • Mobile
  • PC
  • Internet
  • Security
  • Apps
No Result
View All Result
Tech News Hero
No Result
View All Result
Home Gaming

Meet the startup that helped Microsoft build the world of Flight Simulator – TechNewHero

by technewshero
December 14, 2020
in Gaming
0
Meet the startup that helped Microsoft build the world of Flight Simulator – TechNewHero
Share on FacebookShare on Twitter

Microsoft’s new Flight Simulator is a technological marvel that sets a new standard for the genre. But to recreate a world that feels real and alive and contains billions of buildings all in the right spots, Microsoft and Asobo Studios relied on the work of multiple partners.

One of those is the small Austrian startup Blackshark.ai from Graz that, with a team of only about 50 people, recreated every city and town around the world with the help of AI and massive computing resources in the cloud.

Ahead of the launch of the new Flight Simulator, we sat down with Blackshark co-founder and CEO Michael Putz to talk about working with Microsoft and the company’s broader vision.

Image Credits: Microsoft

Blackshark is actually a spin-off of game studio Bongfish, the maker of World of Tanks: Frontline, Motocross Madness and the Stoked snowboarding game series. As Putz told me, it was actually Stoked that set the company on the way to what would become Blackshark.

“One of the first games we did in 2007 was a snowboarding game called Stoked and S Stoked Bigger Edition, which was one of the first games having a full 360-degree mountain where you could use a helicopter to fly around and drop out, land everywhere and go down,” he explained. “The mountain itself was procedurally constructed and described — and also the placement of obstacles of vegetation, of other snowboarders and small animals had been done procedurally. Then we went more into the racing, shooting, driving genre, but we still had this idea of positional placement and descriptions in the back of our minds.”

Bongfish returned to this idea when it worked on World of Tanks, simply because of how time-consuming it is to build such a huge map where every rock is placed by hand.

Based on this experience, Bongfish started building an in-house AI team. That team used a number of machine-learning techniques to build a system that could learn from how designers build maps and then, at some point, build its own AI-created maps. The team actually ended up using this for some of its projects before Microsoft came into the picture.

“By random chance, I met someone from Microsoft who was looking for a studio to help them out on the new Flight Simulator. The core idea of the new Flight Simulator simulator was to use Bing Maps as a playing field, as a map, as a background,” Putz explained.

But Bing Maps’ photogrammetry data only yielded exact 1:1 replicas of 400 cities — for the vast majority of the planet, though, that data doesn’t exist. Microsoft and Asobo Studios needed a system for building the rest.

This is where Blackshark comes in. For Flight Simulator, the studio reconstructed 1.5 billion buildings from 2D satellite images.

Now, while Putz says he met the Microsoft team by chance, there’s a bit more to this. Back in the day, there was a Bing Maps team in Graz, which developed the first cameras and 3D versions of Bing Maps. And while Google Maps won the market, Bing Maps actually beat Google with its 3D maps. Microsoft then launched a research center in Graz and when that closed, Amazon and others came in to snap up the local talent.

“So it was easy for us to fill positions like a PhD in rooftop reconstruction,” Putz said. “I didn’t even know this existed, but this was exactly what we needed — and we found two of them.

“It’s easy to see why reconstructing a 3D building from a 2D map would be hard. Even figuring out a building’s exact outline isn’t easy.

Image Credits: Blackshark.ai

“What we do basically in Flight Simulator is we look at areas, 2D areas and then finding out footprints of buildings, which is actually a computer vision task,” said Putz. “But if a building is obstructed by a shadow of a tree, we actually need machine learning because then it’s not clear anymore what is part of the building and what is not because of the overlap of the shadow — but then machine learning completes the remaining part of the building. That’s a super simple example.”

While Blackshark was able to rely on some other data, too, including photos, sensor data and existing map data, it has to make a determination about the height of the building and some of its characteristics based on very little information.

The obvious next problem is figuring out the height of a building. If there is existing GIS data, then that problem is easy to solve, but for most areas of the world, that data simply doesn’t exist or isn’t readily available. For those areas, the team takes the 2D image and looks for hints in the image, like shadows. To determine the height of a building based on a shadow, you need the time of day, though, and the Bing Maps images aren’t actually timestamped. For other use cases the company is working on, Blackshark has that and that makes things a lot easier. And that’s where machine learning comes in again.

Image Credits: Blackshark.ai

“Machine learning takes a slightly different road,” noted Putz. “It also looks at the shadow, we think — because it’s a black box, we don’t really know what it’s doing. But also, if you look at a flat rooftop, like a skyscraper versus a shopping mall. Both have mostly flat rooftops, but the rooftop furniture is different on a skyscraper than on a shopping mall. This helps the AI to learn when you label it the right way.”

And then, if the system knows that the average height of a shopping mall in a given area is usually three floors, it can work with that.

One thing Blackshark is very open about is that its system will make mistakes — and if you buy Flight Simulator, you will see that there are obvious mistakes in how some of the buildings are placed. Indeed, Putz told me that he believes one of the hardest challenges in the project was to convince the company’s development partners and Microsoft to let them use this approach.

“You’re talking 1.5 billion buildings. At these numbers, you cannot do traditional Q&A anymore. And the traditional finger-pointing in like a level of Halo or something where you say ‘this pixel is not good, fix it,’ does not really work if you develop on a statistical basis like you do with AI. So it might be that 20% of the buildings are off — and it actually is the case I guess in the Flight Simulator — but there’s no other way to tackle this challenge because outsourcing to hand-model 1.5 billion buildings is, just from a logistical level and also budget level, not doable.”

Over time, that system will also improve, and because Microsoft streams a lot of the data to the game from Azure, users will surely see changes over time.

Image Credits: Blackshark.ai

Labeling, though, is still something the team has to do simply to train the model, and that’s actually an area where Blackshark has made a lot of progress, though Putz wouldn’t say too much about it because it’s part of the company’s secret sauce and one of the main reasons why it can do all of this with just about 50 people.

“Data labels had not been a priority for our partners,” he said. “And so we used our own live labeling to basically label the entire planet by two or three guys […] It puts a very powerful tool and user interface in the hands of the data analysts. And basically, if the data analyst wants to detect a ship, he tells the learning algorithm what the ship is and then he gets immediate output of detected ships in a sample image.”

From there, the analyst can then train the algorithm to get even better at detecting a specific object like a ship, in this example, or a mall in Flight Simulator. Other geospatial analysis companies tend to focus on specific niches, Putz also noted, while the company’s tools are agnostic to the type of content being analyzed.

Image Credits: Blackshark.ai

And that’s where Blackshark’s bigger vision comes in. Because while the company is now getting acclaim for its work with Microsoft, Blackshark also works with other companies around reconstructing city scenes for autonomous driving simulations, for example.

“Our bigger vision is a near-real-time digital twin of our planet, particularly the planet’s surface, which opens up a trillion use cases where traditional photogrammetry like a Google Earth or what Apple Maps is doing is not helping because those are just simplified for photos clued on simple geometrical structures. For this we have our cycle where we have been extracting intelligence from aerial data, which might be 2D images, but it also could be 3Dpoint counts, which are already doing another project. And then we are visualizing the semantics.”

Those semantics, which describe the building in very precise detail, have one major advantage over photogrammetry: Shadow and light information is essentially baked into the images, making it hard to relight a scene realistically. Since Blackshark knows everything about that building it is constructing, it can then also place windows and lights in those buildings, which creates the surprisingly realistic night scenes in Flight Simulator.

Point clouds, which aren’t being used in Flight Simulator, are another area Blackshark is focusing on right now. Point clouds are very hard to read for humans, especially once you get very close. Blackshark uses its AI systems to analyze point clouds to find out how many stories a building has.

“The whole company was founded on the idea that we need to have a huge advantage in technology in order to get there, and especially coming from video games, where huge productions like in Assassin’s Creed or GTA are now hitting capacity limits by having thousands of people working on it, which is very hard to scale, very hard to manage over continents and into a timely delivered product. For us, it was clear that there need to be more automated or semi-automated steps in order to do that.”

And though Blackshark found its start in the gaming field — and while it is working on this with Microsoft and Asobo Studios — it’s actually not focused on gaming but instead on things like autonomous driving and geographical analysis. Putz noted that another good example for this is Unreal Engine, which started as a game engine and is now everywhere.

“For me, having been in the games industry for a long time, it’s so encouraging to see, because when you develop games, you know how groundbreaking the technology is compared to other industries,” said Putz. “And when you look at simulators, from military simulators or industrial simulators, they always kind of look like shit compared to what we have in driving games. And the time has come that the game technologies are spreading out of the game stack and helping all those other industries. I think Blackshark is one of those examples for making this possible.”

Previous Post

First Justice League Snyder Cut Trailer Released at DC Fandome

Next Post

DC FanDome Trailers: The Batman, Justice League, and More

technewshero

technewshero

Related Posts

Video game spending increased 27% in 2020 – TechCrunch
Gaming

Video game spending increased 27% in 2020 – TechCrunch

by technewshero
January 16, 2021
DeepMind’s MuZero masters games while learning how to play them – TechNewHero
Gaming

DeepMind’s MuZero masters games while learning how to play them – TechNewHero

by technewshero
December 24, 2020
Discord confirms raising $100M at a valuation of $7B – TechNewHero
Gaming

Discord confirms raising $100M at a valuation of $7B – TechNewHero

by technewshero
December 23, 2020
After lockdowns boost gaming marketplace Eneba, it raises $8M from Practica and InReach – TechNewHero
Gaming

After lockdowns boost gaming marketplace Eneba, it raises $8M from Practica and InReach – TechNewHero

by technewshero
December 22, 2020
Social gaming platform Rec Room scores $20 million Series C – TechNewHero
Gaming

Social gaming platform Rec Room scores $20 million Series C – TechNewHero

by technewshero
December 21, 2020
Next Post
DC FanDome Trailers: The Batman, Justice League, and More

DC FanDome Trailers: The Batman, Justice League, and More

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Most Popular

Here are our best tips to boost your phone’s slow data connection and poor signal strength

Become a Galaxy Note 20 pro with these 10 hidden features

January 13, 2021

iPhone Owners Have to Stop Taking Their Face Masks Off

August 10, 2020

NSW government launches IDCare Identity Recovery Service

July 18, 2019

Browse by Category

  • Apps
  • Gadgets
  • Gaming
  • Internet
  • Mobile
  • News
  • PC & Laptops
  • Security
  • Social
Tech News Hero

© 2020 Tech News Hero.

No Result
View All Result
  • Home
  • Landing Page
  • Buy JNews
  • Support Forum
  • Contact Us

© 2020 Tech News Hero.

Welcome Back!

Login to your account below

Forgotten Password?

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Are you sure want to unlock this post?
Unlock left : 0
Are you sure want to cancel subscription?