Otakar Nieder, Senior Director of Development at Bohemia Interactive Simulations, discusses how simulation apps are different from gaming and the delta needed over gaming engines to build apps for simulation. Host Kanchan Shringi spoke with Nieder about goals of a simulation versus those of a game, as well as about challenges when developing simulations versus games. Nieder spoke about some of the interesting research laying the foundation for the future of training simulations. They also explored how a developer could get started on the journey to create a simulation.
This episode sponsored by SendBird.
Show Notes
Related Links
- Episode 175: Andrew Brownsword on Game Development
- Game Engine Book
- Game Engine Architecture
- Bisimulations
- https://www.bisimulations.com/
- Unreal Engine
Transcript
Transcript brought to you by IEEE Software
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected].
Intro 00:00:00 This is software engineering radio, the podcast for professional developers on the [email protected]. Se radio is brought to you by the computer society. I well, your belief software magazine online at computer.org/software.
Kanchan Shringi 00:00:16 This has content shaggy for software engineering radio. Welcome to our show today on training simulator apps. Our guest today is art Canada, art Curry, senior director of development at Bohemia interactive simulations are bism. They create training simulators, art joined BSM in 2009. Before that he worked in the gaming industry. Welcome to our show today. It’s great to have you here. Is there anything else you’d like to add to your bio before we get started?
Otakar Nieder 00:00:48 Hi, Kanchan, thank you for having me. It is very exciting to be on this podcast. I think you said all the important things. Now, the director of development in Bohemia interactive simulations, where we do these trainings in majors and I’m responsible for all the development in or offices around the world.
Kanchan Shringi 00:01:06 Let’s get started with an example of a game and the simulation, and then we’ll compare and contrast them.
Otakar Nieder 00:01:12 So there are, of course some many, many games we can choose from a lot of them. So there are simple ones like Tetris, for example, you might have it on your, one of your phone and with some other different casual game that you’re playing when you’re somewhere in a train or bus, the gaming or the simulations that is a bit more difficult to choose some that would be known to people. So I think that one example is flight simulator, which is, I think somewhere on the border of the games and simulations, it is a pretty good simulator of flying airplanes, but it is being sold as a game on the gaming platform on steam. So we can buy it as a game. You can play it as a game, but it is also a very, very serious simulator flying. As far as on my side, I can put in our simulator, which is called VBS for, I just say simulator, but it’s used is being used for training military personnel in many different military training cases. There are, of course other simulations do for, especially for some special machinery or again like similar tools that are being used to really train, uh, train pilots. But of course there’s not as many of them as games.
Kanchan Shringi 00:02:22 So what about the infrastructure? So what about, let’s talk about the gaming engine. Is there a difference between the engine for a game versus a simulation?
Otakar Nieder 00:02:33 It is a bit difficult question, but I think to, to be able to answer it, I will a bit limit myself on when I’m talking about game-changing. So let’s talk about the game engine that you can get yourself unreal or unity. These engines you can just basically get whenever you want, and you may want to start to build a game. On the other hand, we would have to simulation engine, which is a bit much more than much more than just the sandbox that you get with unreal and unity. So when you get yourself unreal or unity, you get, as I said, sandbox, and a set of very, very nice tools, which you can use to build your game, use them to create either the 3d content or 2d content. You can use them to create some scripting, some functionality, some interaction, and that at the end on-suite game, on the other hand, the simulation engine that is much more, that is something that is already prepared to do some simulation.
Otakar Nieder 00:03:29 And again, for the sake of this talk, I will be talking about training simulations. So of course you can have different types of simulations, some physics processes also, but I will be talking about engines that are being used for simulating some kind of a training. And I will be mostly referring to the military training because that’s what my expertise is. So you could get a simulation engine like that. You get a lot of functionality that you can start to use immediately to do some of the training. If I should use some kind of analogy, I will use the one that our CEO likes to use. Uh, he compares this to the situation when you are buying a log house, either you can just basically buy a fully featured, fully, build a fully equipped a log house and move in and they can be there, or that would be your simulation engine, or you can buy a piece of land set of power tools and planks, and then you would just build the look, house yourself. That would be the game engine that he would just get unity. So that would be the kind of a difference in how much is already there built to how much you have to build for yourself.
Kanchan Shringi 00:04:41 We’ll drill a little bit more into that in later sections and you know, what is the Delta, but before we get to that can be also discuss how is developing these games or gaming engines or even simulation engines different from standard line of business or web apps.
Otakar Nieder 00:05:00 I think I would like to answer that basically in every game or simulation development, that is a bit of a business or web application development in the business. And that applications development is not much from the games development by that. I I’m trying to say that creating a bit more complex game is a set of really different kinds of things you have to do. You have to think about also very low level stuff, but if you go a bit higher level, you can create user interface. And these days the citizen interfaces are often used or often build by using the technologies that are being used to create apps or like HTML and all the frameworks that are built on that. So you have a bit of a application development, dare games. Of course, these days are using databases to store a of data, to DV, to buy the players of other characters that are inside. So again, you’ll have a bit of a databases which are usually being used into business development. So I would say that that is a really like the gaming is a very complex thing that has a very broad reach of what needs to be done. But I think that the business applications might be more focused on one specific.
Kanchan Shringi 00:06:21 Got it. So the engine is more general purpose. And when you’re talking of simulation engines that are additional features and frameworks and services,
Otakar Nieder 00:06:31 That’s correct. The game engine as such is a complex thing where you have to just think about what kind of hardware you use it for these kinds of things. And we can talk about it maybe a bit, a bit later, that’s similar to simulation engine, but then the more you want to just do when you more, you want to train the more functionality you have to add, and the more different let’s say, areas of expertise you have to use and then create the content for the training.
Kanchan Shringi 00:06:59 You hinted at that earlier. But can you summarize the difference between the goals of a game and that of a simulation? Sure.
Otakar Nieder 00:07:08 Like for the game, I think I don’t want to just, uh, be to generalize, but I think that most of the game we’ll measure it. You have the games, data goal is to entertain, right? Which is basically entertain you to give you some kind of a feeling to give you something you, you enjoy where the simulation is much more serious. Especially when, as I said, as I’m talking about training simulations, it’s about delivering training. It’s about delivering some sort of scales or training, some kind of skills that you will then use in the real life. Maybe in the games you probably don’t have. Maybe you do some kind of terminal from, they got to in gaming, but in simulation training, you have term negative training. So this simulation, if it delivers to you something that you would then learn wrongly and use it wrongly in your real life, you, for example, in the case of military simulations or military industry, you can cause a casualties with the games it’s casual stuff, and more and more about feelings where a simulation, it’s more about really training, learning and delivering some kind of,
Kanchan Shringi 00:08:15 So what are the most common industries that invest heavily in simulations
Otakar Nieder 00:08:20 As I’m working with a company that delivers training simulators for military? I definitely would say that militaries are very interested in simulations. Why? Because simulations save time. They save money. A lot of it because you don’t have to, for example, use munition, basically simulated ammunition is for free where the real life is very expensive. It saves paperwork because if you want to just get out a tank to, to train with it, you have to fill in military paperwork where in the simulation, you just basically jump into the tank and you can train with it. So definitely there is a lot of interest in militaries for training simulated training. But I think I already mentioned, for example, the pilot trainers islands trained on very specialized, the simulators before they actually can go into the real plane. And I see like a surge of interest from other industries, also like the first responders, any kind of industries using very complex machinery that they like to basically train these kinds of skills before they started used a real thing.
Kanchan Shringi 00:09:25 So let’s now dig deeper into the challenges of developing simulations versus game and the differences starting with the gay mentioned architecture. So what would you say are the common components or services or layers between gaming engines and simulation engines?
Otakar Nieder 00:09:45 So I think that’s, if we start from the low level of center and by low levels, I mean really the levels of touching with the hardware, because that’s one thing that is very common. When you’re creating game engine, you have to know what kind of hardware you’re working with. And you have to understand how the hard drive works, how memory works, how GPU or Excel CPU works, and these kinds of things that are very common to game engines and simulation. And because usually for the games and for the simulations, you want to get as much as you want from the hardware you are developing. So that’s where on the low level, the things are very common. I can name here, some kind of a component that is, uh, that is interacting with the file system component. That is managing memory of course, components that are working with the sound cards to create some sounds and the rendering components that are delivering some kind of a 2d or 3d images.
Otakar Nieder 00:10:43 You also have to represent your environment or terrain, whatever you want to do in a game or a simulation. Again, that is a very similar a component. Of course, it depends on how big environments you want to represent in the game. You also definitely want to have at least some kind of simple physics, you need to have user interface and you have to, of course, allow the player, all the trainee in the simulation to input somehow to control what is happening in the 3d environment. So again, there is a component for user interface and also component for digesting the input from, uh, from different controllers like mouse and keyboard. The end of the common component would be that you want the user of your game engine or your simulation to be able to maybe integrated with third party libraries. So for that, you provide some SDK, some API APIs that you can use to, to do that.
Otakar Nieder 00:11:39 And also you want people to be able to create their content that I don’t mean other content. Now it’s really content really models. I mean the functionality and interaction functional D. So for that, you usually provide some kind of scripting, some kind of way of how you can program this thing. And probably last but not least from the very common ones. Of course, you need to deliver some tools that will allow your users to either create the 3d content that you want to populate your world with. Or some editors that can help you create the training scenarios in the simulation. So these components would be the ones that, that are very common to game engine and the simulation engine without them, you wouldn’t have either, but then you go a bit higher level. You go a bit more offering a bit more. And that’s where things start to differ between the gaming gene and the simulation engine.
Otakar Nieder 00:12:35 Because for the gaming gene, you maybe don’t need networking. Maybe you do, maybe you don’t. So it might not come as a integral part of the package. You get the sandbox you get, you might need to write to your own networking or you buy a third party library that actually for that agent provides a network. Same comes with artificial intelligence. There might be some kind of a very simple artificial intelligence included in the game engine, but very, probably you want a different one. So you will have to, again, either create your own or purchase a third party one. So that like around these games in genes, there is of course, a lot of third-party libraries and a lot of people and companies that are creating these libraries, then you can use to enhance your game and to make your game more appealing to your gamers, but to simulation engine, of course, these things have to be there also, but they are very, probably more specialized for the training use case that the simulation delivers.
Otakar Nieder 00:13:39 So of course, in our system, we have an artificial intelligence that is very specialized for simulating military doctrine, behaviors of different groups of soldiers that are doing different kinds of maneuvers. We focus on also the enemy artificial intelligence behaviors that again, need to be somehow representing what the soldiers can interact with in the real world. So these things start to be very much more focused and specialized in the simulation engine. Another example would be physics where again, in the game engine, you might get some simple physics for calculating collisions and these kinds of things. When do you want to do something more complicated again, we’ll probably have to write your own in the simulation engine. You get very specialized, very detailed physics for a listings or for driving vehicles and all the systems that are actually being there on the vehicles. So these like basically when it comes to a more specialized functionality, that’s where the components start to differ a lot because where for your game, you might need the simple physics for the simulation really needed a very, very complex physics.
Otakar Nieder 00:14:54 What for the game you might need when you’re creating it for the mobile phone to operate it with two thumbs for the simulation, you need to simulate a very complex interaction systems that are representing complicated operating systems for different vehicles. And one thing that actually I haven’t seen in any gaming engine, which is actually very common for simulations and training simulations is something called after action review. It is a system where, which is basically being used to record everything that is happening during the training, and then replay it back. So the soldiers can, or trainees can just basically look back to what they did and learn from it, and then maybe do the training again and again and again, and again, look at the after action review in our case, this is not a simple video. It is actually the recording, everything that happens in the 3d environment. So we can replay it again, like exactly how it happened. So you can jump into 3d environment during the replay, see what was happening. You can power pause it. You can see where everybody was, what they were doing, what they were looking at, what they were aiming at. So that is something that I have not seen in games because you really want to return back and replay something. So you get better that all of course, people like that, but this is something that is very, very specialized for simulation cases.
Kanchan Shringi 00:16:21 Thanks for that comprehensive answer. So what I can summarize from that is that you start with a gaming engine, you could start with the gaming engine, which gives you a lot of the components you need, which is, you know, getting the best out of your hardware, the user interface, some of the physics and input device as DK scripting, but then you specialize the algorithms, the collision engine, and you further add features to record and replay. Is that a fair summary? And if so, did you guys start with the gaming engine and build upon it?
Otakar Nieder 00:16:58 That’s a very good question. It is a good summary. It just, of course it depends on how complicated your simulation is. And of course, how complicated your game is because they’re again, simple games and complicated games. The system that we have here in Bohemia, interactive simulations is a result of work of more than a hundred people for 10 years. It’s a really, really big big system. And did we start with a game engine? Yes, we did. We had, uh, actually, uh, back then we started to use a gaming team that was coming from a game Arma two, which was being built by a company, Bohemia interactive studio. Back then we were actually assist the companies these days. We are separate companies, but yes, we have started with a gaming gene. And I think that in our case, it definitely was worth it because we got some of the systems, the low low-level systems. But then as I said, we have to spend 10 hears of many people working on creating the functionality that then delivers the training and the simulation.
Kanchan Shringi 00:18:00 Can we talk about a couple of these areas of specialization? You know, you can pick either the level of mathematics needed or the collision engine and perhaps the use of AI.
Otakar Nieder 00:18:13 So when it comes to simulation, I guess that you probably can imagine that if you’re teaching people something that then they will use in the real world, the level of real life representation has to be very high. It’s like things have to be precise. They have to really represent the real life as close as possible. So you, again, don’t get negative training because if something would be very easy because not very good physics, then you would learn that it’s very easy to do something. And then in the real life, you would find out maybe in a very bad situation that it is not, not as easy as it is. So, you know, for example, to repair a tank threat, it’s a long process. And if it would just take you one minute in your simulation and you learn that it’s very quick, then in the real world, you would be very surprised all you have to do to repair a broken tank threat.
Otakar Nieder 00:19:09 So if I should talk maybe about the collisions and the physics, that’s where we had to probably do the most specific changes or development that usually is not needed in the games. And that’s on the precision side because we are currently having new engine that is simulating the whole earth. It is rendering the whole earth and you can just basically jump everywhere in the world and you would see some kind of preloaded to 3d environment that’s procedurally generated, or you can have actually a very real world data in there for that. You then want to just go anywhere in the world from the place where you jumped in. And very quickly, when you were doing the simulation, you find out that the normal numbers, the 32 bit numbers that are being floating point numbers that are being used in all of the engines, start to be very imprecise.
Otakar Nieder 00:20:07 When you go further and further from the place where you have been. And the physics start to be really is relying on the very precise, floating numbers. It starts to be very unstable. And out of nowhere, you have vehicles that have a small bump fly out to the space because that’s how the floating point numbers are represented in the PC or in the computer. So we had to switch to 64 bit representation of these floating point numbers. So very early, like many, many years ago before the games actually even started to think about it now because the worlds are bigger and bigger in the, they also think about it, but we had to do it very, very early to be able to just see all of these big, big environments. So that would be on the physics and collisions side on the AI side. I think I already mentioned that we are specializing on an AI that is that simulates the behaviors, the military behaviors.
Otakar Nieder 00:21:10 So we have to really create a system that can handle that. And because we are not only selling to one army, but we’re selling to many armies around the world, which have different doctrines. And they expect basically their artificial intelligence soldiers to behave. According to that doctrine, that system has to be highly parameterized. So again, that, uh, that creates a very complex and we spent quite a lot of time, uh, doing it. So these are the systems that we had to just really spend a lot of time on another one is ballistics. Of course, if you want to have a realistic of behavior of your shells and what they do off the day explode, or the particle effects, flashes, and these kinds of things, you have to spend a lot of time talking to the subject matter experts who will tell you what, uh, these things have to do, and then create a specific algorithms that can replicate that.
Kanchan Shringi 00:22:08 Something you said was very interesting is that the gaming engine is also benefiting from all the work that folks are doing on training engines, training, simulator engines, making it more real. So that was interesting to me.
Otakar Nieder 00:22:21 Yes, definitely. Like I think we are, the simulations are just kind of, at least in our case, which pushing what is possible, where the game engine, like what is possible to show what is possible to represent what is possible to interact with. So I think that we’re, we’re pushing the boundaries a lot, maybe more than the games, because again, we need to be very near to the real world experience and some of the games of course try to do that too. They just, as part of that, they want to do that. And if they find out who they can get to near, they probably are okay with it. They can cut corners here and there we can’t. So definitely we are, we are maybe one step before ahead of the gaming development
Kanchan Shringi 00:23:06 Talk about deployment. And the specific question I had at this point was that multiplayer games require cloud-based game service. Is that true for simulations as well? Are there use cases that you have many folks participating in the simulation?
Otakar Nieder 00:23:23 Very good question. I think I forgot to mention it. When I was talking about the components, different components of the simulation engine training, mostly it’s about collaboration and cooperation. We were saying that our training is cooperative. We do not focus on training one soldier doing one thing on heroin. It’s just basically cooperative training. So of course we do support multiple trainees being in the same scenario. So some kind of a multiplayer game, but the difference is that, of course, I think everybody knows about these massive multiplayer online games, which as you say, near the cloud based servers, and there are hundreds of thousands of people playing games with them. We in the simulation area are much more like restricted and they’re much more focused. So we always have to think about what we are training and what is the benefit of doing something for the training having 100 and thousand trainees in the training might not be that good of a training.
Otakar Nieder 00:24:33 I’m not sure what exactly would be training. So currently we are focusing on hundreds of simultaneous trained people. We are currently having customers while training with 200, 250 trainees at once. Of course, our customers are thinking about how to make these trainings bigger. They are thinking about, as I said, like how the training would go. Of course, if you want to go into a brigade level, that starts to be thousands of people. So of course our customers are thinking about bringing more trainees in, but the more trainees are in the more logistics problems. So we, at this moment to return back to your question, we are not having like servers in the cloud and we are not leveraging that functionality. On the other hand, our customers are talking to us and we’re thinking about how we can best actually jump on that because we know that in the cloud, there’s a future. And we are thinking about how to use cloud to allow us to improve our training. And it’s actually currently not in the number of how many players we can support at once, but it’s more for us. How many vehicles really very detailed vehicles with detailed physics we can simulate and how many artificial intelligence units we can have in one training. I see,
Kanchan Shringi 00:25:57 Let’s talk about the hardware. You did mention that it is important to get the most out of the hardware. So are there any pre-recs that you have on the client hardware?
Otakar Nieder 00:26:08 For us, it’s a bit more complicated than for the game risk, because all for the gaming companies, because the games usually just come with the best graphics and for the best graphics you, if you want to buy the latest game item of punk, the gamers are like, okay, well I will buy a new hardware for it because I want to play it on the best of as details and whatever we are working more with a big agencies. And of course the tempo of changing the hardware is not as fast as, as for the individuals. So we have to lift a bit in the past, we have to support hardware that is not top of the shelf hardware. So we always have to focus on not only bringing all creating more complex simulation or more complex environment without thinking about how we always also have to think about, okay, so we want to do this more complex thing, but how can it work on the hardware that is 2, 3, 4 years old, which again makes the work that we’re doing on the simulation side of it, more complicated and to, for the gamers who gaming companies who can maybe just basically assume that the gamers have the latest.
Kanchan Shringi 00:27:20 So you have more constraints.
Otakar Nieder 00:27:22 Yes, we do have more constraints,
Kanchan Shringi 00:27:26 Uh, hardware, peripherals, what are required for your simulations? Joysticks. We are headsets steering wheels, anything else come to mind?
Otakar Nieder 00:27:36 Yes. So definitely a lot of more, I will start with the controllers of course, joysticks. Like actually some of the training is being done with keyboard and mouse and basically the trainees are using our software as they would use some kind of first person shooter shooter game. Of course, everything is much more serious and they have to adhere to the doctrine, but they are using keyboard and mouse as any player of the crispness and shooter on a PC with duke. But then of course come more complicated scenarios and more complicated machinery that you want to train with. So you come to joysticks steering wheels, of course. And then you start to really use very bespoke controllers. We are integrating with pieces of hardware that are one-to-one replicating, what you would have in your hands, inside a tank. It has all the buttons, all the sounds and all the feeling that you would have as if you would be sitting in the tank.
Otakar Nieder 00:28:35 So that is a lot of different peripherals. You can actually use a simulated in Oculus into which we are actually sending the picture. So you’re using that seem like you are using a real life device that looks like real life. We know killers, but inside you see the image from our energy. So these are kind of things that, that we integrate with on the side of the VR and AR devices. We are working with those for many years now, since the first Oculus, another CV one, the previous ones development kits were released. We worked with them and we are not having only access of course, to the ones that you can buy off the shelf. We are working with different companies that are producing headsets that you normally can’t buy because they’re of course very expensive, but they’re also very much superior to what you can buy off the shelf.
Kanchan Shringi 00:29:32 Let’s not talk about team and I’m really interested in two things. The first desk dusting dusting is hard in this case. I cannot imagine how you test. Can you talk about that? How do you know what your building is actually going to work?
Otakar Nieder 00:29:48 I agree that it’s a very complicated, complicated thing, and I think we have it a bit similar with the games. We rely a lot on a manual testing. So on the other hand, we have also one advantage. We have a requirements from our customers, our customers tell us what they want to train. They tell us if they tell us, Hey, I would like to train parachuting. They tell us what they expected from the training. So we of course, uh, put it into requirements. We put it into test cases, and then we turn these requirements to test cases. And then there’s a lot of manual testing being done on those test cases, making sure that the functionality works, where it starts to be really complicated. And that is also again like a very specific, I think for what we’re doing, any new functionality has to also work with all the old functionality, all the previous functionality and anything we change have to be also backwards compatibility that quotes compatible with all the data that were used for the previous functionality.
Otakar Nieder 00:30:52 So we have to keep a lot of the old code in to support previous functional, previous type of data, previous ways of interaction. So that of course creates immense effort. When we want to test our software, it’s just almost impossible. We have to be always very thorough on delivering a new release on testing, doing this. We call it regression testing, trying to find out as many things as possible that actually regressed. From the last time we have had a release, we have suite of automated tests. You kind of automate some of the things, I don’t know. You can automate testing of artificial intelligence. So, you know, if you give it to some way, point, if it gets day, shoot some money, if it can aim and shoot someone, but a lot of the stuff really needs a manual testing and a very, very skilled quality assurance people. So for example, for us, when we hire in you and you test her, it takes several months before that person can actually deliver a quality results because it is so, so complicated, that is so much of knowledge that needs to be put into it when you’re testing.
Kanchan Shringi 00:32:08 Okay. Let’s do off my takeaways from that was that it’s a really, really important to get the requirements correctly. And the second was that in spite of all that, it does take time for the QA folks to really start, you know, doing a close to a perfect job as you require. What about UX designers? How do they then model the personas and how do they get started? And what is the role of product management here?
Otakar Nieder 00:32:37 So for the UX designers, we have started several years ago. We have a basically found out that we really need to do something on the UX side. So we have a UX designers in our team because our customers sometimes, uh, the procurement company. So sometimes we do not have access to the end users, but in many cases we do, we have access to the actual soldiers so we can ask them what they expect, or we have access to the people who would use our software to train their soldiers. So we ask them what they expect from what we are delivering them, how they think that it should operate. We give them some kind of options and we try to work with them as much as we can in this kind of agile approach to achieving the goals. So we constantly try to talk to them and find out what their opinion on the latest iteration is.
Otakar Nieder 00:33:29 That’s the best case. Sometimes as I said, we do not have access to the end users. And then we use our internal people. We have people who served in the military previously. We have people who have experienced training in the military training centers. We have people who work as administrators of a simulation training facilities. So we can talk to dos and we usually try to get the best UX design things from them as for product management. It’s a, of course, very important for us to somehow keep the product going into the future in one consistent and coherent way. That’s a massive job because we are having a lot of projects. We is another, I think, difference between the simulations and the games where in the game, you usually like really focus on one game, maybe with a big team, but the team has one focus on this one game.
Otakar Nieder 00:34:25 We have 20 different projects for four 20 different customers. They want different ways of you want a different new ways of using our software for training. And that’s where product management has to look at all of these and, and kind of say, okay, we need to just shape this one a bit more. So it also serves more of our customers, not only this specific customer. So that’s the role of product management, making sure that what we’re doing is really usable for as many of our customers as possible not to do some, this book development for one specific customer
Kanchan Shringi 00:34:59 On the same lines. What about the customers? How do they measure?
Otakar Nieder 00:35:05 That’s a really complicated question because of course our customers use our software to train the soldiers. And it’s sometimes very hard to find out exactly immediately if what they’re using, if the training has a good outcome, you have to train a lot. And very often to just get some kind of, some of the skills. So we definitely talk to our customers. They are giving us feedback. Sometimes we find out that some of the procedures that are being simulated are maybe too complicated or the peripherals are not delivering as much training as they should, but mostly we have a good feedback. And the most precious feedback is in the form of our customers telling us that because their soldiers were trained using our simulator, then they got away from some very, very difficult situations in real-world scenarios. And we really had a feedback from a military personnel who told us that because the soldiers spend time training on our software, they then were able to react in this situation so that they saved a life. So that is one of very nice feedback that sometimes we get, but really measuring ROI is pretty complex on our software.
Kanchan Shringi 00:36:23 I mean, saving lives is priceless. So, you know, thanks for sharing that. So we’ll talk a little bit later in the show about two topics, you mentioned whole world simulation. So that sounds like an area of research and of course, VR and motion tracking. But before we get to that, I wanted to talk to you and see if we can work through an example. Let’s talk about, let’s say the simulation for parallel parking. If I was a developer who wanted to create this, how would I go about it? What are my options? How would I start?
Otakar Nieder 00:36:56 So if you, as a developer, as a, as a someone who’s sitting at home and trying to think about how to do this, I guess the fastest thing really is to get unreal or unity or one of the same genes. And if you really only want to simulate or create a simulator of parallel parking and nothing more, you, it might be pretty simple using these systems to do that. You would just basically place two, two boxes somewhere. You would maybe use the basic physics that comes with these engines and the basic car simulation that is that, and that you would place the boxes in some distance from each other. And then the player can just jump into the car and try to park and see if the player hits something or not. That would be your simple solution for that. Of course, things start to be more complicated.
Otakar Nieder 00:37:47 If more requirements come in, no, you don’t want to only just train the person to parallel park in a very artificially created environment. You may be want them to learn how to park in the slope or when they’re in the slope or you want to train them parallel park when there’s a traffic around them. And maybe you want to train them how to parallel park, a big cars and small cars and the cars woods with a trailer. And the more you want to train, the more you actually are trying to starting to find out that it’s more and more complex to make all the new requirements to work together. And that’s where you start to think about, oh, maybe I should start to look for a simulation engine, the simulation solution that that would help me fulfill all of these requirements, me having to spend so much time. So to answer your question simply, if you want to do it yourself, definitely maybe get into on real, which are for free, and you can quickly do something simple yourself. But if you are an organization who, for some reason needs to teach their people or some other people to parallel park or in different situations and with different constraints, maybe try to look for if there is any solution that would deliver the training already out of,
Kanchan Shringi 00:39:10 I do want to dig in a little bit more there. You know, you mentioned that one of the things you’re looking for is, Hey, do I hit something? But you know, as I’m thinking about this, what I really want to know is, am I steering correctly? You know, is my car sliding well into the slot?
Otakar Nieder 00:39:30 Probably approach it from different angles and, uh, create different levels of complex solutions. So the most easy one would be probably that you would create some kind of 2d simulation where the player would be just operating with the arrows, a box, or maybe something in the shape of a top-down shape of a car. And you maybe know these hints, which say that you should park next to the car in front of you, and then go back where it’s until are and start to turn. And you can maybe visually you can have the player to just basically turn the car with the keys and keys on the keyboards and show visually when to do things when to just start to turn and when to stop turning. So that would be the most simple thing that you can definitely do that in the unity very easily with the 2d engine, which is there, I think without any additional modules you would, you would need, you then want to maybe do it a bit more immersive of it more, again, we lifelike.
Otakar Nieder 00:40:34 So you would want to do it in 3d, basically placing the player into the car. And again, you can do it so that the game is being operated by keyboard and a mouse where you can maybe with some keys, with the keys, you can operate the car and with the mouse, you can operate the head. And, uh, in that case, you, which is basically be able to operate your head and look behind you and then see where you are the next to the car in front of you. And again, you can provide some kind of hints in the 3d saying, now go backwards. Now go backwards and I’ll stop. And I’ll start to turn that wheel. And you would turn the wheel either with the steering wheel that you would have over the keyboard. And again, the simulation can tell you, Hey, stop now, start to back again and stop.
Otakar Nieder 00:41:22 And now change the turn off the wheel. So you can do that again. If you want to do this kind of a simple thing with, again, with not too complicated environments, just maybe showing some simple approximation of a street and over the car so that you park in between, you would be able to do it in unreal and unity, very simply. And without any too many auditions, I’m not sure about like the basic physics that is in part of the sandboxes, but maybe you would be even good with that. And then you want to go a bit more further, as you said, VR, you want to embarrass the person even more. And of course, with the VR, you can have this 360. I feel like freedom of moving your head and seeing basically everything around you. So that would create another additional layer of immersion.
Otakar Nieder 00:42:14 But for VR, you actually need much more believable world. So you would have to spend time on maybe creating a better interior of your car. And even the exterior, you would have to spend a bit more time on the creating your virtual environment. So that would be a bit more work. You would have to create this 3d content, but on the technology side, again, I think it would be pretty possible to do with just under unreal or unity engines, which now I think natively support VR headsets, but that it’s, again, what gets a bit more complicated if you, as I was talking about previously, want to add more things in a different traffic or different types of cars, maybe different kinds of slopes and conditions that you would need to do parallel parking. So that’s where you would maybe need a bit more sophisticated physics to support different types of vehicles.
Otakar Nieder 00:43:10 You will need some kind of artificial intelligence module to create some pattern of life traffic around you. And you might need a support for more comprehensive controls, like maybe a handbrake or something like that. Clutch, if you would have a manual on your gearbox. And that starts to be a more and more complicated, and you would probably have to do some research on what kind of libraries exist for these genes, or you would have to build them yourself. So really it all depends on how far you want to go and how broadly you want to handle this training scenario of parallel parking.
Kanchan Shringi 00:43:50 Okay. So start with unity. That’s a perfectly good place to start, but then as you make the scenarios more complex, you would need to specialize or replace the physics engine, the AI modules and the support for additional controls.
Otakar Nieder 00:44:09 Yes. The more you want to cover with your training, the more you will learn, probably find that the basic libraries or components that are available there for unity or unreal, that they are not good enough for what you want.
Kanchan Shringi 00:44:23 Okay. That’s a good introduction. Now let’s go back to what I hinted. We would want to talk about areas of research. Let’s start with the whole world simulation. So you said that was an important addition that you guys have been working on at BSM. So a basic question, how do you do that? How do you make a replica of the whole real world using flat images?
Otakar Nieder 00:44:49 So it’s definitely not easy. It took us years. So how much, uh, to actually come with an engine that we are now using for the production. So basically of course there are data that you can use. Satellite images, there are open street map data, which you can access and get, and all the data is actually a reference. So they, from open street map, you can get factors that are representing streets or building or footprints of buildings in the real world coordinates. So you can actually place them there where they should be. The complex thing is how to get all the data into the memory of a PC so that it looks as good as it can. And it still performs very well. That’s the biggest challenge there. How do you represent the whole world to reign geometry in the memory? How do you represent it so that you can actually fit it into the memory, like in the places where you want to create a training?
Otakar Nieder 00:45:56 So there are many, many sophisticated algorithms for that, that our scores around the area for level of detail, where we had to just basically create some bespoke algorithms to cope with that. Of course we are not having all the data at once in the memory where very cleverly streaming the data in there where you want to have the image. We also like you can again, get a data which represents the height map or the basically how high things are in the real world, how high above the sea they are. You can get the data that they are publicly accessible, or you can buy them in a different types of precision. Again, maybe you get to precision of one meter or maybe 10 meters, that data is maybe even publicly accessible or kind of cheap. But if you want to get into the precision of millimeters, then again, the daytime might not even exist.
Otakar Nieder 00:46:54 And if they would access tape would be very, very, very expensive. So again, we had to come up with algorithms that, so we call the process, refinement, refine the data that we have, maybe 10 meter precision. So you only have basically Vertex every 10 meters. So you can’t, for example, two trenches or something like that, or ditches, because you would have to do only ditches that are 10 meter wide. So we’re coming with these algorithms for refinement of the terrain, so that we can go on the centimeter precision and allows us to do very, very refined modifications to the terrain. In case we like customers want to do a, build some trenches or want to have a very specific representation or very real life specification of the terrain. So that is the biggest challenge, how to get all of the data, how to store it so that it is very quickly accessible. And then when you access it, how to put it into the memory, so it can be displayed in a very efficient way. So you still have 100 FPS, 20 frames per second, when you’re moving through the environment as you have in the normal games,
Kanchan Shringi 00:48:08 It’s really fascinating. And I can imagine really, really challenging. What about motion tracking for VR? I imagine that’s absolutely critical for realistic simulations as well. Can you talk a little bit about
Otakar Nieder 00:48:21 When I hear motion tracking, I kind of think about this complicated tracking of all your joints and all your limbs and your head. We do not have to get that complex. Most of the times we have customers who want that we want to just basically track the motion of the soldiers and we provide solutions for that for the VR. It’s actually not that complex because the headsets are usually covered with their inbuilt ways of tracking the motion of the head. They also come with a very clever algorithms to prevent people from getting motion sick. If the screen is not updating as quickly as possible. So that kind of a tracking of the head for the VR headsets is not that much of a complication for us because the headsets usually come up with a cute SDKs that provide us all the information that we need.
Kanchan Shringi 00:49:14 Let’s start to wrap up. Now, is there any topic that you want to talk about that we haven’t touched?
Otakar Nieder 00:49:20 Not specifically. I hope that the session somehow is able to deliver the message that simulations are different to games. Even if we started with a gaming engine, we are now really, really very different to what a game would be on the other hand, as I already said, if you would probably see, you know, some of our customers training their soldiers somewhere in the facility, you would get there and they would be sitting there using that keyboard and mouse. And he would say, well, they’re playing a game that you would have to observe what is happening around that they might be, you might think that they’re playing a game, but everything has a really very strict rules. And it is for us, it’s very important to not deliver it. And they got to training and that’s a, that’s what we do. That’s what we try to achieve to make something that really delivers a very, very good training. So anyone who is using our software is getting out of prepared for what can be ahead of them in the real world. So I would say that the biggest difference of if we should talk about like, if I should close on the topic of difference between game engine, as I described it at the start and the full simulation engine, it’s the vast amount of work that is being spent on the simulation engine. So it can deliver the training that it needs to.
Kanchan Shringi 00:50:40 I did watch some of the videos of BSM on the YouTube channel, and I did find them fascinating, you know, leveraging all the efforts and all the success that gaming engine has done. And channeling that into training is definitely something that I hope our listeners find as interesting as I did. And talking to you today as well. How can people contact you
Otakar Nieder 00:51:04 So people can contact me on my email address if they want to contact me, it’s as simple as neither at BA simulations com they can of course, go to our webpage and contact our company through some of the contacts that are mentioned. That that might be the best way because of course I, um, but basically, so I might not be just answering quickly. So I think the best way would be to go to our web pages, which is BI solutions.com and search for the contact information there,
Kanchan Shringi 00:51:33 We’ll put a link to that in the show notes. Thank you so much for being with us today, ARCA. It was an awesome session. Thank you. Thank
Otakar Nieder 00:51:41 You for inviting me and thank you for your great questions.
Kanchan Shringi 00:51:44 This is Canton stringy for suffering engineering radio. Thanks for listening.
Outro 00:51:49 Thanks for listening to se radio an educational program brought to you by either police software magazine or more about the podcast, including other episodes, visit our [email protected] to provide feedback. You can comment on each episode on the website or reach us on LinkedIn, Facebook, Twitter, or through our slack [email protected]. You can also email [email protected], this and all other episodes of se radio is licensed under creative commons license 2.5. Thanks for listening.
[End of Audio]
SE Radio theme: “Broken Reality” by Kevin MacLeod (incompetech.com — Licensed under Creative Commons: By Attribution 3.0)
