033 - Rendering challenges of Virtual Reality with Dominic Eskofier from NVIDIA

Show notes

This week on ResearchVR we dig deep into the magic behind rendering of VR - where and how can we save performance in extremely demanding virtual reality. We are happy to welcome again on our podcast Dominic Eskofier, NVIDIA's Manager for Europe and beyond.

Topics

Here is a list of most prominent issues we've talked about:

  • differences for VR GPU pipeline
  • improvements in Maxwell & Pascal series of GPUs
  • KPIs of graphical performance for VR
  • Motion-to-photo latency
  • GPU memory importance for VR
  • VR Works - importance and implementation
  • spatial audio through optical raytracing
  • addressing lens distortion
  • asynchronous time warp
  • VR SLI
  • GP-GPU - general purpose GPU calculations
  • CPU vs GPU
  • Quadro GPUs
  • NVidia for 360 videos
  • light field displays
  • eyeray render
  • cloud computing
  • NVIDIAs haptics engine

Links

How to subscribe and contact us

For more details tune in to another episode of ResearchVR! You can subscribe to our RSS Feed in the mp3 or acc format. We hope you learned something new and enjoyed listening.

Thanks to Dominic for sharing his expertise with us and our listeners. Reach our guest at:

Topic Suggestions, Guest Invites, and all other all feedback are helpful! Feel free to contact us under researchvrcast@gmail.com or via Twitter:

Rate us on Itunes and help spread the podcast that way!

Show transcript

00:00:00: Music.

00:00:05: Welcome to the next episode of research me are everything beyond the real good to reality your joint by Miracles Petr like of.

00:00:12: That's where was Christopher izdebski LOL and by Dominic eskofier hello everyone today we will talk about rendering challenges in virtual reality and,

00:00:22: I'm pretty sure that Dominic is the right yes because he's in fact from the media and has a lot of inside not all of them he can share but I hope at least some so you gave today a very good talk you're on the panel VR,

00:00:35: one of the biggest meetups in Germany and what was it about it was about the rendering challenges of with your reality so,

00:00:42: what kind of the problems you have when rendering for VR what types of what types of latex you want to get out of the pipeline the types of things you can do to and Hunter performance to get more out of the GPU to already have,

00:00:55: and also the special challenges that VR offers for those rendering problems,

00:01:00: let's start basic so hold up the whole rendering Pipeline and general work as a game engine the pipes later so usually when you when your rendering for a game engine the GPU was mostly there for the for that for all the 3D geometry for the pickle shaders,

00:01:14: and all of these different things but when you're talking about the are there are a couple of other different things so atypical VR pipeline looks different because.

00:01:23: For rendering the actual view first you get the head pose off the sensors off the headset.

00:01:30: And then that position and translation of data is getting sent to the CPU which then computes the image that the GPU has to.

00:01:38: Provide to the display in the headset.

00:01:40: Photos of the main stepsister you get the headphones from the from the from headset that goes to the sea pew that that goes to the Jeep you and keep you with an outputs that to the display and.

00:01:52: Since we are mostly using aloe persistance this place in virtual reality headsets and you only get a short but like flash of the of the LED display.

00:02:03: And then that image gets then transported by a photons into your retina,

00:02:06: how does a game-ending play inside the road so the hats informations application commission goes into the city with basic and Rams game engine the game engine determines we're looking at what happens our particle effects,

00:02:17: and then basically pushing the rendering commands to GPU right to the GPU is just a piece of Hardware that just friends or doesn't do any calculations to change anything right it does it does it a ton of calculations but only for ethical SS so yes the older all the informations getting sent by the CPU and what the GPU that doesn't go that specialize on,

00:02:36: is rendering the 3D geometry project thing that 3D geometry onto a two-lane which usually means your monitor.

00:02:45: In the case of virtual reality is means the two displays that are side by side on your headset now the are involved see this there is competition so it has rendering,

00:02:55: images of does it renders insidiously as one image how does work is usually a couple of years before the tradition of rendering python look just like a game plan so you can see what you did is you rendering,

00:03:07: you rented a few for the right eye and you have to do both John with your passes as well the shadings and and all the other stuff that's involved,

00:03:14: what we could do now with more Modern Hardware for top of the Mexico serious and the Pascal series of of Nvidia so that the 900 series of the ten years of the 1070 or 970.

00:03:24: There are a couple of things implemented inside on the chip itself so there are a couple things that you can use from a software side that are being and Hans or computer faster.

00:03:36: Due to the fact that we have some something for the patients on the on the hardware itself.

00:03:40: One of them one of them is the geometry pass for example so just as I said before you can complete rendering one for the left I want for the right on what you can do and with a system called a single pass stereo.

00:03:55: Is you use the same geometry information and extract the two viewpoints from that single information so instead of doing to Geometry pass this one for the left one for the right eye you only do one geometry pass and then extract the information for the two for the 200 exactly the same or slightly,

00:04:13: corrected for the position of the corrected so that you have.

00:04:20: But in essence you safe 1 geometry past so instead of doing okay with your passes for one frame,

00:04:25: you doing one geometry Paso any you do it I guess to Safelite and see which is one of the biggest issues and rendering in in that case you're using it mostly for performance enhancements so what that means is,

00:04:36: cuz you have you put less pressure on to the geometry of the geometry part of the GPU and you actually can you use.

00:04:46: More geometry in your scene so you can have a more complex in Geometry wise or you can use that safe performance,

00:04:54: for other parts in the VR experience of free compass for particle effects or correct simulation of cloth or fluids what does performance in this context what are the key,

00:05:06: performance indicators and despatch case indicators for for Aviara little tricky because you have to write her at 90 FPS,

00:05:14: which essentially means you have about seven times to eight times the amount of throughput that you have compared to a normal gaming monitor so if you looking at a gaming monitor that runs a full HD game at 30fps,

00:05:28: it's about 60 megapixels per second of bandwidth that you pushing through with the plan.

00:05:32: The rendering for VR you have two screens and you push both of these screens at 90 FPS which put us up at about 450 mega pixels per second it's about seven times the amount of workload,

00:05:44: GPU has to do and that's also the reason why you need this is rendering tricks.

00:05:52: City enhance the performance so you get more frames out of that you have a more stable experience even on on lower and Hardware that's the main the main kpi so what are the problems of rendering in general except performance and latency is there anything else.

00:06:07: What is latency are definitely the most important parts performance because of the 90 FPS that you have to get which is a lot usually games like a typical Call of Duty game that runs at 30fps or 60fps that is an addition,

00:06:19: and then what you name is also very important the latency parts so.

00:06:22: From the time you move your head till the photo and actually hits your retina that at that amount of time should be as short as possible and a couple of Studies have shown that the magic threshold is about.

00:06:34: 20 milliseconds so if your motion to photo latency is below that rate.

00:06:39: You don't tend to realize that you're looking at a display and you don't get any negative effects like motion sickness for example which can also happen if if the latency goes higher than 20 minutes seconds,

00:06:50: to see the user is essentially the the image of the following is trailing your head movements it isn't right where you are and expect it to be a little bit late,

00:06:59: just take those dk1 an experience exactly that was running at 60fps,

00:07:07: Nolo persistence don't quote me on that but I think it had about 50 minute milliseconds off of latency sorry I'm around there and no time warp which compensate the fact from someone sometimes that friend cannot be calculated Tech super super something that we should talk about later on because there's a couple of things them in the in the.

00:07:26: The trick set that you can feel one step back now the GPU has not only clock and the.

00:07:33: Performance speed of can do like not only is a gigabit per second can talk later but also Memory Ride and when we talk about more megapixels that it needs to put out per second door repair frame you need more memory also an issue,

00:07:47: memories not as much of an issue for VR,

00:07:51: it is it is important obviously but things like latency so how fast you can put that image out is actually more important in general what,

00:07:59: the memory helps with his large texture is so if you have lots of geometry enlarged texture sets on those two parts of geometry is use more texture and memory,

00:08:09: that itself takes up a lot of a lot of space but in general what you want to do what you want to have is.

00:08:15: As much bandwidth as possible at Alaris Lowell latency as possible okay and what are the next steps after you determine Zoe's rendering problems.

00:08:25: Are the next steps are for our for our research and development teams and also are Tech teams to actually Implement those and those Concepts take him they come up with,

00:08:34: I put them into the hardware and then also,

00:08:37: used RV RX SDK to exit eem speak with the with the hardware in a way that it has a performance even for their soul and because maybe something that that's interesting to the to the developer an audience is listening to the podcast,

00:08:50: MDRX is a set of different of different of pools.

00:08:54: Are different S2 case that all help to enhance the performance of a virtual reality experience how much effort do I need to put it in an envelope or to implement those sdks,

00:09:04: it's with the pipeline here for a developer section is most of Developers for feedback and how long did it take you to put that in and.

00:09:14: Could I get for my from from the talk to developers it's about half a day to a day of work to actually Implement a vrx,

00:09:20: I miss you using unity and unreal it might even be easier because those engines are already supporting ambiorix in a couple of branches.

00:09:28: So it's not that not not not that big amount of work and you get a lot of value out of it because all these well how how how much money are we talking about getting out if you don't have to care about the background rendering stuff,

00:09:41: so essentially an Ideal World you take a box in unity.

00:09:45: Then you get all the oldie performance enhancements through single pass stereo that's one of the things that I that I just talked about his where you where you actually put the lower pressure onto the geometry.

00:09:55: Part and there's also there's also a couple of other things and what that means for you as a developer is you can use your resources instead of,

00:10:04: getting whatever 5 or 10 or 15% more performance out of your application you could just use the RX and you get.

00:10:10: Why can you get that forms for free and you can use those resources for whatever creative processes to make your experience better you name it,

00:10:18: the physically more beautiful experience job exactly so it it it it it opens up more performance or actually cool affect lightning article effects or cloth simulation or fluid simulation things make a VR experience much more immersive,

00:10:33: the what would be the process of implementing the artworks so my question is basically why does it take half a day to a day.

00:10:39: I'm usually has to do because you also have to read a more SDK tutorial Tori, but I understand it's something that just for the sake of it,

00:10:51: you want you want to understand of the the the process behind it,

00:10:56: and then also what you can do with it and is very important and then we have a developer relaunch.

00:11:03: A short time ago where we now have.

00:11:06: A lot of tutorials stuff in an app for every part of the vxr is 2K because there's no the only the rendering things like sync up a stereo multi red shading,

00:11:14: we also have audio implemented so surround,

00:11:18: 3D audio so essentially what many people are forgetting is an audio super important for 4vr I mean so many people are saying it audio super important but then in the end it's always treated a little bit like lobster.

00:11:32: Adam again,

00:11:34: rack moralist takes that I'm out of work off of you so you don't have to you don't have to think about your own whatever sweetie audio pipeline but you can use our audio Estacada that's actually based on an RX-8 Racing Engine.

00:11:48: So hey all this is that it's quite a funny story how sound propagates through a room is,

00:11:55: quite similar to halite actually bounced through a room so that way it's right it's it's all it's always and in essence and.

00:12:03: Since you can use Ray tracing for for doing the same thing for graphics I can use that same algorithm and use it in in the audio SDK so that means that.

00:12:15: The full rate at the food trays pause audio get things like River or.

00:12:21: How did how the geometry in the room is is constructed the Echo and also the echo all those all those things,

00:12:27: and that is very important because I mean I don't have to tell you hi Andrea podcast equipment me,

00:12:33: let me know exactly about things like how does a room sound when you're going into a church.

00:12:39: Or into a subway station they sound completely different like the moment even if you would close your eyes and walk into the atmosphere know how far the walls are away,

00:12:50: things like that your brain is very much if it's very much rain do to help provide except it's extremely fascinating to me because we had a project at the University,

00:13:02: a year or two before I trying to reverse the adults not broke that they actually had to abandon,

00:13:08: because it was about the spatial audio it was way too complex to build it 5 years ago,

00:13:13: pretty complex to the point where they simulated exactly one room of the exact position of the loudspeaker and other person I mean I didn't flip out as much as they could do.

00:13:23: In a year or two and half a one year ago on the seabed I got outside to have a discussion with someone about exactly that talking it was like saying it's not it's not possible to write sounds just too much or a conduit on Flex text the Rugrats Hardware just throw some,

00:13:37: the grace and Power on it and I also he was under some kind of magic is Decatur from Nvidia this can do it cause it's not.

00:13:44: It is always an approximation of the real world because physics a pretty complex.

00:13:50: But that approximation from Ray-Ray costing is pretty is pretty powerful so that is that is definitely helpful,

00:13:55: in fear I have a follow-up question to the fluid simulation and other simulators that you doing with your physics how.

00:14:02: Realistic idle simulations how big are proximation do you need to take.

00:14:06: For your simulator or are you taking for your simulations it depends on how much of a graphic for you are you doing wherever some people are.

00:14:14: Really like it cannot be as realistic as it is okay or you will even if you would have a perfect simulation those people would probably say exactly,

00:14:24: this is not this doesn't look like his right now what else does the SDK provide accept audio in geometry,

00:14:32: so this does a couple things umm first of all that your part is is is taken care of by single past area,

00:14:41: the name let me have things that put lower pressure on the pixel Shader.

00:14:46: So those are multi rest trading and blends Blends match trading and these.

00:14:52: There's a special case in VR because you looking through lenses edit display.

00:14:57: There's warping involved the end the warping isn't it part of the image that are on the display that you don't need to render because you don't see them anyway because they vanish in your peripheral vision and multi rotating takes.

00:15:12: That into account and profits from it so I'm what months does it take the outer part of the image that are in the powerful illusion anyway that the lot.

00:15:21: And renders that part at at lower resolution.

00:15:25: So essentially what you what you what you can do is you save resolution in the end that I can freeze up performance for other for other applications is like 48 rendering but your head Direction,

00:15:35: answers in a certain way yes it's it is always fake in a min.

00:15:43: In Mexico but with with Pascal will you to do something called lens matte shading where you're not,

00:15:48: you're not taking the outer parts of the image and render them at lower resolution and the middle part at Justin on the resolution,

00:15:55: but you have a dynamically changing ordinarily changing resolution and across the across the image.

00:16:05: But what that means is that.

00:16:07: You save even you say even a bit more performance but what's really cool is that in since since it's lens match to actually get a clearer image as well.

00:16:16: So because you hurt your rendering the image to the display in a way that's perfectly aligned with how the lenses of the different Texas,

00:16:24: do you want that image you got it you get a clearer image so you basically have a different,

00:16:29: I'll go read messages a 400 closed for 5:40 a week and for always we are so we're supporting was opening all these all these three things.

00:16:37: Answer that so that's geometry pixel shooter part and we have something called preemption which makes an asynchronous Time Warp more efficient.

00:16:46: With this is for your concert it's a bit hard to do.

00:16:50: Let me try to explain very simple and if it's fine we'll just stick to it basically in case with a gear if you put it on your shake your head very strongly you see the image can be,

00:17:01: rendering V to basically something she's a black around now and a headset does not have this,

00:17:07: Time Warp you basically what you see the stable image and you would feel sick but there you see some parts of the image black because the headset moved.

00:17:16: And the calculation of the movement was perceived and then just didn't manage to calculate the whole picture in time so you basically have,

00:17:25: the top of the image missing but you still have the rotation of your head.

00:17:28: Is it sitting at a desk that's that's almost correct it's not it's not that big of a part so you never with with no traffic report you never see half of your internet like and the cool thing about as in quarters time with that you're not looking for it,

00:17:42: very precisely you probably don't realize it and let me try to explain it in another way I'm when you're looking at home.

00:17:48: How are the Armature generated for the headset and there's the most part in this the CPU part and then there's the GPU party at all if your rendering at 90 FPS is that means that each frame has 11.

00:18:02: Milliseconds to be processed during those 11 milliseconds you could have changed your head,

00:18:10: position so what that means is that the post that the sleep you get at the start of the rendering cycle.

00:18:17: Might be too friend from the head post that the user has after those 11 milliseconds have taken.

00:18:24: And what you can do with a synchronous time with you look at the very last point right before sending the image from the GPU to the headset to take a last look at the head post.

00:18:35: And then you shift that image a tiny little bit that is in in is in excess of what you choose what you just all sold and told the audience.

00:18:43: What you can do with a with a preemption in Pascal is usually when looking at that final head pose.

00:18:51: Usually you have to take a very conservative preemption request because you have to wait until the last drop all of the GPU has finished.

00:18:59: In Pascal you don't have to wait anymore for that for that and rockhold to finish so we can interrupt the GPU at a very fine granularity okay what that means that you get closer to the to the scanner to the send out of the image to the display,

00:19:13: and then take a very very last second you with the head post that is and that and that lowers perceive latency.

00:19:19: How does interact with the time warp from a closed and it's actually it's actually implemented with that so I'm here you can you can use but it doesn't it doesn't interfere in any way it's just a find a way to stop the GPU and.

00:19:33: Put in that that the dead apples the information that might have change and and Shifty image just according to the is there any differences in implementation between alcaraz Fife and so on.

00:19:43: The game engines will take care of it right a game engine usually takes care of it and this for your question for a hard question for SDK guys all day they know what they know exactly what the,

00:19:54: what what type of what type of experiences are working on which headsets because as you know older the market is a little bit fragmented right now there's not like one Baseline API that developers and how many factors can talk to.

00:20:08: So it's a little bit of a tricky an approach of integrating a different approach to make.

00:20:15: One Mystic is basically should work with everything sophisticated things for the.

00:20:27: The graphics I don't think that's also VR SLI which uses to GPU that was my next question actually sell is a system where you used to GPU Bridge them together and they're working in unison NM,

00:20:41: does a separate does a separate the rendering I literally rendering exactly do you can you can set up your SLI in a way that,

00:20:48: the the one keep YouTube you number ones and renders left eye and GPU number to rent as the right eye,

00:20:54: Adele music TT performance essentially and that's that's also that's awesome Porton but it's it's more of a small it's more of a small minority of the market that actually has SLI but it brings a lot of values,

00:21:07: yeah if you if you if you want to put out something really cool app with lots of geometry you lots of lots of shaders.

00:21:15: You want to have a snowman because it's right,

00:21:19: the physics I guess also supporting we are physics is also part so I'm the main the main things that we need it be cover our Graphics audio that's what we call the physics AP ISO haptics also.

00:21:31: And then I'm 360 360 video content is also is also part and then a proper physics simulation which is also important in physics is is part of two things so the first of all.

00:21:41: Physics simulation if you were in a virtual reality experience,

00:21:45: usually physics effect have to be even more convincing than a traditional to escreen because your brain again is very much strain on exactly knowing how the real world works the for example if you splash of water or a bucket of paint onto a wall,

00:21:59: you know exactly how that paint will behave you don't have to like there's an image in your head of how the powder fluid behaves.

00:22:06: And if it's different in in virtual reality that lowers your sense of immersion,

00:22:11: if you if you Splash pocket on the wall that looks super we're never going to be like what is this the real life and your brain immediately loses the immersion and effects of Illusion,

00:22:21: now before we go mortuza industrial use cases of Christmas so either about to ask that question,

00:22:28: so Nvidia is also heavily in the hallway our business right if yes then could you say that the same as Frozen buckets and the,

00:22:36: color splashing you would also expect the emphasis to behave smarter and then you could also reuse a GPU for that because it provides a certain Cuda cores for you.

00:22:46: Search SM. There's two things that you could that you could do without with more than GPS because GPS aren't you say anymore just for 3D rendering,

00:22:53: and they can also be used for general purposes a calculations which is GP GPU,

00:22:59: and with the things that you just named Rai so you could make NPCs smarter they could behave in a way with,

00:23:06: just imagine a game like Skyrim you could actually talk to the people,

00:23:11: not by clicking one of the three options but just going there and saying hey how's your day and the AI will know how the day of the NPC was whether he was attacked or whether he had a good day on the farm that's okay and then,

00:23:25: what are rotated arrow to the knee very nice can I order that I would love to play as one of the.

00:23:35: Big topics so I mean you guys are all tech tech and hands yes I mean I'm at Luma is one of these things that where that might,

00:23:43: revolutionize a ton of a ton of Industries and also the lifestyle of a lot of people so just imagine and an AI that the.

00:23:51: Researchvr Maids for you and tell you the most important stuff within 5 minutes I will pay for that and made it a whole topic.

00:24:00: A blue white collar workers being,

00:24:03: working jobs being removed because I I can go with you if it's going to happen one way or another it's just,

00:24:10: quite fascinating that I'm coming from the counter science perspective.

00:24:15: The GPU enable you to have Flex surgery for cheaper times more Foster a simulation of Elsa's is deep neural networks be so hyped I just basically possible because of GPU acceleration otherwise,

00:24:26: skin outruns of us if you send general the whole science of yeah I was pushed strongly by,

00:24:32: super strong the end from we were very very lucky position I mean,

00:24:36: guys better know if you guys watch the NBA of stock price in the last couple of hundred 40% it's it's pretty it's it's between the pretty well and,

00:24:45: I only joined the company as seven months ago but I completely understand why the stock price went that way because we had we're in like a half an Integra in each Tesla.

00:24:54: Don't know it's not only that it's more the general the general direction that the that the company will head in M because we're in the position where we have a CEO that,

00:25:03: realize that he's got these pills are perfectly suited for being for being amusing conjunction with or without GPU zandig things like AI,

00:25:13: autonomous driving peep learning those are all things that that will take a huge part of the future yes and Nvidia set up very well in terms of in terms of,

00:25:23: profiting from that from the market growth and learning is the other thing that I just wanted to talk and talk about an hour in our Elder Scrolls Skyrim example imagine,

00:25:33: I'm just from the way you behave with the NPCs the machine learns about your character and then,

00:25:41: uses that ended that information that is gathered about your behavior.

00:25:45: To be reflected in the world so you have you don't only have NPCs that are smart that they got a real with a smile that you're saying but you also have,

00:25:55: the world in general reacting towards your behaviour which is something that is that is that is that that I would be pretty great and of your experience but they keep learning and something that's that's a ton of other Industries yes definitely,

00:26:09: weekend I think of this point say that there are closed HTC and other Hardware manufacturers that provides the hardware for the developer was basically no Cutting Edge as much as I can and you obviously also have the.

00:26:21: GPU and CPU power that has to be really good and what we so far learn from you as it through tricks and certain.

00:26:28: Improvement of the GPU architecture also extra features you can support we are even more now let's go for some time away from the gaming example.

00:26:37: Can I actually have a stupid question and I am absolutely oblivious to how technology work and bring it totally for using the technology.

00:26:46: Can you tell me.

00:26:48: What's the difference between CPU and GPU with a central processing unit and sheep use a graphics Processing Unit I know but I can basically give it different directed different commands with the desk when spear,

00:27:03: he gets better it's right and then GPU with rather,

00:27:07: fixed a calculation of the desert is rather just geometries a Dominic was referring to and you can reuse it for the journal that works but you can't do all the calculation of GPU is doing on the GPU.

00:27:19: Parts of a 6600 ICP was more of a general approach as you can search the a sheep you could also do the stuff today today traffic processing unit is doing all that a physics Processing Unit stood but very slow.

00:27:31: Then again it's more versatile so you can use it for any type of for any type of a computation a GPU was very much specialised on,

00:27:41: particular calculations so for example for a close shave for the Crusader you always you always use the same kind of algorithms and if you if you the hardware in a way that it's that it's that it can.

00:27:56: Ruffin from the fact that it knows how the software is running it.

00:28:00: Have you get more performance out of it so he sent you a GPU is more specialized but with things like Buddha and and the cheapy cheapy movement that that all that all turned out to change so right now you can,

00:28:12: use if you speak to the to the hardware the right way through the 8th head I tied house that can be fragmented indifference are always exactly if you do and realizing or,

00:28:22: you do something in Photoshop with need to perform a filter picture wise you can basically accelerate each pixel because the Jeep you can do all the pics of symmetry what is if you had to do it over and over fridge picks the right.

00:28:33: What is it GPU a food Drink Machine like a sassy puke and Kennedy Pew technically Tech with everything or is it really is upset,

00:28:42: I think it's a subset I'm glad it's again within that sucks that it's very very Paul that you have,

00:28:49: tons of computation power because they empty pews made bigger steps in terms of raw processing power than that CPUs that because they were able to specialize first and do a special calculations and in from those you were able to use that to use a damn the knowledge you gained or actually general purpose,

00:29:07: general purposes as well thank you for Naturals,

00:29:16: the power that's coming out of the outlet or the frames that are being displayed on the on the on the on the PC monitor,

00:29:23: so it's it's a completely valid question and I'm not even sure if I if I if I answered that correction computer correctly because that there's the whole topic where there's there's a ton of things for the other than ideas,

00:29:35: know exactly that two more industrial applications that's all every topic in science applications already mentioned a deep learning and simulation of their things,

00:29:45: now if you are a production planner,

00:29:49: or you are someone working with huge data to simulate an order for determining the flow of fluids and the Machine.

00:29:57: How can you benefit when you come by as we are too kind of want to look around from leave your friend from the GPU and how is Leah dressings this particular use case.

00:30:07: I got ya keep it was for professional use castor Isaiah Coldplay.

00:30:11: I'm through the quadro serious so serious is something that is especially for professional developers on the one-handed has more processing power and more memory.

00:30:23: I'm the new the new the usual the usual product line what you also get is.

00:30:29: When you're running your application on just normal consumer-grade Hardware there's no weed weekend,

00:30:37: like we can't guarantee that your software will run Stapley on that Hardware her because especially if you're going into research or or professional application is usually not,

00:30:47: easy stuff that you doing it's complex near might involve some new algorithms to name it what you can do with with a Quadro series you get a guaranteed,

00:30:57: that you're cold with run on these Hardware on on that Hardware stable so you get a guarantee and you also get all the support So if you're if you're buying a professional graphic so you obviously One Professional support as well as going to a trade show and your coaches and running,

00:31:14: it said we're going to call his aquadro don't even video Ghostbusters Nvidia one of those two a great time,

00:31:22: people companies does a Quadro GPU is also soles of your features that you described before.

00:31:27: I'm actually has a couple more there's some there's a couple things that are only meant for for the for the for professional for professional developers and.

00:31:36: If you're if you're looking at anything that's B2B or professional the quarter was here is definitely is the best a better choice,

00:31:43: it's it's it's obviously more expensive because you got all that all that extra all that an extra bottom one of the things we show that teaches in Amsterdam for example was running onto Quadro gpus.

00:31:54: 2 of a pascal gpus,

00:31:57: Adam it was essentially Academy Julie station so that was a complete car model with each individual tile with peace,

00:32:07: no matter whether it's safe and it wasn't a CD format it wasn't cat format life tessellation.

00:32:15: Exactly and I'm calling as well as hell until audience,

00:32:19: CD files are not as triangles that you usually would know from the mesh or from the Treaty of your half-brother mathematical formulas exact in order to display is that you need to.

00:32:33: Tessellated anticipation means the correct me if I'm wrong basically going from the nerves form of triangle right click,

00:32:40: and after you did that you end up because there are so many small parts and see if he is not optimized to be very well run about in the yard and there's a lot of stuff that you don't see what usually happens in games Azad,

00:32:53: Things That You Don't See get called away like an totaling and you showed some kind of very aggressive feelings that you had there or,

00:33:01: it was again I can't go into too much it is because the Power Pro visualization team was building that but they were that they build a very very powerful calling algorithm that actually allows you to.

00:33:12: Display the whole model with a thing 17 million polygons which is very alluring it's a lot so it a typical a typical game seed has,

00:33:21: 10 million head million if it if it come if come see if there's a lot happening on an interrelation when you have a gear VR application have to limit yourself to 100K.

00:33:30: So just three later,

00:33:32: so at Daytona polygon CSO to speak and that is something you only get out of the out of the Quad roll out of the coral seriously do if you want to do something professional like that that is that is probably your best and your best choice and then,

00:33:45: three to step back once and I forgot one thing about the about the vxr estoque,

00:33:51: that would be nice we're talking about 360 video content so and we have a 360 video SDK that allows you to do real-time stitching on a GPU.

00:34:02: Which if you're a producer of 360 video how he is pretty helpful because what it the usual production worker that you have right now is you with your big ball of cameras like GoPro so whatever inside of the scene,

00:34:14: then you shoot the scene and then in the end you stated that you look at it through equirectangular image or through a VR headset and only then you realize that.

00:34:24: Oh damn it we missed that shot that it had that work in the in the way that we we thought it would,

00:34:29: what you can do with with Reese on stitching on the on the GPU as you go out into the production set into the field and you put up your camera and you can,

00:34:36: lifesteps dad that the image and you can essentially see to the seams were dusted at the dust the setup of the actors were,

00:34:43: are they gonna close to the camera or too far away you can also stream it directly you can also stream it directly and a Hot Topic right now there's GPU acceleration for thing like stitching for example I just really impressive.

00:34:55: Lifeshield.

00:34:56: The hell that you want to talk about like I always want to talk about this or anything,

00:35:09: I'm the only the only things that I can I can say about life use is that it's definitely something that's interesting I mean obviously it could,

00:35:17: with the fiber content generation that I miss very important the future I mean to give a contact lens begin a life just imagine we just,

00:35:25: spoke about the city files and have this huge Carly kind of want to figure out whether it looks right when there's some kind of different light shining through basic you want to understand how the object looks Landslide are there any Reflections at our noise the driver tried to do that for to read a stick,

00:35:42: either have to have one frame per day or build it all the play I guess but then you would have protected lightfield you're basically,

00:35:52: going from different directions to for someone who doesn't know what I thought I guess you could describe it if you have a camera and you have to refocus it.

00:35:59: I usually have only one focus with light camera you have all the food was so so you could what you usually do with with a zoom on the camera and its focus you can refocus it afterwards.

00:36:08: And the camera just kept just one picture but the light fuse in terms of a scene or basically this information from every angle and every direction,

00:36:15: which sounds a lot of gigabytes of data you can also come present but if you would have that would have if she photorealistic.

00:36:22: I'm precomputed life you and you could actually walk around the car and look everywhere with like look super awesome.

00:36:28: But that's pretty tough to do right it's protect you because it's computation a very very expensive so you have to do a lot of calculation to have a fool to have a full life field.

00:36:38: The thing is I'm there are two things that we're doing it Nvidia one of them,

00:36:42: we were actually one of the first ones to develop a light fuse display so it's half of that you have that information so you have a volume that you recorded.

00:36:54: We're all the different lightfield informations are in ceilings the video of from YouTube it's really awesome it's pretty cool yeah what you have.

00:37:01: Is a fatigue of ice right when you look all the time at 1oak explain explain your eyes get tired and this is what an h and b right now it is but just imagine you could actually focus and this is something that I'm sitting the future the Prototype you showed to do right,

00:37:16: we're going to do a little experiment with the audience the weekend actually explain what what what we are what we just talking about if everyone of you and Petr Chris please do that as well and put one finger in front of your head about 30 centimeters stats,

00:37:30: how many inches 10 inches about 10 inches away from your from let her face just one finger and close one eye if you can do that and I'll focus on the finger.

00:37:39: And then focus on the stuff that's behind it you'll realize that yet that your eyes are actually focusing on different planes and the.

00:37:48: You're focusing on the backplane your finger gets blurred.

00:37:52: And that's essentially something you can replicate in VR headsets right now because you only looking at one Focus plain everything is always crystal clear and I'll be alright Sam doesn't matter where you look exactly it doesn't matter what your eyes are focusing on if you're using a Life Fitness play that's the thing that we build a tear that our Tech Team at Nvidia Bill and,

00:38:12: I could actually do that so you can focus on different parts of the Eminem and the other parts of the image will look blurry,

00:38:20: just like your your eye is is working in the real in the real world. It's a more convincing way to different screens and some kind of,

00:38:30: membrane that was in circulating a different screens behind each other information.

00:38:38: On which plane do I look restaurant your eyes your eyes mytranshealth that's what my tracking do you need to have.

00:38:44: Wait what is it in a VR headset you have to have eye-tracking wife simulate the blower you could simulate the blower I mean in Sinking simulator but if you have a lot to displayed on your the night with you if you if you had a few display you don't need the address because your eyes are just focusing on different planes with in the display,

00:39:02: imagine a stack of transparent displays with Debby Ryan the other was points with different Focus points and your eyes your eyes can actually do that they can focus.

00:39:11: Whatever 10 millimeters away twenty meters away and I many millimeters away to twenty meters away okay bye bye heading different folks place that you can that you can focus your eyes on,

00:39:21: Tech City get the feeling of hate and looking not at display but I'm looking more through a window at a real happy I could use a night wrecker and a normal displaying you basically with tracks of pupil dilation outside.

00:39:33: But it's not perfect because I can also change as a crystal inside basic it was in my phone so it's not just on his pupil dilation Preserve,

00:39:42: Tech more muscles and more effects of volatile you receive your vision on that short distance pupil is no longer at reliable TrackingPoint.

00:39:50: Probably not to the degree that that it's a really realistic but I mean it. The similar as some of the thing has been has been intimate in games for a long time we just called depth of field which essentially means that in a game like Skyrim for example if you looking at the town it's right in front of you.

00:40:04: All the parts are getting blurred from if your Crosshair is on the mountain Parts on the mountain range in the distance the town part in the front gets blurred,

00:40:12: but you could you could use a you could use a tracking system will you just look at it as a gay where is your I looking at the very moment I am turning the train that it's looking at used to see proper information of how far away the,

00:40:25: at at at at parties and then you could you could.

00:40:28: Lowered however is its with blurry. Still with not fix the fact that the screen is just fix the front of your eyes,

00:40:37: your ice in a subpoena subconscious way will always realize that they're not going to realize technically it's technically actually bad for the eyes.

00:40:47: There's if there's a system where convergence and focus of the is is is a system that intertwined.

00:40:55: Individual individual system so if you do the same experiment that we just did with two i's.

00:41:00: I'm you realize that what you're looking with two eyes open sorry if you really realize that if you're looking at the back playing again you see to the finger at convergence when,

00:41:10: and that that that I'm the movement of the eyes looking back and forth between the two planes is very much intertwined so as soon as you converting a different plane,

00:41:19: also focusing on SSO if you can virtual the different object you always you already,

00:41:23: focusing on to that and after that thing at the same time and that that is that is something that is not being replicated in Virtual really gets its right now so you are converging.

00:41:34: But your eye Focus isn't changing and that could be a problem for it's not it's a scientific proven right now.

00:41:41: This could be a problem for children that are still developing that into the intertwined mechanism is a where is a headset basically state.

00:41:54: Look at BR more than the new real life in general.

00:42:05: Long as it's not too long it's not a long time it's not a big problem for children do you get here,

00:42:11: I'm in the same if you're looking at if you looking at a TV all day like you're also just focusing on plane and training your eyes anyway there's no natural that's right that's right natural moving and walls when you're consuming media,

00:42:24: the nigras are you just look out for each other and I can put it for a day on that morning when our nothing bad will happen if time I talk to the Tree of Life.

00:42:34: Are we actually have a in the ra rendering at something at that could be interesting to you what it does it is um,

00:42:41: a volume in which you can move around so I even Envy are you can move your head around and you get,

00:42:49: super photo-realistic Reflections because it's not right there in real time the whole scene or the whole life field volume is pre-rendered,

00:42:58: and I don't get the output at the moment that that the that the that the user looks at him okay that's a plus side as you have Riu photorealistic rendering so you can do things like Global illumination raining relaxing whatever so.

00:43:10: Each ray of light in your eye exactly in the way that you would expect it to and I mean there's also a lot of things I can up to my son's game and she loved it so you describe the lot that you can do in the GPU level,

00:43:21: are there also few things that you can,

00:43:23: do ants have just mentioned goes in that direction you can basically Tech LED light recalculate Shadows the pre calculator clusion Electric in half illusion that thing I can have that snaps in half.

00:43:35: Oh different ways of basically are tending the GPU with a Randall s yep so you can do it before and the steps to describe the already happening on the GPS so basically like a multi-step optimization you have to do the tree Azad and you can also,

00:43:48: basically take a different shaders for example not for Global anti-aging but just for the tech source of generating solar attic a lot of smoke weeks here and there the question is if you're not doing anything else that not doing any features.

00:44:02: And you're doing everything what's the performance difference like I'm very stupid guy in Die Hard,

00:44:08: destroying with a screwdriver all those chips that you care for infants GPU and roll performance where I make a super smart guy that I read all those two cases up to be patience we have literally the same scene.

00:44:19: What would be the difference like on a person 200% it's a very hard I'm not going to be able to give you and just because from my experience as a developer.

00:44:28: She is there something you need and definition 10% at least half of War,

00:44:32: Legacy more when you take a look if you if you do all the rendering tricks in it it it also it it depends on.

00:44:39: Depends on your measurements so if you're if you're taking how realistic it looks for example as a measurement it's very hard to tell different from Metals per second.

00:44:49: Particles that are not in use the amount of Netflix a the amount of particles the amount of geometry and the amount of pixel Shader power that you have.

00:44:59: You get you just kidding it's what you want to have more particles so you want to have maybe something else just getting more conversational powers that you can use to improve something in the scene.

00:45:09: Okay so if you're if using an app in a correct way all these KP eyes will go up,

00:45:13: penfold in love you in a hundred fold for the particle effect for example because if you're using Justin all the sleep you put exhaust and there's not a lot happening on sweet if you're going for a food GPU system,

00:45:25: probably in the hundred fold off of FM,

00:45:28: particle effects or up the amount of particles that you can have on screen at the same time it's hard to tell because.

00:45:37: Well some people some people can Implement stuff even without reading it and if it has 2K tutorials right,

00:45:43: that is magicians I mean John, Caribbean with Angelina something more wire it up different,

00:45:56: 34n warranty fried warranty voided and I don't think he cares,

00:46:04: unless it's it's it's it's hard to tell but in the end.

00:46:07: It really doesn't it doesn't make that much of a difference is 5% or 10% or large you have more more performance test,

00:46:14: the class of resources that you have to put into optimizing your experience to put it out on the market for England yet and it's not to say that you can be lazy,

00:46:24: sure with that but but but he's a resourceful for other parts now.

00:46:30: When we speak about this features of the GPU that I mean you're not changing your GPU is every,

00:46:35: that's right so I guess even if you would have right now have some kind of very good and also features that you developed and you already to put the Jeep you need to wait for the next iteration.

00:46:45: From the manufacturers like an HTC you lost synchronize with them.

00:46:51: How difficult is are just going to go after those pictures you no no no no rrr Tech teams and especially the sdksdk guys are super with a line so they are working very closely with our kiddos nhti mean both.

00:47:05: Both sets of Manufacturers are all they have too many factors need us and we need them for they are just succeed so it's a mutually beneficial relationship and if we can make,

00:47:16: we can do something from our side to make the VR experience more performance and that's more immersive we going to do it,

00:47:23: and if they have the same opportunity they going to do it ahead and go for it so I'm obviously we were talking with talking very closely to to HTC and Acuras on a tech level to make sure that the rendering tricks that we are putting into our Jeep user actually being used,

00:47:38: I the developer said HTC in an alternative to Microsoft and their hmd.

00:47:44: Of course my Microsoft is also I mean Microsoft's announced has announced a complete product line of VR headsets that can be produced by different manufacturers cuz they also have to somehow with talk to you right,

00:47:55: course I mean we're all we're all running we're all running off of the office the same PC Today using some kind of Their Own,

00:48:03: driver that talks to the GPU because I guess I mean,

00:48:07: there's is open the are right and basically run across and HTC and always be Orlando svr yes but I don't think that the Microsoft has its with you the try to have their own implementation.

00:48:20: Because it's something that bothers me.

00:48:22: Correct me if I'm wrong but I don't think they actually talked about the there and their plans that much but we're going to have to reinvent the wheel and I mean I'm going to get into days actually my phone.

00:48:34: Phone number,

00:48:37: Tech from what I read is also basically the same level does it take a 2 from that's okay because they just had to read mans we own stuff and with Microsoft us to do it over and over again what would you do it just implemented.

00:48:47: I don't know is it with labret so basic is Operation the system can take care of it you just use our trip.

00:48:53: Maybe it'll take steps to figure out I always EconTalk for for Michaels or um butt,

00:48:58: just just generally from from from what they've what they've try to accomplish in the last couple of years with their business model as they want to provide General AP is like that X for example.

00:49:10: They want to make sure that and there's something I can I can easily say they want to make sure that that people are using a new species and I'm sure they want to have.

00:49:19: A PS4 for developers so they will use the Vanos ecosystem understand and I think Microsoft is is on track,

00:49:29: m2c virtual reality as one of those Peak Industries for the future and honestly all day they will,

00:49:35: everyone to provide again tools for developers but it was kind of form and I Condrey today but let's be honest do you see any other way other than Windows to use for the AR,

00:49:48: because the make machines don't have enough power,

00:49:50: I can't talk about it I just don't have right now right now it's it's it's pretty hard to run we are on.

00:50:01: Systems that are inspect pretty high-end I mean it's a high-end specs,

00:50:09: what you could do is enable machines that don't have crazy keep you powers in the future through doing call Computing for example of so I don't,

00:50:19: bear with me latency.

00:50:22: We we announced at CES this year our RC whole jar of them announced G-Force now for PCs and for Max so it said she with the means of each machine,

00:50:35: virtual each machine besides lyrics and that is out there on the market and the command the consumer space at least and can run.

00:50:43: Little shoe 3E at crazy high resolution is basically a huge rendering of farms renters game and you seem it and,

00:50:53: good luck to you every other company the try to failed because while you're doing this again for free and I get the idea I guess.

00:51:01: How to make user totally something else right,

00:51:04: but how with me is always fascinating like every other companies a try it so far as literally have not been successful,

00:51:11: I have actually fill out those adult streaming so I followed by car before they were they were bought by Sony with the snow the PlayStation Now the right now,

00:51:20: and also on life which were the first ones to do it.

00:51:23: I know I was totally convinced of well this is going to be the future this is awesome like this is so good I actually had the chance and it was five years ago or something I have to chance to try and I played Mass.

00:51:37: A breaker.

00:51:38: Why didn't you miss the good like they were they took off like I've been in it since it took off they were bought by Sony for 380 million-dollar if I recall correctly.

00:51:49: So they they build a lot of Toddlers and he was supposed to GPU but we had pretty good internet,

00:51:55: Pennsylvania cell I think something like a 6 megabyte actually got a lot of stuff over that it's okay if you want to go if you want to go higher than that,

00:52:06: or latency which brings us back to the topic of reality it's going to be a tough thing to do because,

00:52:14: as you know the rendering pipeline takes should be below 20 minutes I can so that the latency 11 milliseconds to get everything 20 milliseconds is fine,

00:52:24: but if you're doing a roll a whole tour like if you're sending the the the the Sheep you and the whole state of off to the server somewhere across the world.

00:52:34: I'm that would probably be a little bit more tricky because then the whole round trip of that information that a surgery for a little while.

00:52:45: Black Sails go over some of the games exactly you're basically getting 30 40 15 minutes seconds I mean usually have some kind of.

00:52:54: Crappy provider Fireballs whatever it's never going to be kill we need we need a little bit more time to leave,

00:53:00: to action figure out what you can Outsource to the gala we need we need better connection we need better Bend with we need better infrastructure that she one of the topics,

00:53:09: last week at at the Munich.

00:53:13: Last week at the Y I was actually I'm talking to on the EU commissioner for Digital Society I have a thing of admitting and first of all to just get rid of the mean he's actually talking very well it's like there's this old man from the,

00:53:27: 5 years ago or something where they had a speech of him which was Supply out of not the best thing you should you could have now he's like we switch back and forth between German and English,

00:53:35: just to do a little bit the storm hits and I mean his agenda is essentially.

00:53:43: Put on a good foundation for future Technologies in Europe and I made very clear that if we want to go the route of having streamable VR experience that we need,

00:53:56: way better infrastructure yes we were there for people in the pendulum there was a.

00:54:02: Tools from okay so he was kind of the day our side of things that was a cable provider and also the moderator which was from the Wall Street Journal and we all four basic he told miss a thing as well if we with the flooding,

00:54:15: better infrastructure not only for VR for a are for autonomous driving I'm so late,

00:54:21: like it's the foundation like Anna Janae imagine a country at the start of the Industrial Revolution that didn't have steam engines because they just didn't have the money to pay them for doing the electric the HP like a a government that wasn't able to provide power lines now.

00:54:37: Like so many Industries are build on pollen,

00:54:40: so many Industries will be build on good internet providers already on the internet information is only one,

00:54:49: 50 years and years old like that so we are Norma's and already said that people are getting it and I'm very happy.

00:54:57: You told a temperature reading there an app the more people tell with the mr. Bojangles the more is a chance he might get some point.

00:55:03: For our American view Ellis is probably a very very weird topic with you but it's the end of the podcast not much.

00:55:16: they have their own problems that you do not have.

00:55:19: Food City city coverage for a single provider the loss of a different part of a different one is one small a glimpse into the future.

00:55:32: 5G networks are being rolled out to 20/20 okay are actually load they are actually super low latency,

00:55:39: which which kind of surprised me at first but it makes total sense because in the future we want to run things like old enormous driving for example over and over mobile networks and if you want if you wanted to have something where at,

00:55:52: cars that are in the stone the same street at talk to each other,

00:55:56: you have to have super low latency metaphor for things like a sample,

00:56:03: are Foster online Yahoo Violet the number of fiber go into my home from an infrastructure standpoint around the neckline.

00:56:17: Then to just put up in a tower and then and then stream it through the air now but at the desk for for mobile the or something like that or are going to help we went to.

00:56:29: With it but I hope you enjoyed it,

00:56:33: well as Thursday we talked about rendering challenges and one of them put one of them one of them is latency in the future once again any last words famous,

00:56:46: I'm always enjoying this guy so I hope the audience head head head head ever had a good fun with us the next time next time we were talking about a different topic I guess if you can I heard it said a kiss at UK Shin that's one of my pet peeves in,

00:56:59: well I will tell you I still would like to talk about their haptics,

00:57:03: feature that you're providing but that might be at next podcast I'll either Eclipse it's actually pretty short so we are using our existing physics SDK which already simulates home,

00:57:13: objects are interacting with each other in a convincing way and then you use that information for doing the same thing with motion controllers for example so you can,

00:57:21: you know how chicks are interacting you can also use the same information to provide the.

00:57:28: Simulation of how a baby the user is interacting with that okay we said she poops on that and it's meant to be a haptics layer with,

00:57:35: 10B work with with all the different emotion controllers for half an hour or so.

00:57:44: So I would say thank you very much for joining.

00:57:48: Music.

New comment

Your name or nickname, will be shown publicly
At least 10 characters long
By submitting your comment you agree that the content of the field "Name or nickname" will be stored and shown publicly next to your comment. Using your real name is optional.