HOW LARGE PIXEL SPACES CONNECT WITH MEDIA SERVERS AND THE ASSOCIATED CHALLENGES

Hello, this is Les Goldberg, and welcome to The Road Ahead. The Road Ahead podcast is dedicated to the future of the live events business, bringing together industry experts. Hello, production world. This is Les Goldberg on The Road Ahead. Today, I have four amazing guests. I have Rodd McLaughlin. He's a media server programmer and screens producer for RGM Productions. I have Cameron Yeary with Thrasos Media, managing partner. I have Patrick Verhey, Director of Media Systems, R&D, and Training for BrainHub, an ETP company. And I have Sean Borowski, Vice President of Systems Innovation, an ETP company. Guys, welcome to the show, and thanks for joining me today. Hey, thanks for having us here. Thanks for having us on, Les. We are going to talk today about pixels, lots of pixels and media servers. This group of people are kindly some of the smartest people that I know that can opine about pixels and large displays. And the world we're living in with LED and projection seems to have relied and kind of expanded in ways in the past year or two post-COVID that this is becoming such an essential part of doing a show. So I'm going to throw out my first question. It's going to go to Cameron Yeary. Cameron, in the shows that you're doing, what are you finding regarding displays? Are you finding more projection or more LED? What's the experience you're having and what size pixel spaces are you working in? So I think that now we're dealing with much more dense LEDs. But if we talk about actual physical size, I still think projection mapping and things like that are, you know, our biggest spaces as far as physicality goes. You know, now we're seeing LED walls in the, you know, 1.9 and 1.8 to range, you know, so your amount of data that you're having to use and utilize to drive that stuff is going up quite a bit. But obviously the size of that stuff, because it's so dense, is much smaller. So what may be a screen that was, you know, 30 feet by 20 feet 10 years ago might be only a 1920 by 1080. That screen could be as much as 8K now. So interesting. 8K. Got it. So we were happy with 4K cameras. Now we're dealing with 8K displays. Rod McLaughlin, what's the size of the pixel spaces you're dealing with? And are you seeing more projection or LED? What are your what's your thoughts? Definitely a lot of primarily, I should say, LED installations or immersive events seem to be heading towards projection. But for corporate stuff, 100% LED and definitely in excess of 32K pixel canvases on that. And what I was going to throw in there was, I believe what we're really running into is a limitation in the content creators ability to output before running into an issue with our canvas sizes. So you're saying the the machines and the display cards were hitting those physical limitations of what they can process as one of the challenges, is that correct? No, actually, what I'm referring to is that, for instance, After Effects has a 32K limit. So for content creators, what they're looking at is for them to be able to create content in a coherent way. In other words, looking at the entire thing as a single canvas, they're limited. They'll suddenly have to split amongst multiple comps or achieve larger than 32K canvases. From our end, we don't care. Just keep pushing. I don't care if it's 64K. We'll just keep adding boxes. So I think the stumbling point is definitely at the content creator's ability to create coherent spaces to create in. Got it. Sean Borowski, would you share your thoughts on LED walls, projection displays and pixel space sizes? Sure, sure. I echo both Cameron and Rodd in that in a lot of live event presentation spaces, we're seeing a heavy transition to LED-driven. The cost factor of LED has come down. The density has also come down. So you have much finer pixel capability. The install time, there's been a lot of innovation in the rapid deployment of LED that has made it both cost-effective and time-effective compared to projection. To echo what both Cameron and Rodd said, in more creative spaces and more creative applications, we're still seeing a considerable amount of projection. And then as we talk about size, again, along the same lines, 8K, 16K, 32K, especially as the density of on-camera material needs to be increased, right? So from the audience perspective, looking at iMAG screens, looking at the environmental LED, those pixel pitches can be a little bit larger. So we could have 2 mil, 3 mil, 4 mil, 8 mil. But when we have on-camera wall or projection, that needs to be ultra-fine to avoid any kind of artifacting and such. And so those are pushing a lot of our resolutions up. So, yeah, that's approximately what we're seeing right now, both in the live event space as well as in the install space. Patrick Verhey, could you share with me your thoughts on projection, LED, and media pixel spaces? I agree with that, guys. We did see a big change in how high-res those screens are by now. I mean, like 10 years ago, we would consider a 12 mil LED pitch being a high-res screen. Nowadays, we'll probably talk about a 1.2 mil being a normal screen size. Looking at projection versus LED, the goal I've seen in the past, mainly in my main field as corporate shows, was that if it's a straight, flat screen, most of those people went with LED just because of the price, the ease, and the brightness. In broadcast, with cameras, it's always a bit of an issue. But now with the high-res screens, the Moray effect isn't such a big issue anymore. So we see more and more LED screens there as well. But we still see a lot of projection stuff when it comes to projection mapping. So whenever it's a non-flat surface, we can go easily with 10, 12, 15, 20 projectors on something big, like buildings or planes or all kinds of things you want to map something on. So this will still be projection stuff. But I also did a lot of combination shows where we had an LED screen as the main audience screen behind the stage, plus some projection as scenic dropouts on the side of the stage to make it look nice and beautiful. Got it. I like the word nice and beautiful. Those are kind of hopefully our producers that we're doing work for are happy and they want nice and beautiful. This next question is to Cameron. Cameron, when you are talking to your teams that are asking you to put content together, what kind of rules or what kind of guidance do you give them about creating media? And what has changed recently as these pixel spaces have gotten bigger? What are your thoughts? I would say render times definitely are a big deal. They need to think well about organization and flexibility. Those are two things that I always warn them of, of don't get too married to something because there's a good chance it's going to have to change. And if you make it so difficult that you can't change it, then we're up a creek. Taking advantage of layering things and putting things in alpha and stuff like that, those are all really good tips for content creators, especially if they're playouts in media servers. The biggest problem, as we keep saying, is that we're moving past spaces where the softwares can actually create the media in one giant pixel space. So that's probably the biggest pain when we reach those kinds of points. Got it. Rodd, what could you add to the idea of content creation, advice you give to people that are doing that? What are your thoughts? Well, I think that one of the things that we're running into the most with our content creators is that content management itself has become more difficult and it's crucial to the process. Understanding the importance of file naming, consistency of those file names, everybody agreeing on that nomenclature and being able to work with that consistently is a huge thing. To Cameron's point, comprehending render times is really, really crucial in understanding how we might address that prior to even starting the project, as Cameron suggests. Does that affect you if you have to make changes and the render time of a change? 100%, absolutely. I mean, various media servers have some workarounds, some shortcuts in that, but I think content creators themselves can run into issues where we say, well, we just want a match frame or we need something to start at this particular frame and then deliver it that way. It becomes, when in doubt, it becomes a content management issue. We're able to work around these things, but the content providers really need to understand that, understanding which files they're delivering when and for what is crucial to the communication. And then finally, I just want to throw in there, because of that content management side, understanding networking has become much more crucial for our content creators as well because they need to be able to serve us that content in a timely manner. So does that mean if you're doing a show and you're playing back with media servers, you might have a separate content farm that is feeding your media servers and you can go to that farm and grab content and pull it in and they can continue to create? Nothing makes me crankier than a content creator showing up with a MacBook rather than something a bit beefier. They should be showing up with numerous computers, not a laptop. So no laptops for content creators. That's my takeaway so far. No single laptops. No single laptops, a laptop farm. Sean, tell me your thoughts and also add a twist to this about permanent installations and these big exhibits, museums, the big environments. What is the situation about this idea of content creators and what advice we give them and limitations, et cetera? Sure, sure. I know that both Rodd and Cam, working with them extensively, deliver amazing content guides. So just to back up a step to say the communication to the content team, client content team, multiple content teams, multiple content teams in order to ensure that everybody is delivering in a way that is digestible and that is replaceable because very rarely is content right the first time we see it on screen. We are often going into iterations and that's what they were kind of talking through, the process of replacing that content either in chunks or in whole because it's the first time that the content creators have seen it on the final palette. They've looked at it on the largest screen they can get in front of their edit station, but seeing it 40 feet tall is a very different experience. And so staying ahead of that delivery strategy, especially in the larger pixel space projects, you're going to often have a content management individual that is not the media server operator, is not the content editor, but someone that is just managing the iterations of the content to ensure that the appropriate video and piece of content is being played back. And then I'd say, just to tack on to that, different playback devices, different media servers utilize different formats, envelopes, and codecs. And so from the front end, when dealing with large pixel spaces, the content guide that the creation team is working from should very clearly outline what the delivery expectations are and what type of formats. So in some of them, as we've talked about, especially when we're not dealing with real-time rendering, when we're dealing with post-rendering, are going to be a bit more render-intensive in order to take the load off of the playback computers. Delivery codecs like Notch LC and HAP are GPU-allocated so that they take advantage of that. When we talk about the long-term installation, this becomes doubly important because we go through multiple teams of content creators that are roving through months of different projects. A standing venue that is going to have a set palette is going to then have many generations of content teams that are applying the content to those surfaces. And so making sure that the delivery specs and the guide and really the book on how we play content, how we receive content, how we update content in this space is super critical. Sounds like no show starts without a content guide. That's like the Bible. Patrick, your thoughts on, you know, we've been talking about media servers and content and the perspective of the designers. What are your thoughts? I would add a step to what Sean just said, like content is rarely the right piece on the first time you play it. I would say it's never right the first time you play it. I had done shows where we ended up with video version 16, 17 or 18 or even higher numbers of the same piece of content being delivered over and over again until they got it right. So if you take it from there, yes, the lead times need to be very well monitored so that the stuff comes in fast enough and on time. And this is one thing where I see a big issue coming up in the last or which came up in the last two, three, four years. is the lead times to a show get shorter and shorter, the preparation time gets shorter and shorter, while at the same time, we increase the pixel spaces. So the render times go up, the content time, producing times go up, and our clients kind of forget about thinking about that. If you double the screen size, you also double the amount of pixels, which means you double the amount of render time to get a file rendered out of any content creation part. And then at the same time, people started to use content creators from all over the world. And we get a lot of those files sent over by the internet. I've been downloading for a job in Qatar, eight terabytes of content over a local Qatari network line, which kept on dropping out every half an hour. So keep that in mind, if you do that in your planning phase of your show, make sure that you have a proper internet line if you want to have all of the content downloaded from the internet. And what Sean said is completely right. Have a second person or even a third person in your media server team. Don't load everything on the media server operator. I'm a big fan of multi-user sessions on the media server system. So you have two people, one or even three people, one programming the show, one rehearsing the show, and one is only taking files in from the content creation team to make sure they are on the right namings, they're on the right position. Double check the file format so that you don't even load the files into the system to find out it's actually the wrong file you've loaded. So a few things to keep in mind when you plan your shows. You know, my takeaway is that this position is underappreciated. There is a lot of detail in making sure the content is delivered properly on the screen in the right place at the right time on cue. Okay, I'm moving on. This question is back to Cameron. Cameron, what's changed in media servers? If you take me through the last 10 years, you know, where were we? And where were we then? And where are we now? And how have we evolved in the media server world? I'm gonna double this with Rod because I have to use someone else's memory as well. Rod, we're adding your memory in here for this, okay? Feel free to opine. I'd say 10 years would be 2013, right? So we were probably at the, getting towards the end of what would be when Rodd and I used a lot of the inbox, which we still think is not a bad box, just not always the right tool for the job. I think, and you think I'm right there, Rodd? 100%. Okay, cool. I think the biggest thing has been for large displaced sources, getting above probably the multiples of 8K, especially when you're dealing with a single source, we've had to move away from the world of Macs because the Macs do not allow you to gang together outputs or gang together machines and keep them in perfect sync like the Windows machine box as well. So for a lot of our large stuff, we have gone exclusively Windows for, for playback. I think recently, probably within the past five years, we've seen some manufacturers move to focus a lot more on TV and film. And then several have moved to software only options, which is kind of an interesting development because you can pretty much decide how much horsepower you really want. And in that route, I see a lot of people moving away from Intel boxes and using AMDs. So they can get the power of PCI-4 and even PCI-5 now, where you get amazing playback speeds and you get amazing bandwidth to really push every bit of a box, which most of them you're dealing with around four or 4K outs. And really be able to drive every single pixel to its limit without having to stack multiple machines. Wow, lots of acronyms and lots of pixels and pushing pixels. Rod, do you have anything to add to Cameron's memory, going back memory lane from where we are there and to today? Well, I would certainly agree with this statement that we do actually hold M-Box in high regard. It was incredibly capable and did a lot of tricks. And I guess my response to the question, what has changed is that tool sets have expanded some, but primarily they all do the same thing, just they make it easier for the operators to do it. The UI has improved greatly. There's better tools within there just to access things. We're not doing things numerically anymore. The other thing that I would say has been added more recently is APIs. To me, APIs are crucial, the ability for the end user to actually customize some of the environment themselves. There's a great deal of benefit to the manufacturer by letting your user base actually build your media server for you. Interesting, interesting. Sean, would you opine media servers from where we were to where we are? Absolutely, looking back from 2013 to now and where we're headed, I think a simple statement to walk away with is we never have enough. We go from one gigabit to 10 gigabit to 25 gigabit networking in order to move these large files. We go from four 1080p outputs to four 4K outputs to walking into the doors of four 8K outputs and we never have enough. And so whatever technological hurdle we overcome, it seems like the client needs continue to fill that vacuum as quickly as we can open it up. And so certainly the hardware has improved and the ability to push many more pixels to keep up with fine pitch LED or very large pixel spaces and projection, the warping and the mapping tools have become more sophisticated. Certainly the focus of many of these manufacturers on the studio spaces or cinematic spaces, we in the live event presentation industry have gained tool sets from those workflows in incorporating things from the gaming world, from the cinematic world, but I'd say it's never enough less. We continue to need more and more pixels. More and more pixels. That sounds like an awesome hashtag. Patrick, tell me from the journey that you've been on, take me back 10 years in the machine and tell me where we are today in your opinion. Yeah, luckily I don't need to dive into my memory too deep because I've just been on a cruise ship to replace 10 year old media server systems to today's media server systems. And just for the nerds, we're going from Windows XP with two terabyte on the graphic card with DVI outputs, which could run 90, 20 by 1080 pixels on one output. And back in those days to nowadays, I think we are at 12 terabyte on the newest NVIDIA graphic cards with synchronized over sync options. We're running DisplayPort 2, which can easily push 4096 pixels in width. So the hardware did make a massive big change. We're almost 10 times more performant than we used to be. I agree, we need more juice than that. We need to have more power because we're going to increase the sizes. We're always pushing things to the edge and hopefully never over the edge. But we definitely increased a lot there. And also from the user interface side, there's a lot of changes. And as Cameron mentioned, a couple of media server manufacturers are moving away from locking the systems to hardware and offering software only solutions, which especially from our point of view as a rental company is a great step because we can have one incredibly powerful computer which can run four or five different systems. And we can utilize this machine much better than if we would have one computer for each system. So from a value point of view, we can push to much better hardware than we used to be able to push to because we are completely free in designing this hardware on our own and making our own computers to the needs we want them to be. Wow. Wow, wow, wow. That's all I can say. I don't think I could really appreciate this until I've listened to this conversation about all the pixels. So I'm going to throw this question to Cameron. Cameron, when we are incorporating switchers like Spyder, E2, Aquilon, what are the associated challenges dealing with these massive pixel spaces? So for those that know me and know the way that I like to push my workflows, I'm not a huge fan of the media servers feeding those products, but I am a huge fan of those products feeding me. But there are shows where they are the exact right tool because it may be crazy, et cetera, et cetera. I think the biggest thing that I see is those machines having to actually, we actually have to utilize more of those than media servers in order to keep up with the bandwidth. I think the E2 is the weakest link of that three is if I want to take a media server and do four 4K outputs into an E2 as well as maybe another 4K graphics, we're not just using one frame of E2, we're using maybe two, maybe three. A Spyder can do a little bit more, but has its own kind of workflows as well as the Equillian. I think what I'm most super excited and I want to see people develop more of are stuff like the Dexon router products where you can start to mix, mix layers that like an E2 or a Spyder would have as well as multi-views, as well as routers all in those 4K worlds. And I think when you start looking at that kind of stuff, you really start to open up a lot of power from the media server side. Got it. Rod, your thoughts? Spyder, E2, Aquilon or any other switching device, what are your thoughts on having to work through them, incorporate them around them, et cetera? Yeah, regardless of the switcher product because it all comes down to the operator in my experience. The switcher operators experience and knowledge of the gear is paramount in my opinion. They need to understand how to accept signals from us in the most optimized way and they need to know how to ask for that. Our side of things tends to be incredibly flexible. So personally, I tend to let them drive that bus. I can say, however you want to accept these signals is fine by me. It's an incredibly simple fix for me to do. So as I say, I feel like getting an experienced person behind that device is really, really crucial. Got it, okay. Sean, your thoughts on connecting switchers to media servers and your thoughts? Sure, so when utilizing high resolution switchers in larger pixel spaces, especially when being driven large pixel resolutions from media servers, one of the advantages of having a high resolution switcher is latency. When we have to bring live sources through a computer, through a media server product, that has to be captured by the capture card and then passed through the computer and then played out to screen. And that takes a little bit more time than what media servers are really great at, having the content internally and just spitting it out. That essentially is a very real-time process. Now we can overcome that often and it often, to Cameron's point of perspective, is advantageous to not utilize high resolution switchers. It lowers the points of potential failure. It lowers the number of operators. One of the things that it adds when we do add high resolution switching is that management of external sources and minimizes latency to screen so that we don't have bad video lag, bad video delays. So that's an advantage. I echo Rodd's point in high resolution switching products, the operator has to identify how the signals are going to come in and how they're going to go out. And there are many, many different ways to slice that apple. The need is to communicate. Critical communication between the media server content team and the screen control team, if they are independent, is crucial so that we are not having to re-figure out resolution and output count as we go into, to Patrick's point, these shortened load-in projects. So making sure that we have a plan, that we stick to the plan is key. I echo Cameron's point that these products hopscotch, and right now, we're due for another hopscotch where we get more outputs and more inputs. But right now, the Barco E2 product, you tend to need more frames when you're deploying as opposed to the Spyder or the Aqualung product. But a good conversation upfront about how screen management and content distribution is going to happen is really the most important. Got it. Patrick, your thoughts? I got to make an aggressive statement and would like to say that the naming high-risk switcher is wrong because they're actually not high-risk switchers. The amount of times I ran into issues of the E2 being out of its capacity and not having enough inputs or outputs or resources is incredibly high. And if you're running a big show and I'm talking about... 20, 30, 40 4K feeds, there's no way you can handle a high-risk switcher in between there. You need to be very smart in your planning and you need to be very considerate in how you want to make those things work. The latency issue is there, yes, for sure, but just because I need to send an iMac to two of my 20 screens, do I need to send every media server feed through a high-risk switcher system or not? Or could I split this up and just feed these two screens from any two or any switcher and send the rest directly out? This is the thing you need to consider. Backup switching is another thing, but again, there's some big router systems in the world which can do this in a nicer way than any of these high-risk switchers. So, yeah, we're lagging behind in the performance of those switchers because I think they're not really high-risk. They run into the limitation of 4K very, very easily. And as I was saying, if you run shows with 30, 40, 50 outputs of 4K out of media servers, you are hitting the wall with these switchers. One more thing I'd love to add to that. Someone please make some really nice, long digital DP cables along with good DP routers. We really need those. You're going to make a fortune if you make those. Okay, we have our new business plan idea that we're going to go build some cables. And so let me go ask you my last question for this group. And let's start with Cameron. Cameron, how much of the process of delivering with media servers on-site with large playback solutions, how much IT do you have to know? If some future person wanted to get in this business, what level of IT knowledge do you have to have to be successful and for producers and technical people to really appreciate to be at the very top, the best in the business? I just want to know your perspective and hear everyone else's. I think, as with everything, it has to do with scale, right? If you're doing a single 3840x1080 screen and you're going to use something like Volumen, you really don't need to know a lot about networking. If you're doing stuff like Patrick was talking about, where you have 34K outputs and you're needing to move lots of data fast and you might have NASes involved, you're going to need to know a lot about networking. And that's not even talking about the idea that we're starting to really get into the world of 2110 and NDI on use on show site, which is a whole other world that opens up difficult IT structure. So it's really about scale. How big is your show and how much do you really need to use IT? And that has to do with your knowledge. So I think everyone needs a little bit of knowledge in the littlest part, but the bigger show gets, the more important it is. Rod, do you agree? Do you have thoughts on how much IT knowledge? Have you had to take any networking classes? Just curious, your thoughts. I totally agree with Cam. As always, I always agree with Cam. But I think the network infrastructure is key to the overall system operating efficiently. At this point, based on scale, to Cameron's point, having someone on the team to manage all of that is crucial. It's just like the content manager. We also need a network manager. Fortunately, there are several people that seem to have understood this and now they're taking that on as a staff position. That is now yet another position. And I think that there is a specialty to be achieved in that position alone without necessarily being a media server programmer. Interesting. Sean Borowski, your thoughts on how much IT you have to know to be a successful media server operator or deployer of media servers? Yeah, I think it's very difficult to be in the live events AV industry in 2023 and not have a base understanding of network functionality. I highly recommend anyone that's out there that is currently in the business or is interested in getting in the business to take some of the free online Cisco or NDI courses so that you can understand the fundamentals, especially if you're going to be in the hot seat of a media server operator. But I would say as we get beyond a single box pushing pixels, it becomes critical to have a specialist in larger deployments. When we're talking about 30, 40 boxes pushing pixels, that does absolutely warrant a independent network specialist that's ensuring that these 10 gig, 25 gig backbones are meeting expectations and that throughout rehearsals and then going into the most critical show that there's not some kind of network challenge. So it's key. And then we touched on it just a little bit, but we are absolutely racing towards a future where video over IP in the formats currently adopted of NDI, Dante AV and SMPTE 2110 require fairly sophisticated understanding of how network coordination is applied, monitored and planned for. So NDI and Dante AV tend to make the on-ramp a little bit easier. They are more auto-finding type of tools. But if you've got a poorly configured network, they're not going to find their endpoints, they're not going to find their trajectories. And then SMPTE 2110 is a whole other level of full bandwidth video over IP adopted by the broadcast standards. And that really does, they offer full networking certifications on the level of like a Cisco Tier 3 in order to fully understand how to ensure that signal integrity is pristine. So I think it's very important. And I would say moving forward, it's going to become more and increasingly important. You know, the corporate meetings, we have certain positions that are very much the A1, the LD, the video director, the networking engineer is going to become this new standard position if we fast forward into the future to do what we do. Patrick, your thoughts on the discussion of, you know, we just have been having? Yeah, I mean, knowledge is always the key thing for everybody. Always try to be the smartest person in the room, which means you need to have some basic knowledge of all the pieces you touch, because especially with media servers, if something goes wrong, half of the people in the room will point at this black box. It got better over the past years, but when I started working with media servers, it was always the media server's fault if something wouldn't work. And then I totally go with the point where we need to get to the point where we have media server teams. We're not only talking about the media server operator, we need to talk about the media server operator, we need to talk about the content handling person, we need to talk about a systems engineer who's only taking care of the media server system. So he needs to know network, he needs to know computer stuff. It's not only network, it also goes all the way deep down into machines where you get to the side and your machine is not working. You might need to rebuild Windows from scratch or you need to reinstall an image of your Windows machine because your machine is just not booting up correctly. All of those things go into it. So if you are the media server operator, you only want to take care of operating your stuff. You want to program your timeline, you want to do the creative parts of it. And we see a shift of the media server operator being more in the creative world than he used to be with all of those engines coming up, like Unreal and Notch and those kind of things. So that person needs to be close to the content creation team being on the creative side, but we also need very technical people, on the other hand, who are dealing with the machines to be set up. So in an ideal world, I would say in a big show with a media server team, we have at least three or four people in that team. Well, I want you to know I have a new appreciation for what the role of the media server op is on any project from hearing all you guys. And I've geeked out a little bit with some of the acronyms, but I really can appreciate we're pushing the envelope with pixels, and it just sounds like the future we're going to be continuing to push. And it's just going to continue to get more challenging. And I think that is the future. And I want to thank all of you, Rodd, Cameron, Patrick, and Sean. You all are amazing. I look forward to seeing you in a ballroom, on a show site, in an environment where we're building something or putting out some great show. And I look forward to what the future will hold with media servers. So everybody, great job, and thank you so much for joining today. Thank you for having us. Thank you.