As OEMs search for ways of making EVs more engaging, the in-cabin experience is becoming the new battleground.

As vehicle manufacturers search for ways of making electric vehicles more engaging, the in-cabin experience is becoming the new battleground. In a pair of exclusive interviews, we discuss how sound, real-time displays and haptics are being used to create new areas of value for vehicle manufacturers and how updates from the cloud could introduce new commercial opportunities through the lifetime of every vehicle. We talked with Meridian Audio Chief Executive John Buchanan and Epic Games director of automotive, Heiko Wenczel. 

Source: Meridian Audio

The following is an edited transcript of the conversation.

S&P Global Mobility: Both of your businesses have traditionally focused on consumer markets. What is happening in the automotive sector that is making this the right time to invest in developing your automotive original equipment capabilities?

John Buchanan: That is a good question to open with as you are right that there are some significant new forces driving our industry. Meridian Audio has been working with luxury-vehicle manufacturers since our first contract with McLaren Automotive in 2010. This was quickly followed by what has become a long-term collaboration with Jaguar Land Rover. At the start, our automotive clients simply wanted the very best audio system, focused on impressive reproduction of music. Since then, what defines “the best” has evolved and continues to change at pace. The focus has shifted from speaker and amplifier hardware to new software technologies, which can immerse the occupants in exciting and authentic experiences, engaging all the human senses.

Meridian is a pioneer in this area. We have spent years researching how the human brain perceives sound — a science called psychoacoustics — and we have developed a portfolio of proprietary software technologies based on this understanding, which enables us to recreate those emotional, goosebump experiences and deliver that in the vehicle.

However, the automotive sector is changing rapidly, and consumers widely consider the car to be the “third space” in our lives, alongside work and home. Moreover, there is a new generation of ambitious vehicle manufacturers who are new to luxury, and often new to building cars. Instead of evolving what has gone before, they look around at other sectors and ask what is possible. They prize premium “tech” as highly as fine materials and are launching their brands with super quiet EVs packed with advanced driver aids. High-quality sound is a key part of the answer as it is integral to so many new user-experience features in the vehicle, not just for music but for films, games, social media, karaoke (important in China), and even information and warning tones.

Heiko Wenczel: We are seeing demand for high-quality technology, and I would like to call out two specific trends. Unreal Engine, the real-time 3D technology that is the foundation for many of the world’s most popular games, has been used throughout the automotive industry for many years and is now integrated into pretty much every area of business, from styling, engineering and manufacturing planning to configurators and advertising. In the vehicle, we are seeing strong growth in its use to create amazing graphics, increasingly focused on safely delivering more information and providing new features. If you look at our latest human-machine interface (HMI) clients, it includes Lotus for the Eletre, General Motors, Rivian, Sony Honda Mobility for the Afeela and Volvo for the new EX90 — showing how important the digital in-cockpit experience is for the new generation of EVs. Unreal Engine will … power the infotainment of many conventional vehicles too, such as the next-generation Ford Mustang, but it is part of the same trend, one that Ford neatly sums-up as “experience is the new horsepower.”

The third trend that I predict will become even more significant is the desire of vehicle manufacturers to monetize the ownership phase of vehicle life. One of the advantages of a thoroughly proven games engine powering the HMI is that it comes with totally scalable connectivity with the cloud, hooks for monetization services, superb real-time graphics and an over-the-air (OTA) upgrade path that will allow it to continue to offer state-of-the-art capabilities: Important areas that vehicle manufacturers need to add if they are to enhance customer lifetime value (CLV) quickly and with minimum cost. It will also help cars feel from a previous generation much less quickly once they leave the plant.

Great observations, thanks. I will return to a couple of these soon. First, I would like to go deeper into why is the transition to battery-electric vehicles (BEVs) driving the in-car experience?

HW: There are several factors at play here. The most prominent is that BEVs have, until recently, been purchased largely by early adopters who are particularly interested in new technologies. Right from the very first Model S, Tesla emphasized its ground-breaking HMI. Rivian is justifiably proud of its outstanding HMI and the new features that it introduces. For example, combining the low-speed control that electric drive provides with data from active safety sensors and the real-time graphics capability of Unreal Engine has allowed them to create a new feature that guides the driver across difficult terrain.

JB: EVs still tend to be chosen by people who care about being at the leading edge of technology. That naturally leads to an expectation that the in-cabin experience will be more than just good — it must include new concepts, new areas of customer value and new “surprise and delight” features that have not been part of previous-generation vehicles.

HW: Another factor is the unique requirements of EVs. When every car is quiet and refined, we need something pretty special to make it feel like a luxury drive. Manufacturers of EVs who can offer a differentiated, “tech-led” in-cabin experience are creating additional competitive advantage. If you look at the vehicles that are being launched with a games capability, most are EVs. BMW, for example, highlighted this in their recent i5 launch, presenting in-car gaming as something to do when the vehicle is charging.

Picking up on two other technology trends, what impact will full autonomy have on the in-car experience? Will the widespread availability of 5G introduce new opportunities?

JB: For Meridian Audio, 5G offers the potential to add more high-quality sources, features and therefore experiences, notably those that are enabled by the technology such as 8K video, which will provide a big step in the quality of streaming videos. Online games will also take a big step-up in capability. When passengers in a car are playing games or watching films at the quality they get from their console or their TV at home, they expect a similar immersive experience and that requires not just great sound, but very cleverly managed sound to overcome the acoustic challenges of the in-car environment. With 5G, we will have even greater scope to create full spatial, spectral and personal immersion for every occupant, whether they are enjoying media collectively or individually.

HW: 5G is useful for us, not just because it can handle very large volumes of data, but also because it reduces latency from hundredths of a second to a few milliseconds. That allows the gaming experience to be much more realistic, responsive and immersive. 5G also makes it more practical to stream console-quality games into the car with enough of the processing carried out in the cloud to significantly reduce the in-car computing requirement. Of course, connection continuity while driving still needs to improve as that is currently a significant limitation on where this would work. However, that is not much more than making what we have in our homes also available in our cars.

What is really exciting about 5G is that some of the best applications may not even have been thought of yet. For example, high-data rates and ultralow latency makes it possible to download data in real time. Manufacturers are already using augmented reality in vehicles. Combine 5G with Unreal Engine’s real-time capabilities to harmonize images with the real world and you have very exciting opportunities for new AR (augmented reality) applications or even holography.

Regarding autonomy, that is not really within my field of expertise but I can share some insights we have had from vehicle manufacturer clients. Even professional drivers are still nervous about automated intervention, whether full autonomy or higher levels of automated driver assistance. Research shows that visualizing what the vehicle is seeing and the decisions it is making is one of the most important ways of building driver confidence, leading to a safer and more relaxing journey, and greater trust in higher levels of autonomy. The challenge is how to deliver more and richer layers of information without additional distraction or cognitive stress. The answer is not just about good design. For example, almost-imperceptible timing differences between reality and the display can require the driver’s brain to close the gap, translating one onto the other. This increases the cognitive load and means the driver’s eyes are off the road for longer than they should be. Several of our clients have noted that real-time visualization makes a significant contribution to eliminating this stressful latency.

And of course, there is also the possibility for everyone in an autonomous vehicle to play games or engage in other media and online entertainment, increasing the value of those features.

Returning to the observation about vehicle manufacturers developing ways of generating customer lifetime value, what has caused the step change in potential?

HW: The cloud. It is as simple as that. Streaming data into the vehicle enables a vast range of new services. I cannot tell you how this new approach will look in 10 years because I do not think anyone really knows. We have a clear understanding of the opportunities, so we know the first steps are to provide every new vehicle with a flexible technology platform and a user interface that can be updated over the air as ideas and services develop.  Success will ultimately be driven by the innovation and quality of these services and how they are offered and priced, so I can see a lot of different approaches being tried, a lot of tweaking of strategies and a lot of learning before we have an answer.

JB: I am entirely with Heiko on this, but there is another approach that is already showing how additional CLV opportunities can be created. People have become used to downloading apps onto their phones, and then paying to access the more advanced features. Some car companies are already exploring these opportunities. However, why limit it to software? Why not hardware too? Early hardware pilots have attracted some negative comments, but I think it is just because people are used to paying for hardware options up front. That mindset will change.

At Meridian, we are already working with vehicle manufacturers who are exploring the possibility of software-enabled tiers of audio quality. Fitting every car with all the hardware needed for the highest — or maybe the penultimate — quality level will substantially reduce manufacturing and supply chain complexity while usefully adding to economies of scale. I have seen suggestions that this approach could be cost neutral. So, for no additional cost, we have the needed hardware in place that if the vehicle owner decides that, maybe for school holidays, they want a few months with full rear-seat film capability complete with the type of sound quality they are used to at home. Or maybe to play games, all with superb multichannel surround sound. Clearly there is an opportunity cost with the lost option sale (which may still be available as a simple software switch), and very little is currently known about price elasticity for these services and how it might change as the vehicle moves deeper into the used market, but early indications are that this strategy could create significant new revenue streams. The market in China is setting the pace and provides some impressive benchmarks.

The other key trend is that the quality of sound is increasingly a function of the sophistication and tuning of the signal processing software. That provides a range of OTA upgrade options, from downloading the latest digital signal processing (DSP) profiles to adding new features such as additional controls that give vehicle occupants more ways to tailor the sound to their personal taste. At Meridian, we already have a range of these technologies available and many new ones in development.

HW: This topic was discussed at a conference we organized earlier in 2023. Several speakers at the presentations highlighted that sound have been overlooked as an equal partner in the in-car experience, with a speaker from BMW stating that they now treat sound and visuals as “coequal disciplines.”

That is an interesting comment from BMW. What feedback are you getting on the possible role development of sound?

HW: The key word here is “immersive.” As a games creator, we have developed a deep understanding of how to combine the senses to maximize the emotional experience, so it is great that BMW and others now recognize this. I remember decades ago when an automotive audio supplier demonstrated their latest premium system with the dinosaur’s approaching footsteps from “Jurassic Park.” Just the sound alone made the guests shiver. People were looking over their shoulders. Press releases talk about “games-quality graphics” in cars, but really, “games quality” is an immersive experience that is brought to life by a combination of impressive graphics and equally impressive sound. Probably the most important feature today is fluent interaction, a key difference you will feel when using game engines in cars today.

Is it fair to say that in recent years, in-car audio has evolved rather than benefiting from any step changes in technology?

JB: On the hardware side, that is a fair observation. It has progressed, but there have been few step changes. Software, however, has evolved rather more quickly, largely driven by the development of new digital processing systems for the film industry and by companies such as ours investing in research in areas such as psychoacoustics, which helps us develop software that creates a greater perception of realism, clarity and excitement. As a technology-driven business, Meridian invests about 15% of revenue in research and development, with a high proportion allocated to software innovation.

Some of these software technologies are becoming well-established in high-end car audio, for example — systems that take the sense of realism even further by adding the correct height to the sound stage, allowing every vehicle occupant to identify the precise position of the sound source in three dimensions. For music, 3D sound renders all instruments, performers and venues in true scale and location. That is possibly a subtlety valued by audiophiles, but for games and movies, the difference is incredible, making the whole immersive experience more realistic.

Breaking new ground is software such as our Intelli-Q technology, which optimizes audio playback based on data from the vehicle such as speed, window state, occupancy and audio source. This ensures that all occupants in the vehicle enjoy an optimal audio experience at all times. Another example is Meridian’s Horizon technology, which expands the sweet spot and creates a more immersive listening experience in systems with surround or height speakers.

Crossing the boundaries between hardware and software are new developments in haptic audio, where we are introducing a second-generation technology that greatly enhances the in-car media experience while eliminating the compromises associated with first-generation systems. Meridian’s new Vibrohaptic seat technology, again developed with our psychoacoustic experts, is purely to enhance how our bodies receive and interpret the sound and has already delivered impressive results in customer clinics.

That is a fascinating point about 3D audio and games. Looking forward, to what extent will in-car gaming depend on an immersive audio system?

HW: That depends on what level of gaming the vehicle manufacturer offers. Millions of people enjoy games on their phones, but you are right that to take full advantage of the excellent HMIs now being fitted to cars, only immersive audio does the job. It is a trend that our developers have highlighted, leading to the introduction of MetaSounds with the latest version of UE. This allows designers with no programming expertise to generate audio signals synthetically at runtime, shaping new sounds that bring texture, reality and excitement. Without a capable audio system, sounds can lose the detail and vibrancy that create the immersive experience. It is true that the car is a very challenging audio environment, but it is also one of the only perfectly tracked environments to deliver immersive sound to specific areas.

However, it is wrong to assume that high-quality sound is just for games and media. I mentioned before that some of our customers are doing valuable research on how to safely communicate information and warnings. One of the results is an understanding of the importance of real time in sound too. MetaSounds has a timing resolution of just 1/48,000 seconds, or 0.02 milliseconds. I am not an expert in driver alerts, but I am told that in emergencies, these fractions of a second are important.

JB: We are noticing that our systems are increasingly at the heart of an integrated ecosystem of sound and visuals that embraces information, entertainment and services. When a vehicle buyer specifies the higher-level system, all of these are enhanced. Personally, I think this integration into a cabin ecosystem will become a significant design trend.

Just relating this back to the CLV discussion, I can see opportunities to selectively activate the higher quality audio. If the hardware for the high-end audio is installed but not activated for traditional audio sources, it could still be activated for games as part of the games download package, or for active safety as part of that package. Other options could include noise-canceling software that transforms headrest speakers into personal sound zones and our Horizon software technology, already mentioned.

That is a lot of new ideas to deliver. Will vehicle manufacturers have to change their design processes?

JB: Creating this integrated ecosystem inevitably requires the breaking down of silos. My team of acoustic engineers can add more value if they are involved at a very early stage, bringing the vehicle manufacturer’s specialists together to help define and optimize the specification and packaging. However, breaking down silos does not only mean within vehicle design; it means in product planning too. We are holding monthly workshops for one of our European luxury-vehicle customers not just to inform them about what is possible in our field, but also to facilitate discussions between their teams who would not normally plan together at such an early stage. It has proven to be a great forum for creativity that has already helped catalyze new features that they are using to add value and progress their brand positioning.

HW: One of the reasons vehicle manufacturers are developing with Unreal Engine is because so much can be done by the creative or engineering teams without handing their ideas over to the software specialists for implementation. BMW and Volvo have spoken about this, describing how the intuitive user interface allows designers to cut out the “over the wall” iterations that rely on specialist coders, allowing designers to quickly develop and test their ideas in UE alongside colleagues from other specializations. For contained tasks, there is also a global ecosystem of very highly skilled independent developers, many of whom are fizzing with brilliant ideas.

The topic of silos is one that always vexes me. My job provides a helicopter view of what vehicle manufacturers are achieving with Unreal Engine so I can see how they are using it to improve their processes from the first styling sketches through engineering to sales and marketing, but I am also seeing where resources are being wasted in duplication because there is no overarching strategy for a company-wide digital twin of the vehicle; what we call a single source of truth. It really is astonishing how many different departments still develop their own, unique digital models.

A formalized digital twin also brings benefits outside the vehicle development processes, for example, allowing very fast, much lower cost and more highly localized advertising to be generated before the vehicle is ready for creative, simply because a finished, photorealistic model is already there, ready to use. The same is true across engineering, for example, by allowing higher levels of simultaneous engineering with less duplication and fewer late-stage changes, as well as in the vehicle where it gives HMI designers a useful head start. A few companies have really picked up on this in isolated areas, but I am not aware of any vehicle manufacturer that is pulling the big lever to take full advantage of the time and cost savings that a top-down digital vehicle ecosystem can deliver across the organization.

Do these new approaches bring any speed to market benefits for HMI developers?

HW: The depth and complexity of HMI is growing, but we cannot afford to compromise timescales or creativity, especially with the growing requirement to develop new areas of value through the vehicle’s life. New features such as predictive maintenance need entirely new ways of explaining complex concepts for which real-time graphics are ideal, but the complexity must be tightly managed. We have discussed the importance of reducing iterative processes and how this can be achieved using Unreal Engine’s intuitive interface. We have also looked at how digital twins — the single source of truth — can significantly contract timescales across all vehicle manufacturer activities, creating a model-driven ecosystem that allows many activities to begin earlier and to work more efficiently. Our clients already have HMIs in production that use the engineering model of the vehicle to improve the realism of the representation and to slash the time required to create and embed the model in the HMI.

What our industry cannot allow is any compromise in quality, and that is partly where the ecosystem of established developers comes in. We have recently seen several new vehicle launches delayed by software challenges, and most have been from vehicle manufacturers who are trying to wrap their arms around the vehicle software. The digital experience might not be seen as production-critical but it certainly is brand-critical. Yes, we can download fixes, but it is far better to introduce efficient processes that take full advantage of the latest digital tools, new collaborative ways of working and external expert resources, joining up workstreams across the traditional silos to achieve higher levels of integration, with the added value that can bring, alongside reducing timescales. From a customer perspective, a car company cannot afford for their in-car experience to be less refined than other digital-user touchpoints such as the web or phone apps.

Can you reveal any of the vehicle manufacturers you are working with in this new way?

JB: I cannot give away any customer product plans, but we are very proud of our long-term relationship with Jaguar Land Rover. This has helped them introduce innovations such as noise-canceling speakers in the headrests and dedicated sound for the tailgate seating in the new Range Rover. Collaboration as a business model rather than a traditional supplier model has also helped their interior designers be more creative because they understand the latest technology opportunities and how to work with them, rather than in an iterative process with engineering that inevitably increases costs and timescales.

I am also impressed by HiPhi (Human Horizons), an ambitious Chinese luxury-vehicle manufacturer that is brushing aside many of the preconceptions embedded in our industry. We have just completed our third project with them, taking a big step forward in their goal to provide a hi-tech, in-car luxury experience. We have also just announced that our relationship with Kia now includes the all-new EV9, which Kia says delivers leading-edge technology, inside and out, to provide the driving experience of tomorrow. It is interesting that they link tech inside with driving experience, validating what we were saying earlier.

I cannot say which luxury vehicle manufacturer will be first to introduce our Vibrohaptic audio technology for automotive seating, but it is heading for production in 2025 with a European customer, followed quickly by a customer from Asia.

HW: Rivian is a great example of one of our clients that is using Unreal Engine to create new features and new value, but in terms of accelerating development, I would like to compliment BMW on embedding Unreal Engine in many of their processes and Mercedes for pioneering what they call “an online game for engineers” to accelerate remote working. It is also interesting that the first-tier technology suppliers are starting to include Unreal Engine within their offerings so they can supply a complete hardware and software HMI system. Ecarx is very progressive in this area. I also like the way that Volvo is thinking about safety alongside continuously evolving customer value. Unreal powers the HMI on their new EX90 electric-luxury sport utility vehicle (SUV).

Thank you, Heiko Wenczel of Epic Games and John Buchanan of Meridian Audio. It has been a fascinating discussion with each of you, delivering interesting insights for our AutoTechInsight readers.

S&P Global Mobility comment:

Anna Buettner, principal analyst and manager in the Connected Car and Vehicle Experience team at S&P Global Mobility, says: “Global demand for automotive speakers is expected to grow from [approximately] 77.2 million units in 2022 to nearly 90.3 million units by 2029. Penetration rates for branded acoustics system will continue to rise steadily, but the volume vehicle brands still present a large undersaturated market for branded speaker systems. Luxury or volume, suppliers are looking for ways to take advantage of this opportunity by using innovations in software, hardware, materials and audio source content as key differentials to distinguish their in-vehicle HMI experiences.”

The latest S&P Global Mobility report “Dashboard display market and technology trends” provides an understanding of the primary displays in the vehicle dashboard — center-stack display, instrument cluster display and a more recent phenomenon, multimodule displays. It also delves on the demand outlook for such display systems, the main suppliers and users, and the main market drivers at a global and regional level. The report combines historical information from various research teams across S&P Global Mobility and offers a holistic view of the primary in-vehicle displays market.

The latest S&P Global Mobility report “Acoustics and sound technologies — The third dimension of UX” provides a detailed understanding of both branded and non-branded audio technology for cars, an in-depth demand outlook, main players, and main market drivers at regional and global levels. A technology analysis on active noise cancellation, 3D sound, separate sound zones and more is also included.