The multi-dimensional experience of the smart cockpit is upgraded, and Zebra and Unity join forces for underlying innovation

With the continuous improvement of car users' demands for intelligence, personalization and higher comfort, the main entrance of human-vehicle interaction, the intelligent cockpit, is ushering in a stage of rapid development.

According to IHSforecast, the global smart cockpit market space will exceed 40 billion US dollars in 2021, and by 2030, the market size will reach 68.1 billion US dollars.By then, the domestic smart cockpit market will also exceed 160 billion yuan, making it the world's largest smart cockpit market.

Such a broad market has attracted the attention and participation of a large number of enterprises in the past few years.Everyone stands on the same starting line, but because of their different genes, the direction of exertion is not the same.Most of the traditional smart cockpit providers choose to start from the upper-layer application and expand the functions according to the actual needs of users.However, some technology companies have also proposed that the cockpit, as an extension of the social relationship between drivers and passengers, can learn from the relatively mature technical capabilities of the mobile terminal, and realize the upgrade of the intelligent cockpit experience through technological innovation at the operating system level.

Smart car OS technology and service provider Banma Zhixing is a proponent of this line.Recently, Banma Zhixing has reached a cooperation with Unity, which has rich experience in mobile 3D content development. By integrating the Unity engine, especially the latter's 3D interactive content creation capabilities, the AliOS capabilities can be further advanced and better support the multi-dimensional in the smart cockpit. experience upgrade.

From 2D to 3D, the smart cockpit enters a new period of change

The rapid development of automobile intelligence and networking is driving the rapid evolution of the cockpit to the mobile space, and it is also reshaping the user's concept of automobile consumption.Nowadays, people's demand for cockpit is no longer limited to safety, but more and more emphasis on entertainment and comfort, and a large number of functions and contents are integrated in the cockpit.

As demand exploded, so did problems.How to realize the fast information exchange between users and various functional applications while improving the user's sensory experience has become a new problem plaguing the industry.

Especially under the premise that the level of intelligent driving of the whole vehicle is generally not high, if there is a lack of reasonable HMI design, it will cause distracted driving to a certain extent.

To this end, current car companies and technology providers are redefining the HMI interface of the cockpit, such as redesigning functional safety icons, optimizing the layout of interactive information, etc., and even introducing the 3D concept in the field of video games to 3D upgrade the HMI interface.OEMs represented by Mercedes-Benz and BMW have already applied 3D technology in the field of instrumentation. Jidu also announced on ROBODAY that it will create a 3D human-machine co-driving map in its smart cockpit.

But it's not easy.After a hundred years of development in the automotive industry, although some mature development tools and models have been precipitated, these tools used to be more oriented towards 2D interactive content development. Now, when they are used for 3D HMI design, they will face a series of "acclimatization".In particular, traditional car companies and Tier 1s tend to be large in scale and have many departments. In the HMI design process, there are pain points such as workflow breakage and platform fragmentation, which will eventually lead to the final performance in terms of graphic expression and the efficiency of the design concept. There is a certain gap.According to relevant analysis data, based on the current HMI workflow in the automotive industry, it is possible that 50% to 60% of the design concept is lost.

Seeing this pain point in the industry, Banma Zhixing and Unity reached a cooperation, officially connecting to Unity's integrated development environment on the AliOS system layer, becoming the first smart car operating system equipped with Unity engine in China, which means that Unity developers The developed content can be directly exported to AliOS. Together, the two can better solve the problem of "acclimatization" in automotive 3D HMI design, and help the automotive industry explore more possibilities in the field of smart cockpits.

Zebra.com and Unity are leaders in their respective fields.Zebra focuses on the research and development of its own operating system, and the scale of AliOS has exceeded 2 million units.This year, many models of Zebra's cooperation with Zhiji, Roewe, MG, SAIC MAXUS, FAW-Volkswagen, SAIC Volkswagen, etc. will also be launched one after another. It is expected that the number of models equipped with AliOS will also increase significantly.Unity has many years of research and development experience in mobile games and other fields, and has rich experience and resources in cross-platform compatibility, performance optimization, application development ecology, etc., and has been extensively verified by the market.According to relevant statistics, as of 2021, among the top 1,000 mobile games, the number of games made with Unity will reach 72%.

At present, Unity is expanding these technologies and experience to the automotive field, and has launched the Unity HMI Editor, a UI extension package specifically for the HMI development process, and has developed the Unity Automotive HMI Template, which seamlessly connects with the Unity editor. The integration with the operating system can better solve the problem of broken development process.

However, it should be pointed out that compared with the mobile terminal, the development of 3D content on the vehicle side has quite a few limitations and challenges, and the copying of experience is not as easy as imagined.Especially in terms of how to solve the problems of vehicle safety, reliability, and stability, game engine companies represented by Unity have to consider issues.

For example, in the visualization of ADAS-related functions, because it is directly related to driving safety, the quality requirements and real-time requirements for content output will be higher than general application scenarios.After all, in the game, if there is a freeze, it is at most GameOver, but if the car freezes when navigating, it is easy to go the wrong way, and it may even cause a traffic accident.

From this point of view, through the cooperation with Zebra Zhixing, it is expected to help Unity quickly fill this short board.In addition, the advantages of game engines in rapidly generating large amounts of synthetic data to aid in AI reinforcement learning can also contribute from another perspective.Perception Package is based on the Computer Vision function of the Unity engine, which can easily and quickly generate large-scale synthetic data datasets, thereby helping to train and verify related algorithms, reducing costs and increasing efficiency for the entire development process of safe and autonomous driving. .

On the other hand, the deep integration of the AliOS smart cockpit operating system and the Unity engine can create capabilities and immersive experiences that mobile devices do not have.AliOS can realize unified management and scheduling of the smart cockpit domain, and its multi-screen fusion framework can realize the logical virtualization of different physical screens, bringing multi-screen immersive display and cross-screen streaming experience.The multi-zone multimedia framework combined with the 3D capabilities of the Unity engine enables an immersive sound field experience.In addition, the operating system also provides SOA services such as vehicle sensor signals, body control capabilities, high-precision positioning, and interior and exterior perception. These capabilities are deeply integrated with the Unity engine through the application framework of the operating system, enabling 3D digital twin applications and assisted driving. applications and immersive cockpit games, etc.

At present, the technology of cooperation between the two parties has been initially applied to SAIC's high-end intelligent pure electric car Zhiji L7. The teamlab function pioneered by IMOS can present the beauty of flowers in different seasons in full screen and across screens, and supports touch interaction, flowers bloom at the fingertips, creating a unique Immerse yourself in a sensory experience.

The pattern of automobile intelligence opens up, and new technologies create new experiences

In the new round of automobile industry reform,electrificationis the first half, and intelligence is the second half. At present, it is in the golden development period of the rapid popularization and application of intelligent driving.

According to the "China Autonomous Vehicle Market Data Tracking Report" previously released by IDC, from 2021Q1 to 2022Q1, the penetration rates of domestic L2 autonomous vehicles in the passenger vehicle market are 7.5%, 13%, 16%, 21.5%, and 23.2%, respectively. %, showing a rapid upward trend.As far as the technical level is concerned, the industry is currently in the process of evolving from L2 to higher-level L3 or even L4.

However, considering that the supporting regulations and technologies for the implementation of fully automatic driving are not very mature, it is expected that human-machine co-driving will be the norm in the transition stage to high-level autonomous driving.This means that the human-machine co-driving experience will become the main focus of track players in recent years, and how to meet the user experience in this driving mode is a problem that all automotive industry practitioners need to face together.

In this regard, Banma Zhixing is also actively developing related technical reserves, including exploring multi-mode interaction technology that is more suitable for human-machine co-driving scenarios, and expanding AI capabilities such as voice interaction, gesture recognition, and gaze tracking on the basis of the Unity engine capabilities. .

The data shows that during the driving process, 95% of the driver's energy is focused on the driving behavior, and only about 5% of the energy can be extracted to control the in-vehicle system.In fact, multi-channel fusion can essentially solve this problem and reduce the cognitive burden of drivers.Specifically, it is to present the human-computer interaction in a 3D way, and with the sound, it will make the human-machine co-driving more tacit and safer.

Imagine that if a pedestrian suddenly rushes out during the driving of the vehicle, the person will inevitably take a sudden brake to avoid it, which will cause the front of the vehicle to nod and the driver and passenger in the vehicle to tip forward, and in severe cases, personal injury may occur.Under the same conditions, if the car has collected and judged the perception data, the deceleration and braking process will be presented in 3D in advance, supplemented by voice prompts, which may give the driver a comfortable and reassuring experience.

In fact, "the role of HMI is to allow two "drivers" to communicate effectively, resolve differences and conflicts, and help the industrialization of ADAS." said Zhu Xichan, a professor at the School of Automotive Engineering of Tongji University.

To further innovate the 3D interactive experience of the cockpit is also the focus of Zebra and Unity.

Based on car usage scenarios, 3D technology will also be applied to navigation maps in the future.The highly detailed 3D map includes not only the exact location of lane markings, road signs, crosswalks and other infrastructure, but also dynamic information such as lane level data, weather signs, curbs and more.

Under the fusion of static map navigation and dynamic perception data, the perception of the car can be run on the three-dimensional map, and the yaw problem caused by road cover and visual deviation can be easily solved.

By accessing the dynamic data of parking lots such as roadside and shopping malls, it is also possible to actively push the usage of nearby parking lots after the navigation reaches the destination, and freely select a free parking space, initiate navigation with one key, and navigate directly to the parking space to realize Quick stop.

In addition, considering that in the new generation of smart cockpits, the easiest way for users to upgrade the consumption experience is the gaming experience, and Zebra and Unity are also making arrangements in this field.

The Unity engine will bring a wealth of exquisite games, and the SOA capability based on AliOS’s cross-domain connectivity can further enhance Unity’s capabilities. The air conditioners, fragrances, seats, ambient lights, and audio systems in the car can all be linked together. It empowers the in-vehicle gaming experience and creates an immersive and scene-based experience with multi-dimensional senses, which will surpass the gaming experience on mobile phones and PCs.

In fact, game boarding is already an unstoppable trend in the automotive field."Especially when the car is self-driving, entertainment will be critical." Musk has previously said he thinks Tesla can make that happen later this year.To this end, Tesla has been releasing more video games in Tesla Arcade, and it is planning to offer paid packages for different video games in its vehicles. the same business area.

It is worth mentioning that Tesla is using Unity as a development tool to formulate an in-vehicle game strategy around Tesla Arcade to create richer in-vehicle entertainment content.Digging deeper, it can be found that Unity has also reached cooperation with high-end car brands such as BMW and Volvo in related technical fields.This cooperation with Banma Zhixing undoubtedly further reflects Unity's determination to expand into the automotive field.

Facts have also proved that in the new round of automotive industry reform, whether it is the creation of richer smart cockpit applications and experiences, or the development of higher-level smart driving functions, strong support from underlying technologies such as AliOS and Unity engine is required.