Article Mar 18, 2024 4 min read

Why does server-side vs client-side data processing matter for AI?

Pexip

Mar 18, 2024 4 min read

Pexip

First, how is AI used in video conferencing today?

Thanks to AI, you’ll find user experience-enhancing capabilities in your video meetings today, and a lot more are on the way. Applied correctly, AI enables background noise reduction, background replacement, and intelligent framing of people’s faces, which enhances the quality of meetings.

Lately, we’ve seen more services add intelligent and generative AI to video meetings to record sentiment, summarize action items, and much more. Microsoft offers various Teams meeting summary options. These include its Intelligent Meeting Recap, which comes with Teams Premium licenses, and Copilot meeting notes. Both services offer excellent meeting summaries and various other benefits.

Server-side data processing benefits heavy workload tasks like large language models (LLM) used in AI translation

When AI technologies are applied server-side instead of client-side, everyone connected to the meeting benefits from the same technology and performance. This is because server-side data processing happens centrally in the network and is distributed equally to all endpoints/clients. Regardless of the client people then use to connect to a meeting, such as a video conferencing room device, an iPad, or even their laptop computer, everyone will have the same experience.

More importantly, tasks that are compute-heavy, like large language models (LLM), are ideally suited for server-side processing, as the workloads are too heavy to be performed client-side. Server-side applied AI results in much higher performance on the client, benefiting users with improved productivity and quality and the ability to benefit from workload-heavy AI.

Server side processing

Pexip takes a server-side AI approach

This includes audio and video processing for any meeting and call, meeting layouts and experiences, and, of course, the application of AI. Server-side processing is also what enables Pexip to be the industry leader in video meeting interoperability across technology platforms.

Pexip AI Media Server

Pexip is uniquely positioned in the Secure and Custom market/solutions area, catering to organizations that demand self-hosted or private cloud deployments to enable secure by-design video meeting solutions. Today, Pexip serves many of the world's largest government organizations with this capability. Read why private and secure AI is so crucial for video conferencing.

To meet the increased demand of security-conscious organizations, Pexip collaborates with Nvidia to enable server-side language translation and speech-to-text services specifically demanded by organizations for privacy and control. This collaboration solves multiple challenges, for instance:

Everyone gets the same translation performance. Instead of relying on the local device to perform AI translation tasks with often unpredictable results, Pexip takes care of all the translation loads. That means one does not have to trust the device's built-in capabilities.
Updates to services and performance immediately benefit everyone. Instead of users needing to update their devices and systems, everything is controlled and applied using a central resource.
It works on any device. In multi-party and hybrid meetings, knowing what kind of device everyone uses is impossible. Yet organizations need to ensure all users benefit from the same experience regardless. Due to its server-side approach, only Pexip can solve a unified cross-multiplatform experience.

Centrally controlled and governed

A key benefit of server-side AI processing is that it eliminates the need for organizations to purchase new devices to reap AI benefits. The Pexip approach ensures that everyone can enjoy the advancements of AI, irrespective of their device's age or brand.

What about client-side AI?

Specific tasks are managed perfectly well directly on the device or client side. Tasks such as noise reduction, face detection, framing, and other enhancements to video meeting quality are well suited for execution on the device. These are not compute-heavy and do not rely on large data sets. This works well for mobile devices but only sometimes translates to older meeting room devices, such as the countless systems found in enterprises and public sector organizations worldwide.

What’s next for server-side AI processing?

Translation and speech-to-text are merely the beginning of compute-heavy AI applied server-side. More compute-heavy workloads and specialized tasks are on their way as well. Both quality and productivity enhancement capabilities will benefit greatly from being applied server-side, and with Pexip’s unique architecture, we are building a foundational platform for working with and managing large data sets and heavy workloads that benefit users regardless of systems, devices, or location.

Topics: