Ably: Serious, serverless realtime infrastructure - Medium

Live chat: Should you build or buy?

Ably Team — Wed, 25 Feb 2026 11:21:01 GMT

Deciding between building or buying a live chat solution? This post helps you evaluate whether to build in-house, adopt a Chat-as-a-Service (CaaS), or use a realtime platform with a chat SDK like Ably Chat. It covers pros and cons of each approach across speed, control, cost, and scalability.

Live chat has evolved from a nice-to-have feature to an essential component for customer service, user engagement, and community interaction across gaming, SaaS, social networks, and live-streaming apps. As realtime communication becomes increasingly integral, it touches nearly every layer of your stack, from low-latency message delivery and message ordering to presence indicators, moderation, and scaling at global levels.

In fact, the importance of live chat cannot be overstated, with studies showing:

41% of customers prefer live chat over phone support.
52% of customers would buy again from a company that offers live chat.
79% of businesses report increased customer loyalty and sales after implementing live chat.

But with these opportunities comes complexity. Building chat isn’t “just another feature” — it’s a strategic decision that affects your app’s infrastructure, user experience, and long-term scalability. That’s why choosing the right architecture upfront is critical.

The Three Ways to Implement Chat

When choosing the right approach to chat, there are three common routes, each with its advantages and trade-offs:

Build your own chat backend from scratch
Use a chat SDK or Chat-as-a-Service (CaaS) provider
Leverage a realtime platform with a purpose-built chat SDK (like Ably Chat)

Want a deeper look at the technical architecture behind realtime chat? Read our guide on designing chat architecture for reliable message ordering.

Building your own chat backend from scratch

Building a live chat system from scratch gives you full control over every layer of the architecture. If your team has strong expertise in realtime and distributed systems, this might seem appealing. You get to tailor protocols, storage, and even the user experience (UX) to your exact needs.

However, this approach comes with significant challenges. The reality is that building and maintaining a custom solution is a complex and resource-intensive task. First their are the technology choices. Do you rely on a messaging protocol, such as WebSockets, WebRTC, and XMPP or, instead, pull in a realtime library such as Socket.IO? Beyond the initial build, you’ll need to account for scaling globally, low latency delivery, message ordering, and uptime. Also, the deeper you go into realtime chat infrastructure, the more specialized the requirements become.

The pros of building a live chat system from scratch

The upsides of building your live chat system in-house mostly center on control and flexibility:

Already deep in the weeds of infrastructure? Our scalable chat app architecture guide covers what to plan for from day one.

The cons of building a live chat system from scratch

While the advantages of building a live chat system from scratch are evident, there are some challenges to consider:

Best for: Teams with dedicated infra/ops expertise, high custom requirements, and deep pockets.

Use a Chat-as-a-Service (CaaS)

If speed is your priority, a chat SDK or CaaS (Chat-as-a-Service) platform can help you get up and running quickly. These platforms — like Stream, Sendbird, or Twilio — offer prebuilt components like messaging, presence, and reactions. They handle all the infrastructure, scaling, and operations.

However, the trade-off comes with a lack of flexibility. You’ll be limited by the platform’s constraints on custom chat logic, UI design, and integration options. This could be fine for basic needs but may fall short when you need more control over user experience or the ability to extend beyond chat.

The pros of using a CaaS

The advantages of using a CaaS are pretty straightforward:

Low maintenance and roadmap investment: The CaaS provider takes care of keeping the service online as well as delivering new functionality.
Low maintenance and roadmap investment: The CaaS provider takes care of keeping the service online as well as delivering new functionality.

The cons of using a CaaS

On the other hand, chat-as-a-service can be quite limiting.

Best for: Teams needing chat-only functionality, with limited customization needs.

Use a Realtime Platform with a chat SDK (e.g., Ably Chat)

There is a third option that strikes a balance between control and speed. By leveraging a realtime platform with a purpose-built chat SDK, you can tap into a provider’s realtime expertise and infrastructure, while still customizing the chat solution to fit your exact requirements.

A realtime platform like Ably, with its Chat SDK, offers the best of both worlds: it’s quick to implement while giving you ample flexibility to tailor the user experience.

The pros of using a realtime platform

This approach offers a balance of quick implementation and customization, making it ideal for teams that need both flexibility and efficiency.

Quick to implement with customization options: A realtime platform with a chat SDK lets you get your chat solution up and running quickly, while still allowing for extensive customization to meet your specific needs. You don’t have to sacrifice speed for flexibility.
Global infrastructure for low latency and scalability: Backed by a global pub/sub infrastructure, this platform ensures low-latency messaging, seamless scalability, and reliable performance even as your app grows, allowing you to deliver a smooth real-time experience at any scale.
Comprehensive chat features, including presence, reactions, and message history: The platform comes with essential chat features such as presence management, message history, typing indicators, and reactions, all pre-built for ease of use and enhanced user interaction.
Supports a wide range of chat types: Whether you need 1:1 messaging, group chats, or even livestream-style communication, the platform’s flexible architecture supports various chat configurations, making it easy to tailor the experience for different use cases.
Easy integration and built-in moderation tools: The platform integrates seamlessly with your existing tech stack and provides built-in tools for moderation and UI customizations, helping you enforce policies and design a chat interface that aligns with your brand and compliance requirements.
Predictable pricing models to grow with your app: With usage-based or MAU pricing, the platform offers predictable costs that scale with your app, ensuring you can plan for growth without surprises.

Cons:

While this solution offers flexibility, there are a few considerations to keep in mind.

Requires front-end UI development: Unlike some solutions with pre-built UIs, this platform requires you to design and develop the front-end chat interface, offering flexibility but requiring more time and resources to build and integrate.
More setup compared to CaaS, but less infrastructure burden than self-building: Setting up the platform requires more effort than using a CaaS solution, but it’s far less complex than building everything from scratch. The platform takes care of the infrastructure, so you can focus on your app’s features instead of managing backend systems.

Best for: Teams looking for reliable realtime infrastructure with chat primitives, but without the burden of maintaining backend systems.

Requirements for a successful live chat

Regardless of whether you build your own live chat app in house or you rely on a CaaS or realtime platform, there are certain foundational elements that you’ll need to deliver in order to create a successful experience.

WhatsApp and Facebook Messenger, amongst others, have set the bar for what end users expect from live chat. When building your live chat you’ll need to meet those expectations by delivering on these foundational features.

Low latency: Realtime chat sets a baseline of interactivity that can only be realized when there is minimal delay between a message being sent and then received by other participants. That depends on using low latency protocols, such as WebSockets, and having a global network of servers that are geographically close to end users.
Guaranteed delivery: Each message must get to its destination in the correct order and on time. Even at smaller scales, that requires significant engineering effort to deliver reliably and without the possibility of confusion caused by missed, duplicated, or unexpected messages.
Scalability: Increased uptake of your live chat should be reason to celebrate, rather than causing concern over how to serve the expanded user base. A Pub/Sub messaging architecture separates communication and business logic, providing the basis you need to scale on demand.
Elasticity: High performing, resource efficient, chat applications maintain a consistent user experience regardless of usage patterns and traffic spikes. They automatically react to changes in demand, elastically scaling to handle unpredictable and fluctuating numbers of concurrent users.
Key features: To meet user expectations for live chat, you’ll need basic features like authentication, allowing users to log in and assume roles (e.g., seller, buyer). Users will also expect easy access to contacts and message history for ongoing conversations. Beyond that, enabling message editing, deletion, and user tagging is essential. Integrating native notifications ensures participants stay informed, even when not actively engaged in the chat. For more on the features needed for live chat, see our ultimate list (complete with examples).
Integrations: Your live chat solution should seamlessly integrate with the other tools in your tech-stack, otherwise you risk substantial engineering requirements in order to integrate new services into your infrastructure.
Data governance: How data is gathered, stored, processed, and disposed of. Data policies also set out who can access what kinds of data and what kinds of data are under governance.
Chat moderation: The process of monitoring and controlling user-generated messages and content. To meet trust and safety commitments around chat, integrated moderation is essential to ensure all user-generated content is in compliance with your terms of use.
Frontend UI: Ready-made UI design theme components, such as colors and fonts, which can be customized to suit your brand. This enables an easy and fast integration of standard chat features into new or existing client apps.

Chat Architecture Options Compared

To help you make an informed decision, here’s a visual comparison of the trade-offs across the three approaches — self-build, chat SDK/CaaS, and realtime platforms with chat SDKs — covering factors like speed to market, feature flexibility, scalability, and cost.

Making the Decision: Which chat option best fits your team?

Based on the comparison in the tables, here’s how to decide which option is best suited to your needs:

Self-Build: Best for teams with strong infrastructure expertise that require complete control over every aspect of their chat solution. This option is ideal for highly customized use cases, but it requires substantial resources, including dedicated engineering teams and significant ongoing maintenance.
Chat SDK / CaaS: Ideal for teams with limited engineering resources that need to deploy a chat solution quickly. While this option offers great speed to market, you might sacrifice flexibility, and scalability can become an issue as your app grows. It’s best for companies looking for chat-only functionality without the need for advanced features like scalability or custom integrations.
Realtime Platform with Chat SDK (e.g., Ably Chat): A great middle ground for teams that want fast setup, customization, and scalability without the burden of managing infrastructure. Ably’s Chat SDK allows you to quickly implement real-time messaging with rich features while having the flexibility to integrate with other real-time use cases. It’s ideal for teams that want a flexible, scalable, and reliable solution without the need for in-house infrastructure management.

FAQs

What is the best way to build a chat app? It depends on your goals. Self-build offers control, Chat platforms offer speed, and realtime platforms like Ably and our Chat product offer both flexibility and reliability.
When should I use a chat SDK? Use a Chat SDK when speed to market is more important than full customization.
What’s the difference between a chat platform and a realtime platform? A chat platform offers prebuilt messaging features and is limited to chat use cases. A realtime platform like Ably provides the underlying infrastructure and chat primitives, giving more control and scalability — applicable to a wide range of realtime use cases.
Is building a chat backend worth it? Only if you have the time, budget, and need for total control. Building a decent chat architecture from scratch is relatively easy, but building a rock-solid one is an ongoing investment. For most teams, it’s faster and cheaper to build on top of a platform that handles the hard parts of realtime delivery.

Get started with live chat from Ably

Ably’s Chat SDK provides a flexible, feature-rich system for building 1:1, group chat, and even livestream-style interactions. It comes with built-in tools like presence management, message history, reactions, and typing indicators, all supported by Ably’s global pub/sub infrastructure. You also benefit from integrated moderation workflows (with tools like Hive), automatic scaling, and predictable pricing (either MAU or usage-based pricing), ensuring that costs remain manageable as your app grows. Start with a free developer account and begin building today.

Originally published at https://ably.com.

Live chat: Should you build or buy? was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

Scalable chat app architecture: How to get it right the first time

Ably Team — Tue, 24 Feb 2026 11:21:01 GMT

Designing a scalable chat app architecture? Learn why low latency, fault tolerance, message integrity, and global reach are non-negotiable. We explain the core components (e.g. event manager, presence, transport), the trade-offs of protocols, and how to choose between building your own infrastructure or leveraging a realtime platform like Ably Chat.

Live chat is everywhere — from customer support to in-game banter to collaborative workspaces.

Since launching Ably Chat — our fully managed backend for large-scale messaging — we’ve had a front-row seat to the architectural mistakes teams most regret when traffic jumps from thousands to millions of messages. One-off errors, delayed messages, and infrastructure meltdowns are far too common, especially when chat moves from a nice-to-have feature to mission-critical functionality. And, when there’s an outage, it becomes headline news.

It’s not just hypothetical. Slack alone processes more than 8.5 billion messages each week, and every single one needs to be routed correctly, arrive in the right order, and show up instantly.

Getting this right requires more than just a realtime messaging API or a couple of WebSocket endpoints. It takes an architecture designed from the ground up to handle scale, preserve message integrity, and deliver a seamless user experience under pressure.

In this post, we’ll walk through the key architectural components that enable scalable, feature-rich chat — from transport protocols and state management to ordering guarantees and elasticity.

What is chat app architecture and why is it important?

To people using chat, the entire experience seems to be contained in the app itself. But the chat client that lives on the user’s laptop or mobile device is just the tip of a much larger network of components. For example, the presence service monitors which users are currently connected and how. Another service will look after logging the chat history to a database, while yet another handles searching that chat history.

What job each component does and how those components work together is the chat app architecture. The design of that architecture affects just about everything that the user experiences. Perhaps most importantly, though, it dictates how reliable and how scalable the chat application is.

Key considerations for chat app architecture

As the choices you make will have a long lasting and wide ranging impact, it’s important to look at the specific challenges you’ll face when designing your chat app’s architecture. While this isn’t an exhaustive list, it does focus on the considerations most important for a chat application.

Ability to scale

Unlike email or even SMS, chat is an almost exclusively realtime medium. That sets a high bar for user expectations. Demand ebbs and flows depending on events, such as the launch of a new product, and also according to the routines of your user base.

Combined, that means your chat app architecture must be able to respond quickly to changes in demand. Delayed or missing messages, for example, are a consequence of an overloaded architecture and they are a sure way to dent user confidence. But it would be wasteful and costly to run your application on huge servers just in case demand spikes. Instead, you need to build your architecture so that it can scale up and down as needed and that requires two related architectural choices:

Microservices architecture: Break your application into multiple small components that each does one thing well and that can operate independently of the others.
Horizontal scaling: Rather than scaling up with ever larger machines, build your application in such a way that multiple instances of each service can run in parallel. That allows you to add more instances when demand is high and close unneeded instances when demand is low.

Fault tolerance

Lack of capacity is just one reason for application failure. To your end users, though, whether message delivery is failing because your chat backend is over capacity or because it has gone offline altogether, it looks just the same. Further to scale, there are other situations to consider. What if a DDOS attack severely degrades connectivity to the data center, for example? Inevitably, some form of outage will degrade your app’s performance or take it offline entirely. And downtime is costly.

Creating a horizontally scalable microservices architecture can feed into improving your application’s ability to withstand faults. At the heart of enabling that horizontal scalability is the idea that a single service can exist across multiple servers or virtual machines (VM). It’s not a huge leap to use that as a way to overcome many causes of downtime. If a single VM suffers a failure and goes offline, others running that same service can pick up the workload. But what if there’s a wider issue that means multiple other instances of that service are also unavailable?

According to research by the Uptime Institute, 43% of significant outages are caused by power outages. Perhaps ironically, with a majority of those due to uninterruptible power supply (UPS) faults. That could mean that an entire data center or cloud region is unavailable at the same time. In such cases, having those services also running in separate cloud regions or datacenters enables you to route around infrastructure outages.

Latency and global reach

Delivering a realtime experience becomes harder as your user base becomes more geographically dispersed and you look to expand your chat app globally. The further someone is from your servers, the greater the latency they’ll experience.

Latency levels at 100ms is the point at which a software experience feels instantaneous. Beyond that, users notice the delay. According to Verizon, the latency between the UK and Sydney, Australia is around 243 ms. And that’s before any further delay caused by processing on your servers or imperfect network conditions.

The good news is that some of the architectural work necessary to enable fault tolerance and horizontal scalability can also set you up for improving latency. In particular, the ability to run multiple copies of the same service lets them be deployed in different geographical locations. However, the bad news is that the work to actively manage a global network of services and routing traffic between them appropriately is substantially more complex than just having one or two failover regions.

That’s where a platform like Ably comes in. With our own global edge network, our realtime platform maintains reliable data synchronization, at scale, between globally distributed chat clients and your chat backend.

Message synchronization and queuing

Today, it’s a basic expectation that messages sent while participants are offline are waiting for them when they log on.

To solve this challenge — queuing messages for later delivery and also ensuring all of a user’s messages are available wherever they log-in — requires solutions, such as:

Logging each user’s stream of messages and then replaying them when their device connects.
Delivering each message to that user’s chat history in the backend and then sending the client the history when it reconnects.

That makes a database system, one that scales just as well as the rest of your chat app, a core part of your architecture.

Transport mode

The protocol you choose for transporting data between chat client and server will affect the performance and shape of your backend architecture. Each protocol has its pros and cons. For example, choose WebRTC and, under perfect network conditions, communication could be marginally faster than other choices because it relies on UDP. However, as with all UDP traffic, WebRTC offers no delivery or ordering guarantees. As a result, you’ll need to build your own logic to confirm message delivery and enforce message order, as well as to handle retries where needed.

WebSocket, on the other hand, uses TCP at the transport layer. That increases the chance that your messages will be delivered reliably. However, WebSocket is not a silver bullet because it doesn’t provide its own mechanism to automatically recover when connections are terminated.

In circumstances where network conditions are likely to be poor, MQTT offers three levels of message delivery guarantee, with each one adding more protocol overhead and, so, latency:

Implementing push notifications

Just as your chat users will want to catch up on what they’ve missed, there are some messages that they’ll need to see at the moment they arrive. Mobile OS and web browser push notifications draw the user’s focus back to your chat app when, for example, they are tagged in a message or a keyword they follow is mentioned.

Your application will need a service that tracks which platform is used by each individual and triggers a notification on those services. Many engineering teams choose to work with an intermediary service, such as Ably, to simplify sending push notifications.

The structure of chat architecture

While the fundamentals of chat remain the same from one app to the next, use cases vary enormously. At one extreme was Yo, a “chat” app that allowed people to do one thing: send the message “Yo” to other users. At the other end is WeChat, which has grown from instant messaging to now offering payments, social media, video calling, and even games.

Despite that variety, almost all commercial chat applications share a similar client-service architecture. The variety comes in precisely what happens on both the client and the server sides. For example, while pretty much all modern chat platforms will have a service to store messages, some might use a single relational database, while others use a column store such as Casandra for its ability to model data as a time series, or even something like Apache Kafka. Speaking of which, how messages move around the various services in the backend varies greatly. Message streaming tools, such as Apache Kafka and Ably, distribute messages according to their content, source, and destination. That results in an event driven architecture where the message effectively travels from one service to the next.

7 core components of chat app architecture

Just as the client-server architecture remains a fundamental pattern, the core components in a chat application’s architecture remain largely unchanged even as the app’s audience grows and its functionality becomes more ambitious with features like video calling, file transfers, and integrations with third-party services..

And, although they might come in different guises and combinations, such as a home built auth server vs Okta or an append-only data store vs a document database, there are seven components that make-up a typical chat application architecture. Of course, in a distributed microservices architecture each of these components would themselves be split into many smaller parts but the basic pattern remains the same.

1. Application server

This is the heart of your architecture where the logic specific to your application executes. That could be running code directly on the server. For example, parsing and acting on slash commands, such as Slack’s /search command, or triggering other system functionality such as retrieving an image from the media store.

2. Load balancer

Directing inbound traffic to the right place can quickly become complex. Whether it’s routing to the geographically closest server or distributing load amongst multiple instances in the same data center, your load balancer spreads traffic evenly across your available resources so that you can deliver a consistent service.

3. Streaming event manager

Arguably the hardest part to get right, the server-side of your chat application needs to make sure that messages flow reliably and quickly from one service to the next and then to the destination clients. For example, after arriving at the back end, a new message might first go to the logging service and then onto the data store. From there, a command parsing service might look for and act on user commands, such as searching message history or setting reminders. In practice, a message will pass through multiple services before the system then pushes it out to the relevant subscribers.

It’s a substantial job to coordinate each movement so that messages arrive at and depart from services as efficiently as possible. Whether you build it in-house or use a realtime platform-as-a-service provider such as Ably, the event manager is equivalent to your chat app’s blood system. Constantly moving, carrying data to the right place, powered by instructions from its “heart” in the form of the application server.

4. User authentication and user manager

Research by Verizon shows that 82% of data breaches result from people mishandling passwords. Data from IBM, meanwhile, shows that the average cost of a data breach is $3.86 million. The increasingly central role that chat plays in customer service, replacing email in work environments, and other commercially sensitive situations, your chat’s user authentication plays a key role in guarding against potentially bad actors.

The user authentication component of a chat application provides the initial challenge when a user makes their first connection in a particular session and then continues to verify that each subsequent action is authorized. Depending on your particular security needs, that authentication component calls out to a third-party service such as Auth0 or a library that you implement locally, such as PassportJS. You’ll also need to judge the right trade-off between how certain you want to be of someone’s identity versus reducing user experience friction. For example, multi-factor authentication using a hardware token will give you greater confidence in someone’s identity but will make it harder for users to register with and log in to your chat application.

Once the authentication component verifies the user’s identity, another architectural choice is between token or session based authentication. With token authentication, the auth component will create a token, often in the form of a JSON Web Token, that is then stored in the header of each request between client and server. By verifying the token, the server assigns each request to the relevant logged-in user. Session based auth, on the other hand, sets a session ID cookie on the client side and checks that against each request. There are pros and cons to both methods.

The auth component is also responsible for assigning the permissions each user has, along with other profile information. Typically, the auth component will retrieve those details from the chat applications data store. Permissions include the channels to which they’re subscribed and any special roles they play such as moderator.

5. Presence

At scale, managing user presence is a non-trivial problem. A dedicated presence service keeps track of whether each user is connected, which might not always be clear where someone is logged-in on multiple devices, and how. For example, showing that someone is connected using a mobile device helps set expectations around their ability to respond in detail, whether they’re likely to make autocorrect errors, and so on. Other status information, such as someone being “away” or “busy”, also helps to replicate some of the context that is lost from real world interactions.

Whether it’s original video or meme GIFs, rich media is a core part of many chat experiences. However, storing and transporting those media files can be both costly and hard to scale. One way to reduce costs and improve performance is to maintain a central media store within your own back-end architecture and then hand delivery to a content distribution network (CDN).

7. Database

More than likely, the chat backend will have several data stores to handle different types of data. But there needs to be at least one way of storing and querying data such as the chat messages themselves, user status, status messages, and so on.

Options for delivering realtime chat applications

Whether you’re integrating chat into an existing product or looking to launch a new dedicated chat service, you’re almost spoilt for choice when it comes to choosing how to build your server-side application architecture. Let’s look at the three main options available:

Build it all in-house: Building everything from scratch using your own engineering team can seem like a tempting option. On paper, it looks like you get full control of the experience you deliver to users and there’s less to pay out in software licensing fees. Seen from a different angle, though, that freedom comes with the responsibility to plan, build, and maintain everything in-house.
Integrate proven third-party tools: Alternatively, you can retain control without dividing your team’s attention between the competing and varied needs of each specialized part of your chat app’s architecture. Employing a CDN for rich media and a realtime PaaS, such as our own, are two ways you can hand off some of the most challenging aspects of delivering chat.
Use a white-label chat platform: If your chat needs are relatively straightforward and you are happy to trust the future direction of your functionality to another company’s product team, you can integrate a white-label chat platform into your existing application. This option reduces your engineering costs at the expense of flexibility and ongoing licensing costs.

When you come to choose between these options, consider how well each one can help you deliver on the key considerations we looked at earlier, such as low latency delivery, global reach, high uptime, and rapid scaling. Check out our blog for more on how to choose the right solution for your needs.

Ably’s approach to chat app architecture

At Ably, we understand the importance and challenges of delivering scalable, reliable, feature-rich chat applications — and companies like HubSpot use Ably to serve hundreds of thousands of chat sessions every month.

Whether it’s for customer support, connecting users of a multi-side marketplace, engaging gamers, or delivering on unique requirements such as HIPAA compliant chat, Ably handles the hard infrastructure while giving you the flexibility to tailor the solution you need, including:

Globally distributed network: Seven core routing data centers and 385 edge accelerations points of presence provide worldwide low latency reach.
Pub-sub architecture: Ideally suited to chat applications, our pub-sub implementation guarantees precisely once message delivery in order.
Rich integrations: Work with your existing tools and platforms through webhooks, APIs, and language-specific SDKs.

Build scalable chat applications with Ably

Building your chat application with Ably means benefiting from global infrastructure, ongoing improvements from a specialist realtime engineering team, and the freedom to build the right functionality for your audience.

Get started in just a few minutes:

Sign-up for your free Ably account.
Follow our tutorial for building a scalable chat app.
Then go onto learn more with our clear tutorials on how to build rich chat functionality.

FAQs

How do you ensure low latency and global reach in a chat application architecture?
You ensure low latency by deploying edge servers or Points of Presence (PoPs) close to your users, selecting efficient transports (e.g. WebSockets or MQTT where appropriate), and using global load balancing. These reduce round-trip times and help preserve performance even when users are spread across regions.

What are the essential core components of a chat app architecture?
Core components include an application server, user authentication and user manager, presence service, media store, streaming event manager, load balancer, and a reliable database. Together, these components enable features like message ordering, fault tolerance, scalability, and secure user identity.

How do you maintain message integrity and ordering at scale?
You use protocols and systems that guarantee delivery guarantees (exactly-once or at least-once), employ sequence identifiers or buffering, and ensure proper queuing and retry logic.

What trade-offs are there between building your own architecture vs using a realtime PaaS like Ably?
Building your own gives you full control over every component and customization, but it requires more engineering effort, longer development time, and larger maintenance burden. Using a realtime PaaS lets you offload much of the complexity (transport protocols, scaling, edge distribution, message ordering etc.), speeding time to market and improving reliability at scale.

How do you manage faults, outages, and ensure reliability in chat systems?
You design for fault tolerance via horizontal scaling, redundancy (multiple instances of each service), regionally distributed infrastructure, and fallback strategies. Also, using services that automatically reroute traffic or provide high availability helps maintain uptime in case of server, network, or data-center failures.

Originally published at https://ably.com.

Scalable chat app architecture: How to get it right the first time was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

Building a realtime chat app with Next.js and Vercel

Ably Team — Mon, 23 Feb 2026 11:21:00 GMT

This post will walk through the creation of a realtime chat application using Ably Chat with Next.js and deploying it to Vercel.

You’ll learn how to:

Create a brand new Next.js application
Adding realtime functionality with Ably
Create a Next.js Vercel Serverless API
Use React Hooks with Ably
Host your app on Vercel

Check out a running version of the app or fork the app on GitHub.

WebSockets in Vercel with Ably

Vercel allows users to deploy Serverless Functions, which are essentially just blocks of code that provide a response to an HTTP request. However, these functions have a maximum execution timeout, which means that it is not possible to maintain a WebSocket connection this way.

This is where Ably comes in. The client can connect to an Ably Chat room and send and receive messages on it to add realtime functionality to your app by managing your WebSocket connections for you.

We’ll start by going over how to build an app that uses realtime functionality.

Take a look at some of the challenges of implementing a reliable and highly-performant client-side WebSocket solution for JavaScript apps.

What are we going to build?

We’re building a realtime chat app that runs in the browser. It will be built upon the Next.js create-next-app template and contain a React component that uses Ably Chat SDK to send and receive messages. In this tutorial, we’re specifically using @ably/chat/react in Ably Chat SDK instead of the lower-level class API. Hooks let us confine all realtime logic to the browser (so it won't run in Next.js server components), automatically manage subscribe/cleanup lifecycles, and slot neatly into our React component tree on Vercel. We'll walk through integrating the SDK to build a UI for sending and receiving messages in realtime. We'll also write a Next.js serverless function that will be used to connect to the Ably Chat SDK.

Try the live demo → https://next-js-chat-app.vercel.app/

A bit about Ably Chat

At its core, the Ably Chat SDK is built on Ably’s globally-distributed, low-latency data stream network. Messages are exchanged through chat “rooms,” which wrap around Ably Channels to offer chat-specific functionality. When a user sends a message, it is published to the room, and all other members subscribed to the room immediately receive it. This pub/sub model ensures reliable, instant delivery between connected clients.

Traditionally you would publish and subscribe directly on raw channels, but in this tutorial we’re using Ably Chat SDK’s React Hooks to simplify everything. All of these hooks play well in your Client Components, automatically managing lifecycle and cleanup when components mount or unmount.

Beyond basic message exchange, Ably Chat supports a wide range of chat features out of the box, including typing indicators, presence events, read receipts, message history, and even edit/delete support. Check out our docs to learn more.

1. Setting up dependencies

To build this app, you’ll need:

Once you have your Ably account you’ll need an API key to authenticate with the Ably Service. To get an API key:

Visit your app dashboard and click on Create New App.
Give the new app a name.
Copy the Private API key once the app has been created. Keep it safe! This is how you will authenticate with the Ably service.

Vercel already provides the Next.js command line tools to help us. They don’t need to be installed on your system, as they’re executed using npx.

2. Scaffolding the Next.js chat app

Start building the chat application by using create-next-app to scaffold a new Next.js-based application. In your terminal, type:

npx create-next-app@latest

Answer the prompts as follows:

What is your project named? nextjs-chat-app
Would you like to use TypeScript? No
Would you like to use ESLint? Yes
Would you like to use Tailwind CSS? No
Would you like to use `src/` directory? No
Would you like to use Turbopack for `next dev`? No
Would you like to use App Router? (recommended) Yes
Would you like to customize the default import alias? No
What import alias would you like configured? @/*

After the prompts, create-next-app will create a folder with your project name and install the required dependencies. Change directories into your application directory:

cd nextjs-chat-app

Next, create a file called .env in the root of the directory. This is where we'll put the project's environment variables. Add your Ably API key to the .env file:

ABLY_API_KEY=your-ably-api-key:goes-here

Test that the application runs correctly by running the following command:

npm run dev

The Next.js dev server will spin up and you’ll see a default Next.js starter app. This is what we’ll build our chat app on top of.

The realtime chat app architecture

The topology of our Next.js app will look like this:

─ .env
├─ .gitignore
├─ package-lock.json
├─ package.json
├─ README.md
|
├─── components
│    ├─ Chat.jsx
|    ├─ ChatBox.jsx
│    ├─ ChatBox.module.css
|
├─── app
│    ├─ globals.css
│    ├─ layout.js
│    ├─ page.js
│    │
│    └─── api
│         └─ route.js
│
└─── public

/app/page.js is the home page
/app/api/route.js is our Ably token authentication API
/components/Chat.jsx is the main chat host component
/components/ChatBox.jsx the component that contains the actual chat UI
/components/ChatBox.module.css contains the styles for the chat component

Let’s walk through how this application is built.

3. Authenticating an Ably client

There are a number of ways to authenticate an Ably client. For our application, we’ll use token authentication. Token authentication uses a trusted device with an API key, in our case an API route that we’ll build using Next.js, to issue time-limited tokens to untrusted clients.

In version 15, Next.js introduced the new App Router. Built on React Server Components it supports shared layouts, nested routing, loading states, error handling, and more including API routes via its Route Handler feature. Let’s build that API route.

Start by installing the Ably npm package in the root of your new app:

npm install ably
npm install @ably/chat

Next, create a file called ./app/api/route.js. Add the following code:

import Ably from "ably";

// ensure Vercel doesn't cache the result of this route,
// as otherwise the token request data will eventually become outdated
// and we won't be able to authenticate on the client side
export const revalidate = 0;

export async function GET(request) {
  const client = new Ably.Rest(process.env.ABLY_API_KEY);
  const tokenRequestData = await client.auth.createTokenRequest({
    clientId: "ably-chat-demo",
  });
  return Response.json(tokenRequestData);
}

This function uses the Ably SDK to create a tokenRequest with your API key. The token will be used later — it allows you to keep your “real” API key safe while using it in the Next.js app. By default, this API is configured to be available on http://localhost:3000/api/. We’re going to provide this URL to the Ably Chat SDK in our client to authenticate with Ably.

Next.js server vs. client components

Before we start creating pages in our application, it’s important to understand how Next.js renders content. The framework supports multiple rendering methods including server-side rendering (SSR), static-site generation (SSG), and client-side rendering (CSR). There are many pros and cons to each rendering method (too many to cover in this post), so if these concepts are new to you, Google’s web.dev site has a very good introduction to rendering on the web that can help you understand rendering options.

Next.js defaults to server-side rendering-meaning the page HTML is generated on the server for each request and sent to the browser. This means that by default when you create a new component in your React app, you are creating a Server Component.

Server Components offer a lot of benefits, but one drawback is that they don’t have access to browser APIs. To allow you to create components that can access those APIs, Next.js also includes the ability to create Client Components.

To indicate that you want a component to be a Client Component, you use React’s ‘use client’ directive at the top of your component. Doing this tells Next.js that it should render the component in the browser, not on the server.

Our chat UI ( components/Chat.jsx) needs to open a realtime WebSocket connection via the Ably SDK and run React hooks in the browser, so we simply add 'use client'; at the top of that file. Next.js will then automatically exclude it from server renders - no lazy loading or dynamic imports required.

4. Building the components

In the root of your application, create a new components folder and within create a new React component file named Chat.jsx. Scaffold the initial component using the code below:

'use client';
import * as Ably from 'ably';
import { ChatClient, AllFeaturesEnabled } from '@ably/chat';
import { ChatClientProvider, ChatRoomProvider } from '@ably/chat/react';
import ChatBox from './ChatBox.jsx';

const roomOptions = {
  history: { limit: 50 },
  reactions: {}
};

export default function Chat() {
  const realtimeClient = new Ably.Realtime({ authUrl: '/api' });
  const chatClient = new ChatClient(realtimeClient);

  return (
    
      
        
      
    
  );
}

Pages in Next.js are React components, so the app/page.js home page is the React component that contains the page layout.

Open page.js and replace the default scaffolded content with this markup:

import Head from 'next/head';
import dynamic from 'next/dynamic';

const Chat = dynamic(() => import('../components/Chat'), {
  ssr: false,
});

export default function Home() {
  return (
    
      
        Realtime Chat App with Ably Chat, Next.js and Vercel
        
      
      
        Ably Chat Demo

        
      

      
        Powered by Vercel and Ably
        
          
            
            
        
      

    

  )
}

Next, in the same app directory, locate the globals.css file and replace its contents with the basic CSS styles we'll use in our application which you can find in the project's GitHub repository.

Run the app. You should see this:

Run the app to validate everything is working properly. The Chat component is mounted and joined to a room, though you won’t see any visible UI yet. We’ll add the actual chat interface in the next section.

5. Writing the chat component logic

The chat app logic is contained inside a new ChatBox component. To build this, start by creating two new files in the components directory: ChatBox.jsx and ChatBox.module.css.

You can copy the styles for ChatBox.module.css from the project's GitHub repository. Next, open the ChatBox.jsx file and add the import statements we'll need at the top of the file:

import React, { useEffect, useState, useRef } from 'react';
import { useMessages } from '@ably/chat/react';
import styles from './ChatBox.module.css';

Initialize ChatBox

Define the ChatBox component, and add variables to the function to store references to some of the HTML elements in the code.

export default function ChatBox() {
  const inputBox = useRef(null);
  const messageEndRef = useRef(null);
}

Set up the state properties used in the component:

const [messageText, setMessageText] = useState('');
const [messages, setMessages] = useState([]);
const messageTextIsEmpty = messageText.trim().length === 0;

messageText will be bound to a textarea element where messages can be typed
receiveMessages stores the on-screen chat history
messageTextIsEmpty is used to disable the send button when the textarea is empty

Listen for and render incoming messages

We’re using the useMessages hook from @ably/chat/react, which under the hood subscribes to the room's message stream and calls your callback for each new message. You don't need to manage subscribe/unsubscribe yourself - just pass a listener function into useMessages, and it will give you a send method plus realtime updates.

Add the following to ChatBox.jsx:

const { send: sendMessage } = useMessages({
  listener: (payload) => {
    const newMessage = payload.message;
    setMessages((prevMessages) => {
      if (prevMessages.some((existingMessage) => existingMessage.isSameAs(newMessage))) {
        return prevMessages;
      }
      const index = prevMessages.findIndex((existingMessage) =>
        existingMessage.after(newMessage)
      );
      const newMessages = [...prevMessages];
      if (index === -1) {
        newMessages.push(newMessage);
      } else {
        newMessages.splice(index, 0, newMessage);
      }
      return newMessages;
    });
  },
});

Listener callback: Runs for every incoming message payload, extracting the structured message object.
Deduplication: We check isSameAs(newMessage) to avoid showing the same message twice.
Ordered insertion: We compute an index using after(), then insert the new message at the correct chronological position - or append it if it's the newest.
Automatic cleanup: The hook manages subscribe/unsubscribe for you when the component mounts or unmounts.

Handle sending messages

To send a chat message, we’ll define a few helper functions:

sendChatMessage() publishes the message to the Ably room.
handleFormSubmission() is tied to the form's onSubmit event, which prevents a page reload and calls the send function.
handleKeyPress() checks for the Enter key and sends the message directly if pressed inside the textarea.

These get added to to ChatBox.jsx:

const sendChatMessage = async (text) => {
  if (!sendMessage) {
    // hook isn't ready yet
    return;
  }
  try {
    // send via Ably Chat SDK
    await sendMessage({ text });
    // clear out the textarea and restore focus
    setMessageText('');
    inputBox.current?.focus();
  } catch (error) {
    console.error('Error sending message:', error);
  }
};

const handleFormSubmission = (event) => {
  event.preventDefault();
  sendChatMessage(messageText);
};

const handleKeyPress = (event) => {
  // only send on plain Enter (not Shift+Enter)
  if (event.key !== 'Enter' || event.shiftKey) {
    return;
  }
  event.preventDefault();
  sendChatMessage(messageText);
};

Rendering messages in the chat UI

Next, we need to construct the UI elements to display the messages. To do this, we will map the received Ably messages into HTML span elements:

const messageElements = messages.map((message, index) => {
  const key = message.serial ?? index;
  return (
    
      {message.text}
    
  );
});

sendChatMessage(text) checks that sendMessage is available, then awaits it to publish your payload. On success, it clears the input and focuses it; on failure, it logs the error.
handleFormSubmission prevents the default page reload and delegates to sendChatMessage.
handleKeyPress looks for the Enter key (ignoring Shift+Enter line breaks), prevents the newline, and sends the message immediately.

In order to keep the message box scrolled to the most recent message (the one on the bottom) we’ll need to add an empty div element into the message container, which will then be scrolled into view whenever the components re-renders. This is the element that we'll add to the UI:


  {messageElements}

We use a useEffect hook along with scrollIntoView() to scroll the message history to the bottom whenever the component renders.

useEffect(() => {
  messageEndRef.current?.scrollIntoView({ behavior: 'smooth' });
}, [messages]);

Returning the UI markup

Finally, we will write the React component markup with the event handlers all bound to the onChange and onKeyPress events in JSX.

The markup itself is just a few div elements and a form with a textarea for user input. There are two calls to the react ref function, which allows us to capture a reference to the elements when they are rendered so that we can interact with them in JavaScript. The returned markup will look like this:

return (
  
    
      {messageElements}
      

    

    
              ref={inputBox}
        value={messageText}
        placeholder={'Type a message...'}
        onChange={(e) => setMessageText(e.target.value)}
        onKeyPress={handleKeyPress}
        className={styles.textarea}
      >
      
    
  

);

Swap over to your browser (make sure you restart the application server if you need to) and you should see a functioning chat application now.

6. Hosting on Vercel

We’re using Vercel as our development server and build pipeline.

As we mentioned at the beginning, in order to deploy your new chat app to Vercel, you’ll need to:

Create a GitHub account (if you don’t already have one)
Push your app to a GitHub repository
Create a Vercel account
Create a new Vercel app and import your app from your GitHub repository. (This will require you to authorize Vercel to use your GitHub account)
Add your ABLY_API_KEY as an environment variable
Watch your app deploy
Visit the newly created URL in your browser!

7. Make it your own

There are a few ways that this example could be extended:

Add message history

There is currently no chat history in this demo — you’ll only see messages that come in after you join the chat. You could expand this demo by using Ably Chat’s built-in message history to retrieve messages sent earlier, like this:

import { useMessages } from '@ably/chat/react';

const MyComponent = () => {
  const { get } = useMessages();

  const handleGetMessages = () => {
    // fetch the last 3 messages, oldest to newest
    get({ limit: 3, orderBy: OrderBy.OldestFirst })
      .then((result) =>
        console.log('Previous messages: ', result.items));
  };

  return (
    
      
    

  );
};

Add room reactions

Use the useRoomReactions Chat React Hook to both send reactions and listen for incoming reaction events. The hook returns a send function; you register a listener to update your own React state when a new reaction arrives. Here's how to incorporate it in ChatBox.jsx. First, add this import statement at the top:

import { useRoomReactions } from '@ably/chat/react';

Add typing indicators

You can show when other users are typing using the Chat React Hook useTyping. Just incorporate it into the existing ChatBox.jsx with this import statement:

import { useTyping } from '@ably/chat/react';

Learn more about typing indicators in the Ably Chat typing indicator docs.

Let us know

Hope you have enjoyed this tutorial.

If this tutorial was helpful, or you’re using Next.js and Ably in your project, we’d love to hear about it. Drop us a message on X or LinkedIn.

FAQs

What is the Ably Chat SDK and why use it with Next.js?
The Ably Chat SDK provides realtime messaging APIs built on Ably’s globally distributed data network. When integrated with Next.js, it simplifies WebSocket-style communication and enables features like presence, moderation, message history, typing indicators, and more.

How do you authenticate Ably clients securely in a Next.js app?
Authentication is handled using a serverless API route on Vercel. This route issues short-lived tokens based on your Ably API key, ensuring the secret key is never exposed while allowing clients to connect safely.

Can Vercel handle persistent WebSocket connections directly?
Not natively — Vercel serverless functions have timeouts that prevent persistent sockets. Ably solves this by maintaining realtime connections for you, while your Vercel app interacts through APIs and the Chat SDK.

How do you deploy a realtime chat app with Next.js to Vercel?
You push your project to GitHub, connect it to Vercel, set environment variables like ABLY_API_KEY, and deploy. Vercel handles the build and hosting, while Ably manages realtime messaging globally.

What features can I add to enhance the chat experience?

Beyond sending messages, you can use Ably’s Chat SDK to add moderation, typing indicators, room and message reactions, presence, read receipts, or message history. These features improve usability and make your chat app feel polished and production-ready. See the Ably Chat documentation for the full feature list.

Originally published at https://ably.com.

Building a realtime chat app with Next.js and Vercel was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

What it takes to build a realtime messaging app

Ably Team — Fri, 20 Feb 2026 14:21:10 GMT

What does it really take to build a high-quality realtime messaging app? Choosing the right architecture up front — covering latency, reliability, message ordering, persistence, presence, and security can mean the difference between smooth user experiences and costly failures as your audience grows. Learn more about Ably Chat →

We all expect online experiences to happen in realtime. Messages should arrive instantly, dashboards should deliver business metrics as they happen, and live sports scores should broadcast to fans around the world in a blink.

This expectation is even higher for realtime chat and messaging, which is now embedded in everything from e-commerce platforms to online gaming. But building a competitive instant messaging or live chat app requires some heavy lifting-especially if you’re starting from scratch.

To help you hit the ground running, this article provides an overview of what live chat and instant messaging app development involve, including: the features you need, the challenges you can expect, and everything you need to know about delivering reliable, scalable realtime messaging.

The foundations of a successful realtime messaging app

Let’s start by taking a closer look at what every winning chat app and instant messaging platform has in common.

Low latency data delivery: The promise of realtime means users expect to, and should, receive messages on their devices without delay. This relies on a combination of factors, including a high throughput, an efficient, low-latency messaging protocol (like WebSockets), and geographic proximity between users and servers.
Message delivery guarantees: Chat and instant messaging rely on messages being delivered to the right client, in the right order, at the right time. Ensuring data integrity is difficult, but critical to prevent missed, unordered, or duplicate messages.
Scalability: Having a highly available and resilient infrastructure allows a realtime chat or instant messaging app to grow in-line with its user base. The Pub/Sub messaging pattern is a good foundation for this, since it decouples the communication logic from the business logic — making your system easier to scale.
Elasticity: A chat or instant messaging app that crashes when it gets more users than expected will never remain competitive. To succeed in the long term, your app has to be able to react to sudden changes and elastically scale to handle a high, unpredictable, and fluctuating number of concurrent users.
Reliability: Chat and instant messaging need to be always available-even if something goes wrong. Building in redundancy ensures service continuity even in the face of failure, either at a regional level (instances and datacenters) or at a global level (network issues, outages, or even DDoS attacks).
Features for a richer realtime messaging experience: Chat and instant messaging should be fun as well as functional. The standard is to send and receive messages, but the bar is now higher since chat apps like WhatsApp, Messenger, and Slack popularized features like emoji reactions, read receipts, and GIFs to upgrade the chat experience.

What realtime messaging app features to build

The idea of building a chat or instant messaging app is simple, but with the likes of Slack and WhatsApp setting the standard for realtime messaging, it’s not enough to just set up a sleek UI that lets users send and receive messages.

With that in mind, here are the features you need for a well-rounded experience that won’t push users to a better-equipped competitor.

Core instant messaging and chat features: What your app can’t do without

Realtime messaging: Your chat app must enable users to join a chat to send and receive messages in realtime. Pub/Sub messaging is ideal for chat apps since it supports 1:1 messaging as well as 1:many, which is useful for group chat, chat rooms or channels.
Message storage: Persist messages so users can retrieve old conversations. You can choose between different storage options and differing data retention policies, depending on the specifics of your use case.

Authentication and authorization: Your chat app needs a way to manage access for chat participants. This can be through their social media account using a third-party login, or simply their email and phone number. You also need to consider managing permissions for different roles, like only allowing admins to moderate a group chat.
Access to contacts: Your chat app must give users a simple way to import their contacts. It’s also a good idea to show which contacts are already registered in the app and give the option to send an invite to those who aren’t.

Enhanced instant messaging and chat features: Taking your app beyond the basics

Edit/delete messages: Your chat app should give users the option to edit or unsend a published message. This means thinking about the most suitable data structures for searchability, and making sure the message payload is properly updated and stored.
Complex text: Your chat app should be able to interpret text that triggers a specific action. For example, URLs should become clickable or show a media thumbnail. A name tagged with @ should be highlighted and automatically send the tagged user a notification.

File sharing: Adding support for different file formats to your instant messaging or chat app will enable users to send photos, videos and audio to liven up the conversation. This is particularly important for instant messaging apps, where participants are engaging in social interactions. Common protocols for this include WebRTC, WebSockets, and IRC.
Audio/video calling: If you want your chat or instant messaging app to emulate the likes of WhatsApp, then you should enlist protocols like WebRTC to enable video calls within your chat app — or integrate third-party APIs that handle everything for you.
Push notifications: Although the in-app experience is important, ensuring that users are aware of new messages even if they’re not using the app is key to engagement and user communication. This can be achieved by sending notifications to users’ devices.

Rich instant messaging and chat UI and UX features: Upgrading the in-app experience

Read receipts: Update your UI to show users when a message has been successfully sent, delivered, and read. You could also show if a message has been edited.
Presence indicators: Give visibility into what others in the conversation are doing in realtime, like when they’re online or the last time they were active in the app.

Message reactions: Offer the alternative for those in a hurry (or lacking a good response) of adding an emoji to a published message, like a heart or a thumbs up.
App updates: Show what’s changed since the user was last active in the app, like how many unread messages they have, or how many times they’ve been mentioned in a thread.

The challenges of building a realtime messaging app

It’s one thing to map out what to build for your chat app, but engineering a dependable chat system is a whole other ballgame. The time you spend battling code, uprooting issues, and juggling protocols can add up quickly-and time is money.

To help you better prepare, here’s an overview of the most common challenges.

Performance

This isn’t just about minimizing latency and bandwidth requirements to send data end-to-end, but also minimizing the variance in them. Too much variance can make messaging unpredictable and create an exasperating user experience. It can also make building and scaling features tricky since you wouldn’t know for certain whether they’ll perform as expected in varied operating conditions.

This is where a PaaS comes in handy. You’d know what median latency and throughput performance to expect, as well as the specific operating boundaries to design and build new features around.

TIP: Realtime messaging applications, especially those with interactive sessions like voice or video calls, are best experienced with latencies under 100ms.

Reliability

Ensuring reliability requires a stable, fault-tolerant architecture that has redundancy built in at a regional (and sometimes global) level. But maintaining a reliable distributed infrastructure comes at a high operating cost, and there’s a long list of capabilities (like global replication and automatic fail safes) that need to be developed.

A common approach is to pay for a PaaS that covers all your bases, unless you have the time and resources to build a fault-tolerant system.

Scalability

Architecting your chat app for scale takes more effort upfront since you need to spend more time on design before you build. You also have to make sure you have the funds to host, run, and maintain your app as it grows.

Partnering with a reputable PaaS can provide a highly elastic infrastructure so your app is never short on capacity and your team isn’t putting out operational fires at 3AM.

Cross-platform development

It’s no longer enough to develop your chat app for one lonely device. It now has to be accessible across various platforms and operating systems. Of course, the development time and technical challenges multiply with each additional platform, as you’ll need to navigate different SDKs and programming languages.

Luckily there are providers who offer SDKs targeting multiple platforms, so teams can develop using the languages and tools they already know.

Security and compliance

Security is top of mind for most users today, and messaging in particular is expected to be private and secure. Additionally, you’ll most likely need to meet compliance requirements (GDPR, SOC2) and any industry-specific frameworks, like HIPAA to protect patient privacy.

At a minimum, you need to encrypt data in transit and at rest, and implement robust authentication, as well as a permissions system.

Capabilities to build rich chat

There’s a lot more to chat once you’ve built out the basics. You also have to think about channels, emojis, read receipts, communicating user presence (online, offline, typing), and more. Presence is a particularly tricky feature to handle at scale.

These are many moving parts to develop and maintain in-house, so it’s a good idea to integrate APIs that already offer these additional features to let your team focus on core chat capabilities.

How to build a realtime messaging app

As we’ve outlined above, the path to building a realtime messaging app can be fraught with challenges. The approach you should take depends on the timeline and budget for your project, the engineering resources you have available, as well as your ambitions.

Whilst most engineering teams can knock out an MVP (Minimal Viable Product) in around 3–5 months, with a budget in the range of $30,000 to $50,000 for a single platform (like Android), the offering will be fairly basic. According to AppInventive, building a chat app like WhatsApp can cost up to $250,000. And that’s just the initial cost to build and manage the system-there’s no telling how much you’ll spend over time with ongoing scaling, maintenance, and future enhancements.

93% of projects required at least 4 in-house engineers to build the realtime messaging infrastructure.
10.2 person-months was the average time taken to complete a realtime messaging infrastructure project in-house.
46% said escalating costs as a major challenge during the development of their realtime messaging infrastructure.
50% of self-built realtime messaging platforms required $100K-$200K a year in upkeep.

Given that a lot of the challenges faced in building a realtime messaging app — performance, reliability, and scalability — are associated with the realtime infrastructure behind the chat app, a common fork in the road is deciding whether to build the infrastructure yourself or use a PaaS that does most of the work for you. Let’s dig into the two options.

Option 1: Building the realtime messaging infrastructure yourself

WebSockets are widely used as a key technology for enabling realtime, two-way communication for chat and instant messaging apps. You have the choice of building your own bespoke protocol on top of raw WebSockets, or you can use an open-source WebSocket library to build your chat app.

For example, Socket.IO is a popular WebSocket library that’s often used to develop chat apps for its ability to broadcast events to multiple clients, fallback options, and automatic reconnections. The downsides of Socket.IO are that it’s difficult to scale, has limited native security features, it’s designed to work in a single region, and doesn’t guarantee message ordering.

Keep in mind that whether you build your own protocol or use an open-source library, you have to host, scale, and manage the infrastructure yourself. This brings us to the next DIY option, which is to stitch together different cloud-based services that will host the servers for you. Even so, you still need to configure and optimize how the different services connecting your chat app should interact and operate.

Overall, the DIY approach is certainly doable, but most are barebones implementations that leave a lot for you to figure out on your own. They can prove challenging if you have a particular need to scale, or have a limited workforce to do so. In our survey of over 500 engineering leaders in the UK and US, we discovered that:

60% switched to PaaS to improve live UX with stable infrastructure.
56% ranked risk reduction as the second most compelling reason to swap to a PaaS.
55% said the third critical reason to move to a PaaS was to redeploy engineers into core product work.
65% of DIY solutions had an outage or significant downtime in the last 12–18 months.

The reality is that building realtime infrastructure is hard and the timeline can be unpredictable. Plus, not every organization has the time, money, or expertise to build realtime infrastructure in-house. Even Adobe figured it would be easier to buy Figma for $20bn than build realtime multiplayer collaboration into their products.

Option 2: Leveraging a PaaS for your realtime messaging app infrastructure

If you’re looking to deliver a production-ready chat or instant messaging app without the complexity of building everything yourself, a PaaS is your best route.

For example, you could enlist an app development platform like Google-owned Firebase to build, test, release, and monitor web and mobile chat apps. However, it can be limiting when it’s time to scale and build features beyond a simple MVP. Not to mention vendor lock-in, as it restricts you to Google’s ecosystem.

Another option is to choose a flexible PaaS, like Ably, which is a serverless WebSocket PaaS that helps developers build scalable, realtime experiences-like chat-without the hassle of managing the underlying infrastructure.

Predictable performance: Realtime messaging with dependable latencies and global availability, even under extreme load.
Integrity built-in: Exactly-once delivery and ordered messages ensure your conversations never lose context.
Scalable efficiency: A flexible, consumption-based pricing model-plus upcoming features like batching and aggregation-keeps costs predictable as you grow.
Seamless integration: Ably slots neatly between your frontend and backend, with managed integrations to protocols and third-party services.

The case for using a PaaS gets stronger the deeper you dig into the challenges of building and maintaining the infrastructure. In addition to the statistics above, our survey revealed that:

The bottom line is: if you want to get your instant messaging or chat app to market sooner and grow it quickly, you’re better off using a PaaS that takes care of the heavy lifting and can guarantee predictable performance at scale.

Simplifying your realtime messaging app development with Ably

Building a reliable, realtime messaging app is notoriously hard. That’s why companies such as HubSpot, 17LIVE, SportsBet, InvitePeople, and more choose Ably to power their chat experiences.

gives you everything you need to deliver modern chat at speed. Purpose-built APIs and SDKs across multiple platforms make it easy to add realtime messaging, presence tracking, typing indicators, moderation, reactions, and more. Whether you’re building for livestreams, in-game communication, customer support, or social features in SaaS products, Ably Chat helps you ship a great chat experience 3× faster.

Powered by the Ably platform. Under the hood, Ably Chat runs on the same realtime infrastructure trusted worldwide. That means you inherit:

With the complexities of latency, message integrity, scalability, and cross-platform support handled seamlessly, you can shift your energy to what truly matters — innovating your chat experience, not maintaining its plumbing.

And with Ably Chat SDKs available today, your teams get consistent, idiomatic APIs across every platform, so you ship faster, smarter, and stand out in a world where realtime matters most.

FAQs

How do you ensure message ordering and reliability in realtime chat systems?
You ensure this by using protocols that support delivery guarantees (e.g. exactly-once or ordered delivery), buffering and retry logic, and having infrastructure that persists messages so that clients who briefly disconnect can resume without losing or reordering messages.

When should I build my own realtime infrastructure versus using a PaaS like Ably?
Building your own gives you full control but demands more engineering, cost, and operational overhead; using a Platform as a Service reduces time-to-market, offloads complexity, and helps with scalability and global delivery. The right choice depends on your team’s resources, performance needs, and how critical uptime and latency are for your users.

How do you handle message history and offline users when building realtime chat?
You persist messages, so they can be retrieved after disconnections or for new users joining a conversation, and include logic to fetch unread or historical messages. This ensures consistent user experience and continuity even when people are offline or switching devices.

What kind of security and presence features are essential in a messaging app?
Essential features include user authentication and authorization, secure transport (e.g. TLS), control over permissions, and the ability to see which users are online or offline (presence). Also, mechanisms for moderation and audit are important for maintaining trust and compliance, especially in large-scale or global applications.

Originally published at https://ably.com.

What it takes to build a realtime messaging app was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

AI UX: Reliable, resumable token streaming

Ably Team — Fri, 20 Feb 2026 14:20:01 GMT

Refresh the page, lose signal, switch tabs — the AI conversation just keeps going. That’s what reliable, resumable token streaming makes possible. No restarts, no lost context, just the same response picking up right where it left off. It keeps users in flow and builds trust, making conversations feel seamless. Even better, it unlocks things like switching devices mid-stream without missing a beat. This post explains why users expect it, why it’s hard to build, and what your infrastructure needs to make it work.

The new expectation for seamless AI interactions

As AI becomes woven into everyday apps, users have rising expectations for seamless interactions. An emerging baseline is that an AI conversation or generated answer should not be fragile. Reliable token streaming that survives crashes or reloads is quickly becoming expected behaviour. People now assume an ongoing AI response will continue uninterrupted despite a temporary failure. If your browser tab crashes or you hit refresh accidentally, you’d expect the AI to keep going in the background and resume when you return — just like a video resumes where you left off.

This expectation isn’t theoretical; it’s showing up as a real demand from users and developers. Many have noticed the annoyance of a chatty AI that goes silent after a network blip and forces you to retry the prompt from scratch. Forward-looking teams are already experimenting with ways to make AI streams more resilient. In short, the bar for a smooth AI experience is rising: reliable, resumable streaming is moving from a nice-to-have to a must-have.

What users want, and why this enhances the experience

Users want AI conversations that continue uninterrupted despite failures or reloads. They don’t want to babysit the AI or repeat themselves due to technical faults. Consider this scenario: You’re halfway through a detailed AI response when the page reloads or the network drops. When things come back, you expect the conversation to pick up right where it left off. The same question, same response, no rewind. That’s the baseline now: the AI should just handle it. In practice, this means users now expect a few key behaviours:

Streamed responses resume instantly after a reload. The AI’s answer picks up exactly where it stopped when you refreshed.
Incomplete prompts persist across failures. If you submit a question and the app crashes or you go offline, the AI still finishes the answer. You don’t lose your query or its partial response.
Reconnection doesn’t trigger full re-generation. Coming back online or reopening the app doesn’t make the AI start the answer over from scratch; it continues as if nothing happened.

Under the hood, delivering these behaviours requires that the AI’s generation process not be tied to a single fragile connection. Even if the user disconnects, the system must carry on generating tokens so it can seamlessly resume later. In other words, the conversation’s state should survive independently of the user’s browser tab or device session. This greatly enhances the user experience by ensuring the AI is always “in sync” with the user, no matter the hiccups along the way.

Why this is proving challenging

Building seamless, resumable streaming sounds simple on the surface: keep the tokens flowing, even when something goes wrong. But under the hood, it’s anything but. Most web stacks were never designed for this kind of continuity, and the gaps show quickly when you try to implement it. There are a few core failure points that make reliable streaming difficult to get right:

Stateless protocols like HTTP drop the stream on failure

Most web interactions (think HTTP requests, REST APIs) are stateless and short-lived. If you’re streaming an AI response over a standard HTTP connection and it drops, that request is gone. HTTP has no native concept of resuming a half-finished response. It wasn’t designed for long-lived, continuous streams. This makes it fundamentally ill-suited to delivering multi-turn, token-based output with realtime guarantees.

Streaming logic often lives only in the browser

Many apps place the responsibility for handling AI output in the client — usually the browser tab. If that tab crashes or is closed, any awareness of the current conversation state disappears. Unless the server is explicitly maintaining progress (e.g. buffering the partial response), the result is a hard reset. Even a minor network blip or page reload can cause the entire generation to be lost, forcing the user to re-issue the prompt and wait again. From the developer’s side, this means wasted tokens and potentially double the LLM costs for the same request.

Server infrastructure rarely stores stream state by default

Even when WebSockets or similar protocols are used, many backends treat streaming as fire-and-forget. Once tokens are emitted, they’re not stored. If a client reconnects and asks “what did I miss?”, the server has no answer unless a stateful resume mechanism is in place. That means tracking client progress, buffering streamed output, handling retries, and ensuring correct ordering. None of which are trivial to bolt on after the fact. Building this kind of infrastructure requires careful design, and is one reason robust streaming support remains rare despite user demand.

These three limitations add up to a brittle default behaviour: A typical implementation ends up tying AI generation directly to a single client connection. If that connection drops due to a refresh, crash, or network issue, the stream is lost. Undelivered tokens vanish, and the user is left with an incomplete response and no way to recover. They have to restart the prompt and wait all over again. The result is a jarring, inefficient experience that breaks user trust.

Why you need a drop-in AI transport layer

Making streaming resilient isn’t something you can bolt on later. It needs to be handled at the transport layer, by infrastructure that treats continuity as the default. This isn’t one feature, it’s a set of behaviours working together to keep streams intact even when clients disconnect, reload, or crash. At a minimum, the transport should handle:

Persistent streaming connections

Instead of one-request-per-response, the client should use a persistent connection (e.g. WebSocket) that stays open for streaming. A persistent channel enables realtime, bi-directional, delivery and helps guarantee in-order arrival of tokens.

Unlike stateless HTTP calls, a WebSocket connection can continuously push data to the client without needing to re-establish a new HTTP request for each chunk. This drastically reduces the chance of interruption. And if the connection does break, the protocol can attempt automatic reconnection quickly, avoiding the overhead of starting a whole new session from scratch. In short, a long-lived connection is the foundation for uninterrupted token streams.

Reliable, resumable token streaming: Your AI keeps streaming even after reloads, tab crashes, or network drops. No restarts. No lost context.

Server-side output buffering and replay

A resilient transport layer buffers the AI’s output on the server side as it’s being generated. Every token or chunk is stored (in memory or a fast store) at least until it’s safely delivered. Why? Because if the client disconnects momentarily, those tokens must still be available to send later. Ably’s messaging platform, for instance, persists messages while the client is reconnecting, guaranteeing no data is lost in transit. This buffered backlog enables catch-up: when a client reconnects, the transport can replay any missed tokens from the buffer before returning to live streaming.

The user doesn’t see gaps in the text, because the system fills in everything that was generated while they were away. Without server-side buffering, there’s simply no way to recover what was produced during a disconnect.

Rejoin & instantly hydrate state: When someone comes back, they instantly see the live state of the conversation.

Session tracking across client restarts

To resume a stream, the system needs to know who’s reconnecting and where to pick things up. That means tracking session state across connections. That typically means using a stable session or conversation ID that stays the same even if the page reloads or the device changes.

When the client reconnects, it should tell the server what it last received (for example, “I got up to token 123”). The server then uses that information to send only the tokens the client missed. This handshake (where the client shares its last-seen message ID) is what lets the stream continue cleanly, without starting over.

Platforms like Ably support this by using resume tokens or last-event IDs. The client includes that token on reconnect, and t he stream resumes from exactly the right point. As a result, even after a crash, refresh, or switching devices, the AI response carries on with no manual sync needed.

Multi-device & multi-tab continuity: Start a chat on your laptop, continue on your phone; open three tabs. it all stays in perfect sync.

Ordered delivery guarantees and reconnection state

Maintaining the correct order of tokens is critical. We can’t have jumbled or duplicated text when a stream resumes. The transport layer must guarantee that messages (tokens) are delivered in order, exactly once. This involves assigning each chunk a unique sequence identifier and ensuring the client never processes the same chunk twice even in the face of retries.

Upon reconnection, the system should replay missed tokens in the original order, with none missing and none repeated. Achieving this reliably often means the server needs to temporarily queue messages and only release them in sequence. Ably’s approach, for example, replays any messages the client missed during a disconnection and ensures no gaps or duplicates in the data. In practice, that means when your AI resumes its answer after a drop the user sees a continuous, correctly ordered completion as if no disconnect ever happened.

Exactly-once, in-order delivery: Every message, token, event, or state update arrives once, arrives only once, and arrives in the correct sequence.

All of this points to a deeper architectural issue. To make streaming reliable, you need to rethink where state lives and how streams are managed across connections:

Putting it all together

A drop-in transport layer for AI needs to manage these concerns transparently. It keeps a persistent pipe open, buffers the token stream, tracks session offsets, and enforces ordering and exactly-once semantics. For the developers, this means you don’t have to build custom state management for every AI session — the transport layer provides the assurances that “your AI will keep streaming, no matter what.” Essentially, it’s infrastructure that transforms the unreliable web into a dependable conduit for AI data.

How the experience maps to the transport layer

To better illustrate, here’s how specific user expectations translate into transport-layer requirements and technical implementations:

Answer continues after page refresh : The AI’s response resumes exactly where it left off when the user reloads the page, with no repetition.

Stream resumption on reconnect

Connection recovery using a resume token or last message ID (e.g. sending an SSE Last-Event-ID or a WebSocket reconnection handshake) so the server knows what data to replay.

Delivering reliable, resumable streaming today

Reliable, resumable token streaming isn’t theoretical anymore. You can ship it now. You don’t need to redesign your whole architecture or stitch together a fragile set of custom reconnection hacks.

Ably AI Transport gives you the infrastructure required to keep AI responses flowing, even when the client drops. Persistent connections, ordered replay, automatic resume, and delivery guarantees come as part of the platform. Your generation process keeps running, and users see the response continue exactly where it left off when they return.

If you’re building Gen-2 AI experiences and want streaming that survives reloads, outages, and device switches, we’d be happy to help.

Reach out for early access or to learn how we can support reliable, resumable streaming in your AI products.

Originally published at https://ably.com.

AI UX: Reliable, resumable token streaming was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

Ably AI Transport is now available

Ably Team — Fri, 20 Feb 2026 14:19:04 GMT

Today we’re launching Ably AI Transport: a drop-in realtime delivery and session layer that sits between agents and devices, so AI experiences stay continuous across refreshes, reconnects, and device switches — without an architecture rewrite.

The gap: HTTP streaming breaks down for stateful AI UX

AI has moved from “type and wait” requests to experiences that are long-running and stateful: responses stream, users steer mid-flight, and work needs to carry across tabs and devices. That shift changes what “working” means in production. It’s not just whether the model can generate tokens, it’s whether the experience stays continuous when real users behave like real users do.

Most AI apps still start with a connection-oriented setup: the client opens a streaming connection (SSE, fetch streaming, sometimes WebSockets), the agent generates tokens, and the UI renders them as they arrive. It’s low friction and demos well.

But HTTP streaming really solves only the first part of the problem, and it’s not a good place to end.

First: continuity. When output is tied to a specific connection, the experience becomes fragile by default. Refreshes, network changes, backgrounding, multiple tabs, device switches, agent handovers (even agent crashes) are normal behaviour. And they’re exactly where teams see partial output, missing tokens, duplicated messages, drifting state, and “start again” recovery paths. That’s where user trust gets lost.

Second: capability. A connection-first transport layer doesn’t just make UX fragile. It limits what you can build. Once you want true collaborative patterns like barge-in, live steering, copilot-style bidirectional exchange, multi-agent coordination, or a seamless human takeover with full context, you need more than “a stream.” You need a stateful conversation layer that can support multiple participants, resumable delivery, and shared session state.

So teams patch it: buffering, replay, offsets, reconnection logic, session IDs, routing rules for interrupts and tool results, multi-subscriber consistency, and observability once production incidents start. It’s critical work — but it’s not differentiation.

What Ably AI Transport does

AI Transport gives each AI conversation a durable bi-directional session that isn’t tied to one tab, connection or agent. Agents publish output into a session channel, clients subscribe from any device, and Ably handles the delivery guarantees you’d otherwise rebuild yourself: ordered delivery, recovery after reconnects, and fan-out to multiple subscribers.

It’s deliberately model and framework-agnostic. You keep your agent runtime and orchestration. AI Transport handles the delivery and session layer underneath.

The key shift: sessions become channels

In a connection-oriented setup, the “session” effectively lives inside the streaming pipe. When the pipe breaks, continuity becomes a headache.

With AI Transport, the session is created once and represented as a durable channel. Agents and clients can join independently. Refresh becomes reattach and hydrate. Device switching becomes another subscriber joining the same session. Multi-device behaviour becomes fan-out rather than custom routing. Agents and humans become truly connected over a transport designed for AI bi-directional low latency conversations.

How Ably AI Transport ensures a resilient, stateful AI UX

Resumable, ordered token streaming: A great AI UX depends on durable streaming. Output is treated as session data, so clients can catch up cleanly after refreshes, brief dropouts, and network handoffs.

Multi-device continuity: Conversations are user-scoped, not tab-scoped. Multiple clients can join the same session without split threads, duplication, or drifting state.

Live steering and interruption: Modern AI UX needs control, not just output. Interrupts, redirects, and approvals route through the same bi-directional session fabric as the response stream, so steering works even across reconnects and devices.

Presence-aware sessions: Once agents do real work, wasted compute becomes a serious cost problem. Presence provides a reliable signal for whether the user is currently connected (or fully offline across devices), so you can throttle, defer, or resume work accordingly.

Agents that collaborate and act with awareness: As soon as you have more than one agent (or an agent plus tools/workers), coordination becomes the product. Shared session state and routing prevent clashing replies, duplicated context, and “two brains answering at once,” so multiple agents can communicate directly with users coherently.

Seamless human takeover when it really matters: When an agent hits a boundary (risk, uncertainty, or policy) a human should be able to step in with full context and continue the session immediately. The handoff keeps the same session history and controls, so there’s no repeated questions, no “start again,” and no losing track of what happened mid-flight.

Identity and access control: Beyond toy demos, you need to know who can read, write, steer, or approve actions. Verified identity plus fine-grained permissions let multi-party sessions stay secure without inventing a bespoke access model.

Observability and governance: When AI UX breaks in production, it’s rarely obvious where. Built-in visibility into session delivery and continuity makes failures diagnosable and auditable instead of “black box streaming incidents.”

Concrete examples

Multi-device copilots: A user starts a long-running answer on desktop, switches to mobile mid-response, and the session continues without restarting. Steering and approvals apply to the same session regardless of device.

Long-running agents: A research agent runs multi-step tool work for minutes. If the user disconnects, the work continues; when the user returns, the client hydrates from session history instead of resetting.

Getting started (low friction)

You can get a basic session running in minutes

What to do next

Sign-up for a free developer account and follow our guides on setting up a publisher and subscriber locally to test refresh/reconnect/device-switch behaviour — that’s where you’ll feel the value immediately.

Originally published at https://ably.com.

Ably AI Transport is now available was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

The evolution of realtime AI: The transport layer needed for stateful, steerable AI UX

Ably Team — Fri, 20 Feb 2026 14:18:24 GMT

When we launched Ably in 2016, we set out to solve a fundamental problem: delivering reliable, low-latency real-time experiences at scale. So we set out to build a globally distributed system that didn’t force developers to choose between latency, integrity, and reliability — trade-offs that had defined the realtime infrastructure space for years.

Fast forward to today, and we’re reaching 2 billion devices monthly, processing 2 trillion operations for customers who demand rock-solid infrastructure for their mission-critical features. But over the past year, as AI has transformed from a backend optimisation tool into a front-and-centre user experience, we’ve been asking ourselves a critical question: What’s Ably’s role in the AI ecosystem?

From “nice-to-have” to essential infrastructure

A year ago, if you’d asked us about Ably’s AI story, we would have told you that yes, customers were using us. Companies like HubSpot and Intercom were leveraging Ably for token streaming and realtime AI features. But honestly? The value proposition felt incremental. Traditional LLM interactions followed a simple request-response pattern: send a query, stream back tokens, done. HTTP streaming handled this reasonably well, and while Ably offered benefits, there wasn’t a smoking gun reason to use us specifically for AI.

That’s changed dramatically.

The shift to Gen 2 AI experiences

What we’re calling “Gen 2” AI experiences are fundamentally different from what came before. Instead of simply querying a model’s training data, today’s AI agents reason, search the web, call APIs, interact with tools via MCP (Model Context Protocol), and orchestrate complex multi-step workflows. Just look at how Perplexity searches, or how ChatGPT now breaks down complex requests into observable reasoning steps.

This shift introduces an entirely new set of challenges:

Modern AI UX problems

Async by default: When an AI agent needs 30 seconds or a minute to complete a task (not 3 seconds) user behaviour changes. They switch tabs, check their phone, or start other work. A simple HTTP request suddenly needs to handle disconnections, reconnections, and state recovery.

Continuous feedback is mandatory: Users need to know what’s happening. “Searching the web… Analysing documents… Calling your CRM…” This isn’t a nice-to-have anymore. Without feedback, users assume the system has failed.

Multi-threading conversations: Imagine asking a support agent about your order status. While they’re checking, you ask another question. Now you have two parallel operations that need coordination. The agent needs to know what else is in flight and potentially prioritise or sequence responses intelligently.

Cross-device continuity: Users want to be able to set tasks running and then pick-up later from where they left off. They may start a deep research query on their laptop, close it, and then want to check progress on their phone an hour later. The entire conversation state needs to seamlessly transfer.

The transport layer modern AI needs

Our vision for addressing these challenges centres on what we’re calling the Ably AI Transport. A drop-in solution that handles the complexity of making the AI UX resilient and multi-device — so developers can focus on building great agent experiences, not wrestling with networking problems.

We focus on everything between your AI agents and end-user devices, leaving orchestration, LLM selection, and business logic where they belong — in your control.

Layer 1: The foundation

At the base level, the AI Transport provides what you’d expect from Ably: bulletproof reliability, multi-device synchronisation, and automatic resume capabilities. But the real shift is architectural. Instead of your agent responding directly to requests, it returns a conversation ID. Devices subscribe to that conversation, and from that point forward, the agent pushes updates through Ably.

This simple change unlocks powerful capabilities:

Decoupling: Agents and devices can disconnect and reconnect independently without losing continuity
Bidirectional control: Need to stop an agent mid-task or ask a follow-up question? There’s a direct communication channel that doesn’t require complex routing infrastructure
State recovery: Reconnecting devices don’t replay every token. They get current state and resume from there

Layer 2: Richer orchestration

The next layer introduces shared state on channels using live shared state. This enables sophisticated coordination:

Agent presence and status: Devices know if agents are active, thinking, or have crashed. Agents can broadcast their current focus (“Analyzing Q4 data…”) as state rather than events.
Multi-agent coordination: When multiple agents work simultaneously — say, one handling a technical query while another processes a billing question — they can see each other’s state and coordinate without stepping on each other’s work.
Context-aware prioritisation: Agents can see if a user is actively waiting versus having backgrounded their session, enabling smarter resource allocation.
Client-side tool calls: In co-pilot scenarios, agents can query the client directly about user context “Is the user currently editing this field?” without roundtripping through backend systems.

Layer 3: Enterprise-grade observability

Because everything flows through Ably, you gain comprehensive visibility into the last mile of your AI experience. Stream observability into your existing systems, integrate with Kafka for audit trails, and leverage enterprise features like SSO and SOC 2 compliance that come standard with Ably’s infrastructure.

A new way of building

What excites us most is the idea of providing a stateful conversation layer that removes infrastructure concerns from the developer’s plate. Think of it as abstract storage for conversation state, combined with the realtime capabilities developers need for modern AI UX.

The developers building these experiences don’t want to solve networking problems. They want to focus on prompts, orchestration, RAG pipelines, and agent logic. The transport layer shouldn’t be where they spend their time — but it will become critical as user expectations evolve to match what ChatGPT, Perplexity, and Claude are demonstrating daily.

Framework-agnostic by design

One pattern we’ve noticed: as engineering teams mature in their AI journey, they tend to move away from monolithic frameworks and build custom orchestration logic. This makes sense — these systems become core to their business differentiation.

That’s why the Ably AI Transport is deliberately framework-agnostic. Yes, we’re building drop-in integrations with OpenAI’s agent framework, LangChain, LangGraph, Vercel AI SDK, ag-ui and others to make getting started trivially easy. But the architecture doesn’t lock you in. Swap out your orchestration layer, change LLM providers, rebuild your agent logic — your transport layer and device communication remain consistent.

The road ahead

If you’re building AI experiences and wrestling with questions like “How do I handle interruptions?”, “What happens when users switch devices mid-conversation?”, or “How do I coordinate multiple parallel agent tasks?” — we’d love to talk. We’re convinced there’s a better way to build these experiences, and it starts with not having to rebuild the real-time infrastructure layer from scratch.

The plumbing shouldn’t be your problem. Building great AI experiences should be.

Originally published at https://ably.com.

The evolution of realtime AI: The transport layer needed for stateful, steerable AI UX was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

The new Ably dashboard: realtime visibility in your hands

Ably Team — Fri, 20 Feb 2026 14:17:48 GMT

We’ve rebuilt the Ably dashboard to give developers clear, realtime visibility into how their applications behave.

This isn’t just a cosmetic refresh. It’s a shift from a configuration-first dashboard to a live observability surface. One that lets you see channels, connections, messages, and errors as they happen, debug issues instantly, and understand usage without stitching together logs and tooling.

Why we rebuilt the dashboard

The previous dashboard did a solid job in helping you manage your apps, API keys, and configurations. But it didn’t show what was actually happening inside your realtime system. When something broke or behaved unexpectedly, you were left piecing together clues from SDK logs, APIs, and external tools. There wasn’t a single place to answer operational questions like who’s connected right now, what’s happening on a particular channel, or whether a pattern of errors is brand new or recurring.

The new dashboard brings realtime observability directly into the browser. No setup, no extra tooling, no context switching — just a live window into Ably.

What’s new (and why it matters)

At the heart of this release are four capability upgrades that change how your team operates realtime systems on Ably. Each one is useful on its own; together, they make it far easier to understand behavior in your apps, debug faster, and understand what’s driving usage.

Channel and connection inspectors

The new inspectors provide realtime visibility into how your system behaves as data flows through Ably.

The channel inspector shows you who’s attached to the channel, which messages are being sent, and what’s happening in the presence set. You can also see which rules and integrations are active on that channel. Alongside that live activity, it surfaces realtime metrics like message rates, occupancy, and connection counts, so you can see performance as it changes — not after the fact.

The connection inspector enables you to see who’s connected to your app. You can choose a specific connection and see which channels it’s attached to and view live statistics such as the rate of messages being published by that connection. Additionally, you can see information such as the geographical location of the connection and the SDK it’s connected to Ably with. Combined, the inspectors put realtime visibility into who’s interacting with your app, and which channels they’re interacting with. You now have the ability to debug any issues far more easily, such as ‘why is this channel still active?’

Log search

Live streams are great for what’s happening right now, but many investigations start with a timestamp. Log search lets you query historical platform events so you can understand past behavior, trace what happened and understand why it occurred. You can then compare today’s traffic with last week’s. It’s ideal for debugging anomalies and spotting patterns, especially when you’re trying to answer whether a problem is new or recurring.

Reports and analytics

The new reports section gives you aggregated visibility into how your apps are used over time — message volumes, connection and channel durations — so you can understand where consumption is coming from and what’s driving traffic. This is particularly helpful when needing to explain usage internally, plan scaling work, or map realtime costs back to product features.

This is the foundation for deeper analytics arriving in future releases, including more granular breakdowns by product, SDK and device, plus finer‑grained views by app, channel, and namespace.

Web CLI

We’ve also introduced a new web CLI. A browser-based command line that lets you run Ably commands instantly. You can publish and subscribe to messages, enter presence, and manage your app configuration without any local setup. It complements the redesigned dashboard to give you a fast way to interact with Ably from anywhere. The web CLI is a powerful tool that’s invaluable for exploring Ably features without requiring you to write any code. It is especially useful during support calls where you need to quickly reproduce a certain behavior or send a specific set of messages.

Logs tail and live telemetry

Each inspector includes a live, realtime log stream scoped to the resource you’re viewing. If you’re inspecting a channel, you see the events relevant to that channel; if you’re inspecting a connection, you see the events relevant to that connection. This means you can trace behaviour as it happens, correlate spikes in live metrics with specific platform events, and debug instantly rather than collecting evidence after the incident.

A more modern, product-first dashboard

Alongside these new capabilities, we’ve modernized the dashboard itself. Navigation is cleaner and faster, with dedicated sections for each Ably product. The result is a more intuitive experience that helps teams get to the right tools quicker; whether they’re debugging, trying out a new product, testing new features, or monitoring live traffic.

What’s coming next

This release is a major step toward making the dashboard a full observability layer for Ably. We’re working towards live logs: a global realtime stream of platform events across all resources to complement the existing log search functionality — so you can see what is happening live, and what has happened previously. We’re also continuing to expand the reports section to provide richer visualization of your usage, performance, and reliability, across all your apps.

In parallel, we’re continuing to modernize the remaining areas of the dashboard so that all your resources benefit from enhanced observability and analytics.

Originally published at https://ably.com.

The new Ably dashboard: realtime visibility in your hands was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

Gen‑2 AI UX: Conversations that stay in sync across every device

Ably Team — Fri, 20 Feb 2026 14:17:03 GMT

Start a conversation on your laptop, finish it on your phone. The context just follows you. That’s what cross-device AI sync delivers. No reloading history, no reintroducing yourself, just one continuous thread across every screen. It builds trust, reduces friction, and makes the assistant feel like a single, persistent presence. This post unpacks why users expect it, what makes it technically tricky, and what your system needs to get it right.

AI conversations must survive the device switch

Modern users have grown to expect realtime, seamless experiences from their apps and AI tools. They want instantaneous responses, continuous interactions, and no interruptions as they move between devices. This expectation extends to AI-powered experiences: if you start a conversation with an AI assistant on your laptop, you should be able to pick it up on your phone or another tab without missing a beat. Equally, if you have initiated a long-running asynchronous task you want to be notified once it’s completed, no matter which device you’re using at the time.

What users want, and why this enhances the experience

Users want continuous, identity-aware AI conversations that follow them across devices. In practice, this means an AI chat session linked to their identity rather than a single tab or device. The conversation should feel like a single thread they can return to at any time.

That continuity builds trust. The AI isn’t “forgetting” just because you switched devices. A remembered history signals reliability and intent, helping users feel the assistant is genuinely useful. Multi-turn conversations flow naturally, and users avoid repeating themselves or reconstructing context.

This matters even more once AI systems move beyond simple chat. When an LLM is running long-lived, asynchronous work such as multi-step research, tool calls, or background analysis, users expect to see progress and results wherever they happen to be at the time. You might start a task on your desktop, step away while the model works, and then pick up your tablet to see the output appear as soon as it’s ready.

Real-world usage makes multi-device continuity unavoidable. These moments must be frictionless and reinforce the sense that the AI is persistent, reliable, and working on your behalf rather than being tied to a fragile client session.

Why this is proving challenging

HTTP is fundamentally stateless. Each request stands alone, meaning conversation context has to be manually preserved and restored for both the client and the server. This makes cross-device AI session continuity complex.

Having clients poll for updates is inefficient and adds latency. Long-polling or server-sent events help, but only partially. They don’t enable simultaneous bi-directional, low latency messaging, which is what smooth AI conversations require.

Handling reconnections, preserving message order, and managing updates across multiple active clients requires considerable infrastructure. Doing this reliably, at scale, across networks and devices, is beyond what the typical product team can or should build from scratch.

Why you need a drop-in AI transport

Given the challenges described, building a robust system for realtime synchronisation from scratch can significantly drain engineering resources and slow product velocity. This is where a drop-in AI transport layer becomes essential.

Persistent, bi-directional messaging

To support conversations that stay in sync across devices, such a layer must offer persistent, bi-directional messaging using protocols like WebSockets for AI streaming. This allows for continuous, low-latency communication in both directions, enabling the AI to push updates and the client to send input without waiting for discrete request/response cycles.

Persistent, bi-directional messaging: Low-latency, always-on two-way messaging.

Identity aware fan-out

Equally important is identity-aware fan-out. The transport system needs to recognize all active sessions associated with a single user and ensure that every message or state update is sent to all of those endpoints. That means when a user sends a message on one device, every other device they’re signed in on should immediately reflect the change.

Identity aware fan-out: Sync updates across all a user’s devices.

Ordering and session recovery

The system also needs to preserve message ordering and support reliable session recovery. If the connection drops momentarily, say from a device switch or network disruption, the user shouldn’t lose messages or see them out of order. A well-designed transport layer offers mechanisms to replay missed events and keep message sequences intact, ensuring consistency in the conversation.

Keep messages ordered, recover after drops.

Presence tracking

This enables the backend to know which devices are currently online and active. It helps coordinate updates, prevents redundant notifications, and can be used to power features like realtime indicators for typing or collaborative editing across devices.

Streaming support

To maintain a high-quality conversational UX, the transport layer must support LLM token streaming. This includes delivering partial, realtime updates from the AI model as it generates responses. That stream must arrive quickly, in order, and appear simultaneously on any active device.

Stream tokens fast, in order, everywhere.

Ably’s infrastructure supports all of these capabilities as part of its realtime platform. It eliminates the need to custom-build low-level transport solutions, allowing engineering teams to focus on building intelligent, agentic features instead of protocol logic.

How the desired experience maps to the transport layer

The table below breaks down what users expect from cross-device AI conversations, what your transport layer must support to deliver those experiences, and the technical mechanics that make it all work.

Cross device AI? You can ship it now.

Seamless cross-device conversations aren’t futuristic — they’re achievable today. You don’t need to rebuild your stack to make it happen.

Ably AI Transport provides all you need to support persistent, identity-aware, streaming AI experiences across multiple clients. If you’re working on Gen‑2 AI products and want to get this right, we’d love to talk.

Originally published at http://ably.com.

Gen‑2 AI UX: Conversations that stay in sync across every device was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.

How we built an AI-first culture at Ably

Ably Team — Fri, 20 Feb 2026 14:16:20 GMT

Most companies talk about being “AI-first.” At Ably, we decided to actually become one. We build realtime infrastructure for AI applications. To do that credibly, we need to live and breathe AI ourselves — not just in our product, but in how we work every day.

A year ago, we began a company-wide push for AI adoption. This post breaks down how we did it: the pillars, the tooling, the MCP advantage, the early mistakes, the wins across engineering, marketing, sales, and finance, and the cultural momentum that turned a mandate into a mindset.

Building an AI-first company culture

When Jamie Newcomb, Ably’s Head of Product Operations, began championing internal AI adoption, the approach was straightforward: everyone at Ably should explore how AI could make them more effective. No exceptions.

“We might want to tone down the language,” Jamie admits with a laugh, “but it really is mandated. Everyone at Ably should be using AI to see how they can make themselves more effective. But it’s not just about doing things faster. It’s about doing things you couldn’t do before. The goal is to shift the mindset, where people stop asking ‘can AI help with this?’ and start assuming it can, then push further: what’s now possible that wasn’t?”

Today, that mandate has evolved into something far more organic. A company-wide culture where AI isn’t just accepted, it’s expected.

For a company processing 2 trillion operations monthly, this isn’t about following trends, it’s about credibility. It’s about walking the walk. To build AI Transport that developers can trust for agentic workloads, we need firsthand experience of how AI performs in real operational environments, both the advantages and the pitfalls.

Three pillars of successful AI adoption

Ably’s approach to AI rests on three interconnected pillars:

Internal AI adoption and enablement: Integrating AI into workflows and processes across every team to enhance capabilities and drive productivity improvements. The goal isn’t just providing tools, it’s automating repetitive, time-consuming tasks so people can focus on strategic thinking and creative problem-solving.
AI developer experience: Using AI to make Ably’s platform more discoverable and easier to use for developers. This means AI-enhanced documentation, intelligent tooling, and optimized SDK experiences, empowering developers to build real-time products faster with the help of LLMs. The goal is to position Ably as essential infrastructure for real-time user experiences powered by AI.
AI product enhancement: Making proactive, explicit efforts to understand AI use cases where Ably delivers value, determining what we need to enable those use cases, and ensuring those capabilities are part of our roadmap. This pillar is about building infrastructure informed by real customer needs, both known and yet to be discovered.

“My main role is about process efficiency in product engineering,” Jamie explains. “And that naturally extended to AI adoption. We believe there are significant productivity improvements we can make if everyone adopts AI thoughtfully across the company.”

These pillars aren’t separate initiatives, they’re a unified strategy. Internal productivity adoption teaches us what works in practice. Developer experience ensures we’re making Ably discoverable and easy to use for the growing number of developers building with AI. And AI product enhancement ensures we’re building infrastructure informed by real customer needs, not just theory. This article focuses primarily on the first pillar, but the three are deeply connected. What we learn from using AI internally shapes how we build for developers using AI externally.

The MCP advantage

Perhaps the most significant internal development has been Ably’s adoption of the Model Context Protocol (MCP), built over the summer of 2025.

“The Ably MCP connects all our internal tools together,” Jamie explains. “It lets people access data across systems via AI assistants. Building this and seeing it genuinely change how people work has been incredibly rewarding.”

What started as an experiment to see what was possible has grown into a company-wide platform that’s now critical to daily workflows, integrating 15+ services through over 140 tools. Engineers can check CI build status and debug workflow failures without leaving their conversation. Product managers search across Jira issues, GitHub PRs, and Slack threads in a single query. Sales teams pull Gong call transcripts and HubSpot contact history to prepare for customer meetings. The breadth is significant: GitHub, Jira, Confluence, Slack, HubSpot, Gong, Jellyfish, Metabase, PagerDuty, GSuite and more, all accessible through natural conversation.

Before MCP, every AI interaction started from zero, engineers manually explaining Ably’s infrastructure, marketers pasting in brand guidelines, constant context-switching that made AI feel like more work rather than less.

Now when an Ably employee opens Claude, they’re not starting from scratch. Through MCP, they have immediate access to:

Shared context and prompt library
Company knowledge and documentation
Ably’s tone of voice guidelines and style guides
Live data from internal tools and systems

Instead of “Here’s what Ably is, here’s our tone of voice, now help me write this email,” it becomes simply: “Help me write this customer email about latency improvements.” The AI already knows.

Scaling to 140+ tools created its own challenge: context limits. Ably solved this with a tool registry that lets the AI discover only what it needs for each task, keeping interactions lean and responsive.

“That context library is really important,” Jamie emphasises. “The prompts for critical workflows (like our ICP matching) are all version controlled. When something needs adjusting, it’s not about AI being wrong. It’s about iterating on what you’re asking the AI to do.”

The platform continues to evolve based on team feedback. When engineers noticed they were dropping out the terminal to check GitHub Actions builds, new workflow tools were shipped within hours. Claude Code is used heavily to maintain and extend the MCP itself, with Claude’s Agent SDK integrated throughout the development workflow. Using AI to build AI tooling is a big part of why the velocity is so high. That responsiveness, treating internal AI tooling as a living product rather than a one-off project, reflects how deeply AI has become embedded in Ably’s operating culture.

Jamie spoke at length with Jellyfish on how Ably moved beyond data retrieval to unlock real analysis and insights through MCP, and you can read the full article here.

AI tool selection

When Ably first encouraged company-wide AI adoption, the approach was deliberately open-ended. People experimented with ChatGPT, Claude, and workflow orchestration tools like N8N, Zapier, and Relay.

“We’ve settled on Claude for our primary AI, particularly Claude Code for engineers, but people have the freedom to use whatever works best for them,” Jamie says. “If someone has a strong case for a different tool, that’s fine. We’re not prescriptive about it.”

Everyone at Ably has access to Claude for day-to-day work, whether that’s drafting documents, thinking through problems, or exploring ideas. For workflow automation, Relay emerged as the orchestration layer, handling the multi-stage pipelines that power lead enrichment, ICP scoring, and sales alerts. The combination of Claude for reasoning and Relay for orchestration has become Ably’s default stack, though teams remain free to experiment.

This flexibility matters, especially given Ably’s positioning around AI Transport. “We can’t just say ‘use Claude’ when we’re building infrastructure that works with any LLM provider,” Jamie notes. “We need to show that our approach works regardless of which AI you’re using.”

Results by team

Engineering

All engineers now use Claude Code for agentic coding, but the workflows vary based on the task.

For narrow, well-defined tickets, Claude can often one-shot a solution. Engineers point it at the relevant files, describe what they want, and use test-driven development as a guardrail. Claude writes the test first, sees it fail, writes the implementation, and confirms the test passes. For larger tasks, the approach is more iterative: Claude generates a plan as a markdown file, the engineer reviews and refines it, then kicks off implementation in a fresh context with the plan as input.

Discovery is another common use case. Engineers ask Claude questions about the codebase, “where does X get used?”, “how does a message get from acceptance to being broadcast out to clients?”, using it as a way to navigate complex systems without reading through thousands of lines of code.

The Ably MCP bridges the gap between documentation and code. Engineers pull context from Confluence docs, have Claude synthesise summaries, and feed those into coding sessions, turning scattered documentation into usable implementation context. Some are experimenting with Claude Code running asynchronously in the browser, queuing up tasks from a phone and reviewing the work later.

Beyond individual workflows, Claude is integrated into the development pipeline itself. Claude’s Agent SDK is connected to GitHub to generate implementation context, review PRs, and fix CI issues before code reaches production. When a PR goes up, AI reviews it for obvious issues first, then engineers review it as they would any other colleague’s work.

One principle remains constant: a single human author owns every PR, regardless of how much was AI-generated. The practice of engineering judgment, knowing what to accept, what to push back on, and what to rewrite, is still the job.

Marketing

The Marketing team wanted to spend more time shaping the narrative and shipping campaigns, not doing repetitive admin. There are always multiple activities in flight, each needing planning, research, execution, reporting, and analysis. That’s where AI has been a huge productivity lever: the team has adopted it to streamline the “admin” layer so they can increase both output and quality without adding headcount.

Today, the team uses a small stack of AI tools across the lifecycle. They analyse Gong calls to accelerate market research and tighten messaging and positioning. They use Claude to pull and synthesise data from multiple sources to scope and validate content opportunities faster. They also automate lead validation and categorisation for sales follow-up, enriching contact and company data so the first human touch starts from context, not guesswork. And they map the customer journey with attribution, using AI to connect what prospects do pre-signup to intent signals, so they can prioritise the right audiences and double down on what’s actually working.

For example lead qualification that used to take hours, is now a multi-stage AI pipeline that runs automatically on every signup. The system researches companies across 6+ data sources (Crunchbase, LinkedIn, SEC filings, PitchBook), extracts structured data, scores against 8 ICP criteria, classifies personas, and routes alerts to Slack with tier assignments and recommended actions — all before anyone on the team sees the lead.

“Marketing used to spend considerable time on this,” Jamie recalls. “Now the first time they see a lead, it already has a confidence-scored ICP assessment, enriched company data, and suggested next steps.”

Sales

New lead assignment uses multi-signal analysis (employee count, funding raised, revenue for public companies) to automatically route accounts to Commercial, Enterprise, or Strategic segments. For qualified leads, AI generates personalised email sequences based on the ICP analysis, tailoring messaging to the prospect’s industry, technical challenges, and relevant customer references.

For existing customers, AI monitors self-service accounts against usage limits, surfacing expansion opportunities when customers approach thresholds and flagging critical capacity alerts that need immediate outreach. Relay handles the orchestration across all workflows.

Finance

Finance operations at Ably are treated like a tech product, using AI as a force multiplier to engineer away repetitive work.

The team systematically verifies contracts, builds smarter revenue models, and automates reconciliation work. A recent hackathon project eliminates thousands of monthly clicks in the Stripe-to-Xero process, the kind of repetitive work that most finance teams wouldn’t know where to start automating.

They use Ably’s MCP to retrieve data from Xero, then create and update sheets directly through Claude, turning what would be manual exports and data entry into conversational requests. It’s a small example of how the platform extends beyond engineering and into every corner of the business.

The cultural shift

Creating an AI-first culture isn’t just about providing tools — it’s about enablement, support, and honest assessment of where you are versus where you’re headed.

We run AI drop-in sessions every Friday where team members can bring questions, share what they’ve built, or explore new ideas. An internal Slack channel serves as a continuous stream of AI experiments, wins, and collaborative problem-solving.

“When Charlotte [Delivery Manager] and I approach teams, we don’t even talk about AI initially,” Jamie reveals. “We ask: what are your repetitive processes? Once teams understand their processes, then you can start the AI conversation.”

“Anyone can build something now,” Jamie says. “The barrier to solving a problem has basically been removed because people can use AI to build the solution themselves.”

The result is what Jamie calls the “wow moment”: when someone successfully builds their first AI-powered solution, a ceiling lifts. “Once people have that moment, they just keep building.”

What’s next

But Jamie is candid about where Ably still has room to grow. “To be completely honest, we haven’t hit our potential yet,” he admits. “We’ve made real progress, but there’s still more impact to unlock from AI across how we work, our processes, and how we achieve our product outcomes. And when we think we’ve got there, there’ll still be more room to grow.”

The vision for what’s next is clear: continuing to integrate AI deeper into how Ably works. The foundations are in place: agentic coding, AI-assisted PR reviews, automated workflows across teams. But the true potential lies in making these the default across every function, not just the teams that adopted early.

“The biggest gains come from how people think, not just what tools they use,” Jamie explains. “When people stop asking ‘can AI help with this?’ and start assuming it can, that’s where the real impact comes from.”

The most significant outcome isn’t any specific tool or workflow, it’s that cultural shift in action. “We don’t have a problem at Ably where people are on the fence about whether AI can help them,” Jamie reflects. “We’ve shown that it can. Now it’s about enablement and encouraging people to identify problems they can solve themselves.”

The same infrastructure philosophy that powers our internal AI adoption powers our AI Transport product. Read how Ably enables reliable, scalable realtime experiences for conversational AI here.

Originally published at https://ably.com.

How we built an AI-first culture at Ably was originally published in Ably: Serious, serverless realtime infrastructure on Medium, where people are continuing the conversation by highlighting and responding to this story.