Exploring UX for AI Agents: Chat Interfaces

At Sequoia’s AI Ascent conference in March, LangChain Blog highlighted three significant limitations for AI agents: planning, UX, and memory. The blog has now embarked on a detailed exploration of these issues, starting with user experience (UX) for agents, particularly focusing on chat interfaces. This in-depth discussion is split into a three-part series, with the first part dedicated to chat, courtesy of insights from Nuno Campos, a founding engineer at LangChain.

Streaming Chat

The “streaming chat” UX has emerged as the most dominant interaction pattern for AI agents. This format, exemplified by ChatGPT, streams an agent’s thoughts and actions in real-time. Despite its apparent simplicity, streaming chat offers several advantages.

Primarily, it facilitates direct interaction with the language model (LLM) through natural language, eliminating barriers between the user and the LLM. This interaction is akin to the early computer terminals, providing low-level and direct access to the underlying system. Over time, more sophisticated UX paradigms may develop, but the low-level access provided by streaming chat is beneficial, especially in the early stages.

Streaming chat also allows users to observe the LLM’s intermediate actions and thought processes, enhancing transparency and understanding. Additionally, it provides a natural interface for correcting and guiding the LLM, leveraging users’ familiarity with iterative conversations.

However, streaming chat has its drawbacks. Existing chat platforms like iMessage and Slack do not natively support streaming chat, making integration challenging. It can also be awkward for longer-running tasks, as users may not want to wait and watch the agent work. Moreover, streaming chat typically requires human initiation, keeping the user in the loop.

Non-streaming Chat

Non-streaming chat, though seemingly outdated, shares many characteristics with streaming chat. It allows direct interaction with the LLM and facilitates natural corrections. The key difference is that responses are received in complete batches, keeping users unaware of ongoing processes.

This opacity requires trust but enables task delegation without micromanagement, as highlighted by Linus Lee. It is also more suitable for longer-running tasks, as users do not expect immediate responses, aligning with established communication norms.

However, non-streaming chat can lead to issues like “double-texting,” where users send new messages before the agent completes its task. Despite this, it is more naturally integrated into existing workflows, as people are accustomed to texting and can easily adapt to texting with AI.

Is There More Than Just Chat?

This blog post is the first of a three-part series, indicating that there are more UX paradigms to explore beyond chat. While chat remains a highly effective UX due to its direct interaction and ease of follow-up questions or corrections, other paradigms may emerge as the field evolves.

In conclusion, both streaming and non-streaming chat offer unique advantages and challenges. Streaming chat provides transparency and immediacy, while non-streaming chat aligns with natural communication patterns and supports longer tasks. As AI agents continue to develop, the UX paradigms for interacting with them will likely expand and diversify.

For more detailed insights, visit the original post on the LangChain Blog.

Image source: Shutterstock