TL;DR
LLM should be transparent
Focus on one use case
Non-AI stuff is still important
LLM & UX: Why chatbots are not the way to go
The arrival of ChatGPT almost a year ago felt like a leap into the future. It felt like we had a decade worth of technology development all in just one year. It's mind bending how fast we got accustomed to this Jarvis-like technology.
Despite these advancements, the capabilities of Large Language Models (LLMs) come with their own set of challenges. The processing costs are high, and the response times for processing long texts, images, and videos can be very slow, posing obstacles to creating stress-free user experiences with LLMs.
At first glance LLM seems like a magical hammer capable of tackling any problem. But the reality is far more nuanced. LLM comes with many inherent limitations which requires careful consideration in their application. That is why LLM is best viewed as a tool and not a universal solution. We need to be selective on when to use LLMs.
Take, for example, the chat user interface. The impact of ChatGPT has been so profound that many products in the past year have revolved around the concept of "Chat with X". But is chat the best user interface for any application? I disagree.
Think about the simple act of turning off the lights. Would you rather turn the lights off with a physical switch or by asking ChatGPT to turn it off for you?
While natural language interface does lower the barrier for entry it comes with a downside by placing too much freedom and by extension, too much responsibility on the user.
Integration of LLM into products and services effectively requires a deep understanding of both its capabilities and limitations. Here are three key principles that I like to share.
LLM Should Be Transparent
User's do not care whether a product uses LLM or not; what truly matters is that the value the product delivers by solving their problem. We often encounter products where it is developed with head-first-slide-into-AI approach, focusing on incorporating AI from the get go rather than identifying a problem and then considering if LLM could provide a solution. This approach is backwards.
I find the effective use of AI in products often goes unnoticed by the user. It operates behind the scenes, automating mundane tasks to reduce costs and enhance the overall user experience.
It's okay for LLM to be invisible to the user. It's okay that the user is not directly interacting with LLMs. Often times you find it's actually better that way.
Focus on One Use Case
LLM is so versatile it is possible, but impractical and unfavourable to apply them to every conceivable problem. This scattergun approach is problematic for several reasons: LLMs can be slow, resulting in poor user experience; frequent API calls to LLMs can pile up costs; and a broad application makes evaluation difficult.
Having a single strong use case allows for the creation of a specific test dataset and a quantitative evaluation of the model. There are many LLM products in the wild with an unmonitored use of AI where developers lack insight into performance.
Non-AI (LLM) Stuff Is Still Important
The most important point of this blog is to remind you that LLM is just one component of a much larger ecosystem. Successful integration of LLM into products and services requires a solid foundation in traditional software development practices. Reliable infrastructure, robust development processes, and continual improvement are all critical elements that fall outside the realm of AI but are essential for delivering a superior user experience. As LLMs continue to capture our imagination with their potential, it's vital not to overlook the fundamentals of software development and business operations that enable these technologies to truly thrive.
Finally
The opportunities for LLMs, especially in the enterprise domain, are yet to be discovered. If you represent a forward-thinking company interested in leveraging AI/LLM technologies to enhance your business, let's explore collaboration opportunities!
