Will we need UX data engineers?

LLMs are part of the user experience — but how do we design them well?

Press enter or click to view image in full size

I was speaking recently about the balance between humans and AI and user research and design. It was a great topic, a great venue, a great crowd, and a great conversation. One of the things that came up was the role of UX professionals in ensuring that AI products deliver quality experiences to users. I was asked if I felt that the LLM is part of UX.

I most certainly do.

When humans interact with software, we develop and design them for minimal friction, optimal efficiency, and maximum delight. Our goal is to solve users' problems while leaving them better off than if they had solved the problem themselves, and helping them feel a positive emotion when the workflow is complete. Our goal is to make people's lives better.

This goes for generative AI as well. What's interesting here is that the workflows we use to interact with gen AI tools are relatively simple. Most chatbots are only a few elements: a place to enter text, a button to send, and an interface to read the messaging. If it is more sophisticated than that, it may allow for people to speak instead of type, and then there are going to possibly be places where you can see saved conversations. It is highly simplistic because the complexity is not in the workflow, but instead in the outputs of the LLM.

So then, the user experience is transferred to whatever the LLM responds with when a user prompts it.

What does this mean for UX? To start, it means that we need to thoughtfully adhere to simple workflows that won't make it harder for a user to interact with the LLM. This is not a place for us to be overly innovative or create more complexity in the system than is necessary. After that, there is a need for critical and thorough user research to ensure that the outputs of the LLM are effective, usable, and trustworthy. There is a need to ensure there is proper tone to match the expectations of the user based on the context in which both the user and the chatbot exist. There also needs to be a way to get out of situations that are problematic, correct the LLM when necessary, and rebound when something goes wrong.

One of the problems I brought up during our discussion is that so many of the generative AI tools out there are a user interface over the top of an API call to another system. This leaves the bulk of the fine-tuning of the LLM outputs to prompt engineering, and the entire system at the mercy of changes made to the source from which the API call draws. I can only imagine the havoc that the change from GPT 4 to 5 may have wreaked on tools built on top of it.

The best way to address this issue is to utilize private data sets instead of API calls to third parties. This can certainly be a challenge because many companies do not have access to the proper data sets to do their work. However, if they are able to do this, they will have both increased control over how the models are structured, and also can refine outputs more easily. A narrowly drawn LLM trained on a data set focused on nutrition for example, is less likely to pull incorrect information from the internet and can provide sounder and more useful advice to the user. While it may be more work upfront, there is less likelihood for misinformation, bias, and drift when the data set is known and tested against the user persona and their most relevant use cases.

So this begs the question: will the rise of LLMs lead to the need of an additional layer of UX expertise? I'm thinking something like a UX data engineer that is focused on developing data sets that will most effectively meet user needs. While I know many companies do have really excellent data engineers, I do wonder whether or not they are training on things like personas, UX heuristics, and the critical elements of the user experience.

As we continue to see a shake-up in the field, with the flattening of roles and the addition of AI expertise as part of job descriptions, will we also see the inclusion of data and data science as user experience into the equation? I think this will be both an interesting and exciting area to watch.