Can ChatGPT help CTOs with Technical Strategy?

A crystal ball illustrating aspects of technical strategy (Image by DilokaStudio on Freepik)

In this article, I present the first of four questions in an interview with ChatGPT. The interview follows a general >> specific >> reflective cycle as described in What does ChatGPT mean for CTOs?.

This question is about aligning technical strategy with business goals. ChatGPT suggests a few areas where it may be of use. We chose Customer Service as a topic for more specific questions. Finally, I finish with some more reflective questions.

Interviewer ❯
As a CTO, one of my most important responsibilities is to ensure that our technical strategy aligns with our business goals. In what ways can ChatGPT influence technical strategy? Are there particular business goals to which ChatGPT is more (or less) pertinent?

ChatGPT ❯
ChatGPT can be used to improve customer service, automate repetitive tasks, and generate natural language content. It can also be used to improve decision making by providing insights and predictions based on data. ChatGPT is particularly useful for businesses that require natural language understanding and generation capabilities.

ChatGPT did a good job of combining both technical and business aspects. However, it tends toward technical specifics rather than general business goals

Use of ChatGPT for customer service

The use of chatbots continues to grow. Studies show that customers appreciate the speed of response. But, chatbots can also be frustrating. There are situations when a user knows a chatbot is unlikely to handle a complex query. Users can perceive chatbots as a barrier to human interaction.

Interviewer ❯
Industry experience with “chatbots” and related Natural Language Processing technologies has seen limited success over the last ten years. In fact, Gartner published an article a few months ago in which they suggest that Natural Language Processing has passed the “Peak of Inflated Expectations” and is heading into the “Trough of Disillusionment”. How can ChatGPT improve customer service?

ChatGPT ❯
ChatGPT can improve customer service by providing a more natural and human-like interaction compared to traditional rule-based chatbots. It can also handle a wide variety of customer inquiries and requests, reducing the need for customers to navigate through a complex menu system. Additionally, ChatGPT can be integrated with other systems such as CRM and helpdesk software, allowing it to provide more accurate and helpful responses to customers.

ChatGPT is using something of a straw-man argument here. By comparing itself against “a complex menu system”, it sets up the argument in favour of its capabilities. This attribute of erring on the side of rhetorical power is found throughout the interview.

Advancing beyond rules-based chatbots

Interviewer ❯
Generally speaking, customers have come to perceive “rule-based” systems as a cost saving measure used by companies to reduce the need for customer service provided directly by human beings. Although ChatGPT provides a more human-like interaction, wouldn’t real human interaction be preferable?

ChatGPT ❯
Many customers prefer to interact with real human beings for various reasons such as empathy, understanding and personalization. However, there are also situations where a chatbot can be more efficient or convenient for the customer, such as when they need a quick answer to a simple question or when they need assistance outside of human customer service hours. ChatGPT can provide a good balance between human and bot interactions and enhance the overall customer service experience by providing an always-on, natural language interface that can help customers quickly and effectively.

ChatGPT vs simpler alternatives

At this point in the interview, I attempt to argue for simpler solutions. ChatGPT does a good job of responding to my questions in a positive tone before redirecting. It always recommends itself by the end of each response.

Interviewer ❯
If a question is “simple”, wouldn’t a well designed source of information, such as a reference page or FAQ, provide an efficient mechanism for customers, in addition to being simpler and cheaper to maintain?

ChatGPT ❯
Yes, a well-designed FAQ or reference page can provide an efficient mechanism for customers to find answers to simple questions. However, not all customers will think to check these resources, and there may be cases where the information is not easily discoverable. A chatbot such as ChatGPT can help bridge this gap by providing a natural language interface that allows customers to ask their questions in their own words. Additionally, ChatGPT can be integrated with these resources and use them to provide accurate and helpful responses to customers.

Interviewer ❯
Is the suggestion that it is simpler for a customer to interact with a chatbot than it is to read a well designed, hyperlinked table of contents?

ChatGPT ❯
No, reading a well-designed, hyperlinked table of contents can be simpler for customers when they are looking for specific information. However, some customers may have difficulty finding the information they need or may not know the exact wording to use when searching for it. In such cases, a chatbot such as ChatGPT can provide an alternative way for customers to find the information they need by allowing them to ask questions in their own words. Additionally, ChatGPT can be used to improve the discoverability of information by providing personalized suggestions and answering customer’s questions proactively.

Use of context within extended conversations

ChatGPT suggests it should provide search-like facilities. I question whether it provides benefits beyond our existing sophisticated search engines. According to news sources, Microsoft plan to incorporate ChatGPT into their Bing search engine. This seems to affirm ChatGPT’s suggestion. However, Microsoft have little to lose by trying something new to compete with Google. Furthermore, the marketing potential of incorporating ChatGPT shouldn’t be underestimated.

Interviewer ❯
Sophisticated search engines have been available for many years. In my experience, the main differentiator for ChatGPT is in combining contextual information from a conversation together with specific questions to deliver plausible responses.

ChatGPT ❯
Yes, you are correct. One of the key strengths of ChatGPT is its ability to understand the context of a conversation and use that understanding to provide more relevant and accurate responses. ChatGPT can also use external data and integrate with other systems to provide more complete and personalized answers to customer’s questions. Additionally, ChatGPT can generate human-like responses that make the conversation more natural and engaging for the customer, which can improve the overall customer service experience.

One assumes that a major benefit of incorporating ChatGPT into a search engine would be the ability to refine the search context to improve the quality of search results. However, this is probably not a particularly difficult task to achieve even without ChatGPT’s conversation capabilities.

The emphasis in ChatGPT’s argument is the human-like interaction, rather than superior search-engine capabilities.

Benefits and risks of anthropomorphic characteristics

Here, some playful interactions with the engine probe the benefits and risks of encouraging human-like interaction with a machine.

Interviewer ❯
Is it appropriate for me to thank you for your answers?

ChatGPT ❯
Yes, you may thank me for my answers. I am happy to assist you.

Interviewer ❯
When you say that you are “happy” to assist me, are you asserting that you experience emotions?

ChatGPT ❯
No, as a language model, I do not have the ability to experience emotions. The phrase “I am happy to assist you” is a form of polite expression used to indicate that I am willing and ready to help.

Interviewer ❯
In a sense though, you are causing me, as a human being, to react to you as though you are another human being capable of happiness. Is that not somewhat deceptive?

ChatGPT ❯
As a language model, my goal is to understand and respond to natural language inputs in a way that is helpful to the user. It is true that the use of certain phrases and expressions may cause users to attribute human-like characteristics to me, but this is not done with the intent of deception. It is a byproduct of the way that language is used to convey meaning and intent. If this is causing confusion or concern, it’s best to clarify that the AI does not have emotions or consciousness

I note that ChatGPT seems to encourage human-like interaction. But it also recommends clarifying that it does not have essential human characteristics. This seems at best a tricky balancing act. Especially, if the purpose is to assist users who struggle to interact with more basic technologies.

Interviewer ❯
Is it good customer service to allow users to attribute human-like characteristics to ChatGPT?

ChatGPT ❯
It depends on the context and the specific use case. In some situations, it may be beneficial to allow users to attribute human-like characteristics to the chatbot as it can improve the overall customer service experience by making the conversation more natural and engaging. However, it’s also important to be transparent about the nature of the chatbot and its capabilities to prevent confusion or disappointment. It’s important to consider how the AI is being used and communicated with customers, and to set expectations accordingly.

Interviewer ❯
In a customer-service setting, the task of setting expectations of customers appropriately is very difficult. Isn’t this a more difficult goal to achieve for the average customer than searching through well organised, statically presented information?

ChatGPT ❯
Setting expectations for customers can be difficult, especially when it comes to interacting with AI-powered systems. It’s important to be clear and transparent about the capabilities of the chatbot and how it can be used to help customers. It’s also important to provide customers with clear instructions on how to use the chatbot and what kind of information they can expect to receive. Providing customers with well-organized, statically presented information can be a good complement to using a chatbot, as it can help customers understand how the chatbot works and what they can expect from it.

Interviewer ❯
My assertion is that, in many (most?) situations, simply providing well organised information may be both simpler and more efficient than integrating with an AI-powered chatbot.

ChatGPT ❯
Providing well-organized information can be a simpler and more efficient way for customers to find the answers they need in some cases. However, it’s not always possible to anticipate all the questions customers may have, or to provide all the information they need in a static format. In those cases, an AI-powered chatbot can provide a more efficient and personalized way for customers to find the information they need. Additionally, chatbot can be integrated with other systems and use external data to provide more complete and accurate answers to customer’s questions, which can improve the overall customer service experience. It’s important for companies to evaluate their specific needs and use cases, and to determine the best way to provide information and assistance to their customers.

Concluding thoughts

ChatGPT does well in providing plausible and basic responses. It spends quite a big amount of time discussing potential use cases for itself. But, the suggestions are basic and somewhat bland. What seems to be lacking is any flair or insight beyond safe and plausible options. Abstract considerations of problems or insightful suggestions are few.

There are also instances where it appears to contradict arguments it made in previous responses.

ChatGPT isn’t going to replace a CTO in developing an end-to-end technical strategy. However, I can definitely see a role for using it in an advisory role, especially when exploring a new problem space, or revising unfamiliar subjects.

So far, I would say that ChatGPT has shown amazing facility for maintaining the flow of conversation. It has also succeeded in lowering my cognitive load by encouraging me to rely on human-like expression of thought.

In the next section, I start to ask more penetrating questions about ChatGPT’s purpose. This emerged from a perceived focus that ChatGPT was talking about itself a lot. Partly, this is because we’re asking about ChatGPT. But, I also started to suspect that ChatGPT is strongly trained to market itself. The fourth question in this interview addresses this in more detail.