MEMBER LOGIN

Should CFOs Use ChatGPT

For years, we’ve been anticipating the arrival of ubiquitous AI. And for years, we’ve seen glimmers of hope both in our personal lives, like when an AI system bested Jeopardy’s grand champs, and in business, like when product searches started actually providing useful suggestions of additional items to buy and good alternatives to the ones in your search. Science has produced some truly amazing applications of AI too, like systems that help radiologists spot abnormalities on lungs X-rays and systems that can run billions of simulations for drug companies seeking cures.

But no use of AI has captured our collective imagination like ChatGPT.

GPT stands for Generative Pre-Trained Transformer, which refers to the language model that the chatbot uses. In short, you ask ChatGPT to answer a question or write something for you, and it does its best to respond based on extensive training with data pulled from across the internet. Fundamentally, it’s built to tell stories in fairly clear and concise English. ChatGPT has also passed numerous exams, both scholastic and professional.

This all raises an obvious question for CFOs: How should you use the technology to your advantage?

We spent some time with ChatGPT, running version 3.5 of its model. It’s said that version 4.0 is a substantial upgrade. However, it’s still working with data from 2021. So if your queries are around evolving issues — say improving supply chains, the impact of rising interest rates, or consumers moving from goods-centric purchases to services-centric over the past 18 months or so — ChatGPT won’t have recent information to use on that movement. It’ll answer your question, but it will base that answer on data older than you’d like to consider.

Even as the model improves, it’s likely that search engines will still give better answers in some instances, particularly to more detailed queries. That’s because ChatGPT doesn’t retain every bit of data it used in its training. Instead, it saved enough to draw sophisticated connections between the patterns it found in its learning, and it reconnects them based on your instructions. So if you ask ChatGPT to summarize a highly discussed topic, it does a good job.

We asked ChatGPT to write 500 words on the primary risks that United States banks are facing. It did pretty well, making a list of seven risk areas including credit, market, operational, liquidity, cybersecurity, compliance, and reputation. Sure, these are quite standard, but the explanations for each were good. We then asked about risks that Silicon Valley Bank (SVB) faces. ChatGPT gave us the same list of risks, with some specifics on SVB’s business model. Conclusion: Asking it for more detail doesn’t always get you more insights.

Finally, we asked whether keeping a company’s money in SVB was riskier than keeping it at Zions Bancorp. ChatGPT noted that Zions has greater total assets (it’s still considered a regional mid sized bank), is older, and more diversified and that SVB’s startup lending is riskier. But it also noted that both banks are subject to the same regulations and FDIC insurance. ChatGPT’s bottom line was “it depends.” The technology essentially admitted it didn’t have enough information to make a call. You’ll find this theme in many of its responses.

So if you’re interested in a refresher on banking risks, asking ChatGPT is great. If you want help evaluating two somewhat similar banks, you’ll be much better off doing your own research. It’s worth noting that ChatGPT completely missed the notion of checking with the ratings institutions like S&P, Moody’s, Fitch, or Morningstar. Generally, the more specific your request, the more likely you won’t be satisfied with ChatGPT’s response. This is very apparent with version 3.5, though specificity may be less of an issue with subsequent versions.

We also wanted to check ChatGPT’s ability to advise on matters important to CFOs. ChatGPT is highly inclined to equivocate in any answer it gives. For example, we asked how much a company should spend on accounts receivable, as a percentage of revenue. The first (very long) paragraph was all equivocation, saying it would require more detail to answer that question.

Fair point. So, we asked what percent of revenue a contract manufacturer in the Midwest with $5 million in annual revenue should spend on accounts receivable. Still equivocation, but we got an answer: 10-15%. This range took us by surprise. So we asked the same question but changed “accounts receivable” to “the finance team.” The answer was 2-4%. Then, we asked “Is accounts receivable part of finance?” The answer was yes. You see the problem.

There’s likely just far too little information on the web about what such companies actually spend on these functions. Would you put your business’s financial information out there for everyone to see? Neither would we. So ChatGPT, always eager to please, came up with an answer using far too little information — and we don’t know exactly which information it considered, because ChatGPT doesn’t reveal its sources.

That anecdote should trouble you in four ways:

  • First, ChatGPT can sound very confident while delivering spurious conclusions, so you have to know enough about a given topic to recognize that one of its answers is wrong. How many of its answers are wrong-ish — close, but not really right enough to be useful?
  • Second, ChatGPT didn’t understand that it had made a mistake even after we walked it through questions that would’ve made any human say, “Oh, yeah. … That doesn’t make sense, does it?”
  • Third, ChatGPT is extremely eager to please. If you ask it to write 250 words on why your alma mater is the best school in the country, it’ll do it. Not the best way for CFOs to get advice.
  • And finally, that lack of sources significantly limits ChatGPT’s usefulness in answering detailed queries.

 

So how should CFOs use ChatGPT?

If you’re facing a problem and not quite sure of your options in solving it, ChatGPT will give you a list — sometimes including a nugget you haven’t thought of.

But more useful is that ChatGPT knows English well, and we’ve found it to be a good editor. For instance, say you’ve thrown together an email for your team on performance benchmarks and the email seems too lengthy. You can ask ChatGPT to shorten it, improve it, make it sound more conversational, and so on.

We’ve polled CFOs about the skills they’d like to improve. The top three are data analytics, financing, and communications. ChatGPT can certainly help with communications. It’s quick and easy to use, and it will at least give you some ideas on how to clarify or simplify your writing or speaking.

ChatGPT amounts to a thought-provoking new tool that will only get better. Newer versions will improve quickly and other sorts of AI will be incredibly helpful to CFOs. But for now, ChatGPT is the quick and helpful editor we need from time to time.

And it’s worth noting that generative AI isn’t the only area of AI research that’s bearing fruit. Along with the scientific endeavors mentioned at the outset, there are many business applications where AI is hard at work – from helping to score sales leads, to identifying risk, to improving user and customer experience and more, AI is finding its way into a myriad of tools as well as standalone applications for business analysts.

About NetSuite

As the first cloud ERP system, NetSuite helps more than 33,000 customers gain the visibility, control and agility to build and run a successful business. With an integrated system that includes ERP, financials, commerce, inventory management, HR, supply chain, CRM and more – NetSuite enables businesses of all sizes and across all industries to work more effectively by automating core processes and providing real-time insights into operational and financial performance.