With all the hype about AI, I initially expected using it to feel like… well, magic. When I encountered wrong answers and occasional responses that made no sense, I felt disappointed.

Now, I see it differently. It’s as if someone handed me a Hogwarts magic wand. A magnificent tool, if you also know the spells to use. In inexperienced hands, though, it can cause chaos.

What changed for me?

Recently, I attended a Stanford course on using AI for Financial Executives.

Today, I’m sharing what I learned.

Dana recently attended a Stanford course on AI. Today, she shares what she learned. Share on X

---

---

AI Tools – Like a Hogwarts Magic Wand

(AKA:  What I learned from attending an AI class at Stanford)

In July, I attended an in-person AI for Financial Executives course at Stanford. This was a dream come true. I’d always wanted to go to Stanford.

Each professor covered a different aspect of AI, but the collective impression I took away was that AI is a fundamentally trajectory-altering technology, comparable to—or perhaps even more significant than—the agricultural or industrial revolution.

Three themes stuck with me:

• AI is a “general-purpose technology,”
• Its usefulness depends on whether a problem is “rough-edged” or “sharp-edged,” and
• It is still in its infancy.

---

General-Purpose Technology

“General-purpose” hints at how many roles AI could eventually play in our lives. Fritz imagined some of those possibilities in his excellent 2023 fictional story,  “The Impact of AI on Retirement.”

In his story, HAL, from the sci-fi film 2001: A Space Odyssey, serves as your constant companion, nurse, social director, investment advisor, and administrative assistant. HAL….

• Books appointments,
• Reminds you to take medications,
• Monitors your sleep,
• Arranges transportation, and
• Tells you when to sell part of a stock position.

My recent experience? I tried a much-touted AI assistant app. It couldn’t book my oil change.

That gap illustrates something I learned at Stanford: a general-purpose tool can require time and effort to make it useful for a specific purpose.

My oil change occurs infrequently. It was faster to call and book it myself than to provide all the information and instructions the AI assistant needed.

But for a recurring business task that could save hundreds of hours a year, that setup effort is well worth it.

When I spend time giving AI tools context, instructions, and feedback, I find them incredibly useful. Understanding this has helped me see how often I used to expect a tool to be an expert out of the gate.

Still, better instructions don’t guarantee a correct answer. AI can confidently tell you something wrong.

That brings me to the second lesson: the difference between rough-edged and sharp-edged problems.

---

Rough-Edged vs. Sharp-Edged Problems

How can a technology with so much potential confidently give you the wrong answer? Even worse, when questioned, it may double down and invent reasons why it is correct.

At Stanford, we explored the distinction between “rough-edged” and “sharp-edged” problems.

Rough-edged problems don’t have a single right answer. Examples include:

• Write an outline
• Draw a picture of a cat
• General research

Sharp-edged problems leave less room for error. Some require an exact answer; others require recommendations that account for specific constraints and personal circumstances. Examples include:

• Bug-free coding
• Math or modeling problems
• Making personalized recommendations

The difference comes down to the task’s tolerance for error. If AI writes a blog post outline, I may use a few ideas and scrap the rest. But if I ask it to calculate an internal rate of return and the answer is incorrect, I lose confidence in using it for similar tasks.

Writing allows room for variation. Math requires the right answer. Personalized recommendations present another challenge: they require nuance, context, and an understanding of human values.

Many finance- and retirement-related questions fall closer to the sharp-edged category.

AI generates responses by predicting patterns. For a calculation, though, a plausible-looking answer isn’t enough. You need to know that the numbers are correct.

That’s why you—or the tool you’re using—must build in accuracy checks with “automatically verifiable outcomes.” The process should check the result against a defined requirement before moving to the next step.

What happens when the checks fail?

---

When AI Gets Financial Answers Wrong

A Facebook forum for a popular do-it-yourself financial planning program offers two examples.

One user reported the AI recommended delaying Social Security until age 73. Fortunately, the user knew there is never a reason to wait past 70.

Another user reported that the AI modeled a reverse mortgage that allowed them to borrow substantially more than the equity in their home. Again, the user knew enough to recognize the math wasn’t right.

Any accuracy checks intended to prevent those errors were ineffective in those instances. The users caught the mistakes. But what happens when someone doesn’t know enough to question the answer? That concerns me.

As developers improve these tools and build in verification steps, I expect better results. Yet the Stanford professors cautioned that even as accuracy improves, we should never expect AI to be 100% accurate on sharp-edged problems.

That was one of the most surprising things I heard during the course. It changed how I use AI—and what I expect from it.

---

Where AI Helps Me—and Where I Question It

I’ve learned to treat my AI tools as a very capable intern. With patience, feedback, guidance, corrections, and questions, they become valuable contributors to my work.

I’m now more patient. I question answers, verify results, and push back when I know something is off.

What excites me most is AI’s ability to streamline tedious tasks, such as organizing information and moving data between systems. That work isn’t the best use of my skill set. Automating some of it feels like real magic.

I can find information faster, build better workflows, and create higher-quality work (sometimes, but not always, in less time). There is no question: AI is improving the quality of my work and my team’s.

---

Where I Still Need Human Judgment

If I need to find vendors for a specific task, I give AI my criteria. It quickly develops a shortlist that I can investigate.

But when I ask it to analyze our company’s financial statements and recommend how to meet a strategic objective, I find its recommendations are often disconnected from the realities of our business.

Those decisions require years of knowledge about our firm, staff, clientele, and values. No matter how much data I provide, the recommendations often lack “heart”: the human element that shapes how a values-based organization—or an upcoming retiree—makes decisions.

---

What About Your Retirement Plan?

Some clients have begun forwarding us their financial conversations with AI.

One client uploaded their portfolio allocation and asked AI to analyze it, then sent me the response for my thoughts. I appreciated that they questioned its recommendations.

AI did a great job organizing the allocation data. It did a terrible job recommending changes.

It didn’t understand the portfolio model we use, consider capital gains taxes, or account for the different investment choices available in a 401(k) versus a brokerage account.

Organizing the information was useful. Making recommendations required context it hadn’t accounted for.

These experiences have helped me identify a few practical ways to use AI more successfully.

---

Four Keys to Using AI Successfully

Companies and research institutions can build specialized AI environments that most of us can’t.

At Stanford, we heard about a lab where AI agents learned medical specialties and passed competency exams before joining other specialty agents in a human-monitored research environment. One lesson stood out: assigning an agent to serve as a critic was essential to better outcomes.

Most of us use off-the-shelf tools rather than specialized research environments. For better outcomes, along with playing the critic role, we must be deliberate about how we use the tools.

Here are four practices that guide my use of AI.

1. Understand the Privacy Settings

I use only paid AI tools, which may offer more privacy options, and I always turn off settings that allow them to train on my data.

Payment alone doesn’t establish privacy protection. Check the specific tool’s policies and settings, including how it stores your information and what access you grant when connecting apps.

2. Be the Critic

Don’t assume AI is an expert. Ask for sources, open them, and verify that they support the answer.

Give it specific guidelines for reviewing its work. For calculations, require a check you can verify independently. A confident explanation is not proof that an answer is correct.

3. Use It to Organize Information

One client uploaded expense data from Monarch (no affiliation), a popular personal finance app, and asked AI to help organize his total spending into a monthly average in preparation for retirement.

That’s a useful application: organizing transactions and identifying spending patterns. For accuracy, still check that the totals reconcile to the original records, and review how it handles transfers, duplicate entries, and unusual expenses.

Use that organized information as an input to your retirement plan. Don’t assume it establishes how much you can safely spend or captures every future spending need.

4. Use It to Enhance Your Learning

Our in-house tax expert has been delighted by how AI has accelerated his learning. Meanwhile, I’ve received incorrect tax information from it several times.

His delight reflects his expertise. He recognizes when an answer doesn’t sound right, asks for IRS sources, and verifies them. He uses the process like an advanced course to expand his knowledge.

You can use AI to explain unfamiliar concepts and develop better questions. Just remember that when you’re learning a new subject, you may also have a harder time spotting its mistakes.

---

Conclusion: Learning the Spells

One day, I can imagine many aspects of Fritz’s fictional story becoming reality. AI could help manage more of my daily life and handle tedious business tasks, freeing up time for the work and people I care about.

That potential excites me. But today, using AI successfully means understanding its limitations, providing guidance, and checking its work. When your retirement decisions are involved, those checks matter.

I started out expecting magic. Now, I’m learning the spells—and realizing that, as with most things in life, it’s up to me to make the magic happen.

What about you? How are you using AI? What has been useful, and where have you found concerns? We’d love to hear your experiences in the comments below.

P.S. In the comments, I’ll post a summary of how I use AI with my writing.