Failure: Auditor General finds CRA's 'Charlie' chatbot answered only 2 of 6 tax questions correctly
Canada Revenue Agency · Ottawa, ON · Government · Roles: Customer Support, Tax
Failure / lessonTaxCustomer SupportFailure
What was done
As part of a 2025 audit of CRA contact centres, the Auditor General tested CRA's public chatbot, Charlie, on general tax questions.
Outcome
Charlie answered 2 of 6 general tax questions accurately, while other public AI tools tested got 5 of 6. The audit also found low accuracy among phone agents. The takeaway for Canadian businesses is to double-check tax answers from any bot against CRA guidance.
Evidence: Auditor General report / news. Figures are as reported by the source and have not been independently audited by AgentsNorth.
Agents & tools used
Prompt Reconstructed by AgentsNorth
The source didn't publish its prompt. This is our reconstruction of a starting prompt for a similar workflow, not the company's actual prompt.
Answer this Canadian tax question, then list the specific CRA page or Income Tax Act section supporting each claim. If guidance changed in the last two years or depends on facts you don't have, say so clearly. Question: [question]
Source
CBC News · published 2025-10 · link checked 2026-10-15