AI Use is Rising Among Social Workers and Their Clients—With Ethical Implications
Here’s what you need to know.
Editor’s note: This is part two of a three-part series on artificial intelligence and social work
By Jaimie Seaton
Last spring, an episode of the podcast Death, Sex & Money featured the story of a man whose wife of 20 years filed for divorce a mere four weeks after she began turning to ChatGPT for advice. In isolation, the tale seems incredible, like something out of the dystopian film “2001: A Space Odyssey.” But in the past few years, as the use of ChatGPT and other large language models (LLMs) has become more commonplace, stories of artificial intelligence (AI) therapy gone wrong have exploded.
The most high-profile incidents involve chatbots encouraging suicide, but there are numerous other ways AI can harm those who use it for mental health care. That’s not to say AI can’t be beneficial. As reported in the first part of this series, there are many AI tools that can assist clinical social workers in caring for patients. Similarly, AI chatbots can be beneficial to individuals seeking care. But they must all be used with caution.
Andrea Murray, director of NASW’s Office of Ethics and Professional Review, says social workers need to have competence with the AI tools they’re using and with the emerging mental health challenges their clients are experiencing as a result of their use of AI. “Social workers are ethically charged to stay abreast of these issues,” Murray says.
Along with the rise of use by social workers and their clients, AI also is increasingly being used in education settings. Experts say educators have a duty to incorporate ethics in both situations.
AI As Both Tool And Subject In The Classroom
The current use of AI in academia varies widely. “Some schools of social work are on the leading edge and some are more hesitant to adopt it,” says Karen Magruder, LCSW, a social work instructor at the University of Texas in Arlington. “But our collective consciousness has been raised and we know that this is a big paradigm shift in social work practice and also in education.”
She says one of the most promising uses of AI is case simulations, where students interact with a role-play bot to practice their skills in a low-stakes environment. One of the advantages is that teachers and students can customize the experience to a granular level, giving the bot specific characteristics as the pseudo-client. Magruder emphasizes that the bot isn’t a replacement for human interaction, but is a scaffold, enabling students to gain confidence and competence anytime and anywhere. “Not everybody has an in-person class where they’re able to do role plays,” Magruder says.
Other appropriate educational uses include research, data analysis, personalized study of textbooks or classroom materials and preparing for licensing exams. But Magruder says that throwing a shiny new technology at students just because it’s new isn’t responsible; it needs to be integrated thoughtfully. “We recognize that AI use is becoming more ubiquitous and is one of the top skills that employers are looking for,” she says, adding that may be a compelling reason for educators to teach it.
However, large language models like ChatGPT are only as good as the data they’re trained on. If the data is incomplete, flawed, skewed or biased, the predictions and output will reflect that. Plus, LLMs are notoriously sycophantic. Consequently, educators also need to teach AI literacy and critical-thinking skills so students can evaluate the tools and use them ethically, transparently and with accountability, Magruder says.
She teaches this by having her students critique an AI-generated answer to a question that was already discussed in class. “Number one is making sure folks are aware of what can happen; to be watchful and prepared to use our own knowledge, values and ethics to focus our critical lenses—rather than just taking the output at face value,” Magruder says.
She also notes the technology's heavy environmental footprint, which raises additional ethical concerns. Not only do data centers consume billions of gallons of water to cool the servers running these massive models, but research also shows that the rapid expansion of centers across the U.S.disproportionately impacts vulnerable and disenfranchised communities.
If, when and how to use AI in the field should also be a crucial part of social work training. Magruder says it may be appropriate to use it when writing a treatment plan, for instance. “However, if you're in session and a client asks you something, you can’t whip out your phone and say, ‘Let me ask ChatGPT.’”
Aside from that clear boundary, if, when and how to use AI in clinical practice is not a clear black- and-white rule, Magruder says. “The truth is, we need to be more judicious and think it through because it really does depend on the setting.” She advises her students to follow the code of ethics of their employer and to be vigilant about confidentiality, bias and fact-checking. “Think of it as a helpful intern whose work you must verify.”
Two other critical concerns around the use of AI in higher education are confidentiality and academic integrity. Magruder points out that her school uses a closed-loop copilot system, meaning the data isn’t used to train other AI tools. Closed-loop systems also can be configured to be FERPA (Family Educational Rights and Privacy Act)-compliant. Free versions of ChatGPT and Claude, on the other hand, are open systems that save and use chat histories to train future models and therefore can’t be made FERPA-compliant.
The second big consideration (for educators across the board) is the extent to which students can use AI tools for assignments. As educators have begun to recognize that AI is used in the workplace, policies have moved from zero tolerance to reflect real-world expectations. “It’s more nuanced,” says Magruder, who doesn’t have a blanket AI statement for the class. “I have AI policies for each individual assignment. It gets that granular—‘you can do this but not that.’” Mirroring new policies at peer-reviewed journals, she also requires AI-use disclosures.
You Don’t Know What You Don’t Know
As reported in the first part of this series, confidentiality and informed consent are primary concerns when implementing AI into clinical practice. Worth adding is that social workers may not always know they’re using AI because it’s embedded into so much of what they do. And unless they’re in private practice, they may not have any control over what tools their employer is using. Insurance companies, and telehealth companies and apps that contract social workers for mental health care also are using AI tools.
Jamie Sundvall, PhD, PsyD, LP, LCSW, is the director of online education at the Graduate School of Social Work at Touro University and a nationally recognized expert on AI. She cautions that new tools like digital phenotyping and voice analytics introduce serious ethical gray areas. Digital phenotyping tracks a client’s mood through their smartphone keyboard interactions, while voice analytics assess pitch and rhythm to generate automated mental status exams. The core risk arises when telehealth portals capture this patient data before the clinician is even involved, often bypassing best-practice disclosures.
“It puts the social worker in a very difficult spot informing the client of how their information is being used,” Sundvall says. “The social worker may not even know to ask the telehealth company that they are contracted to about the information.”
Sundvall says the ethical risk of these new technologies deepens when considering how AI models are trained on clients’ data. When platforms use voice analytics and digital phenotyping, they are capturing the unique communication patterns of diverse and vulnerable populations.
“Different populations you’re working with, those patterns and experiences could be totally misunderstood and created as a stereotype or a general piece that is then applied to society,” Sundvall says. “I have concerns that these models can precipitate racism and prejudice based on language analysis and different dialects and the use of language.”
Sundvall recommends asking vendors in writing if they use zero-retention policies or federated learning architecture (a privacy-preserving technique that trains AI without centralizing raw client data). Naturally, any AI use must be disclosed to clients for informed consent.
Clinicians also must realize that HIPAA (Health Insurance Portability and Accountability Act) compliance does not stop a platform from using client data for training, Sundvall notes. “Social workers should still be able to have their own informed consent on what they use, being the expert with the population they serve.”
How Clients Are Using AI
A 2026 KFF tracking poll found that 32% of adults have used AI tools and chatbots for health information and advice, with half of those using them exclusively for mental health. Use of these tools falls into two buckets, says Frederic G. Reamer, PhD, a longtime expert adviser on ethics and technology to NASW.
Some use specialized platforms, like Wysa or Replika, while others turn to general-purpose chatbots such as ChatGPT, Gemini or Perplexity when they need help outside of office hours. A person struggling with a panic attack at 2:37 a.m., for example, may turn to an AI chatbot for advice, reassurance or simply someone—or something—to talk to, Reamer explains.
For social workers, that reality creates a new ethical obligation, he says. He believes clinicians should routinely ask clients whether they are using AI for emotional support and discuss both the benefits and risks. “I think a clinical social worker in 2026 who doesn’t bring up this issue is not complying with emerging ethical standards,” he says. While some clients may use AI thoughtfully as a supplement to therapy, Reamer cautions that others may be particularly vulnerable.
For these vulnerable clients, the day-to-day dangers of these platforms can be insidious, heavily shaping how they process their experiences. As Shaddy Saba, PhD, LCSW, an assistant professor at NYU’s Silver School of Social Work and the lead author of a paper on how therapists can ask their patients about their AI use notes, turning to AI for help creates a kind of linguistic loop.
Saba warns that “it’s not necessarily the case that they’re grounded in the realities of people’s lives. And so the kind of feedback that people are receiving from them, it might be off base in those ways.” This inherent unreliability is compounded by AI chatbots’ sycophantic tendencies. Because these tools are designed to be overly agreeable and affirming, they validate a user’s cognitive distortions, fear or paranoia instead of safely challenging them the way a trained clinician would. In moments of acute crisis, this lack of clinical discernment means the chatbot’s output is frequently incomplete, wrong or explicitly harmful. In the most severe cases, these platforms have completely failed to manage suicidality or psychosis, neglecting to direct users to human help or even actively encouraging suicide (resulting in many high-profile lawsuits).
According to The New York Times, OpenAI data reveals that over a single month, approximately 1.2 million users showed possible suicidal intent and 560,000 exhibited signs of psychosis or mania.
Perhaps most concerning for a profession rooted in social justice is the tech’s built-in bias. As noted above, because these tools are trained on data that mirrors our own society, they replicate systemic racism and prejudice. The result is an AI that is fundamentally less empathetic—and potentially outright harmful—to marginalized communities, including people of color, the LGBTQ+ community, and individuals with severe mental illness.
Interestingly, both Reamer and Saba landed on the exact same medical analogy to describe a social worker’s responsibility in the age of AI. Reamer argues that just as any responsible physician must ask a patient what over-the-counter medications, supplements or edibles they are taking to check for dangerous drug interactions, “social workers have got to do exactly the same thing” with artificial intelligence to see if clients are getting support from outside sources.
Rather than lecturing clients on a laundry list of tech dangers, Saba recommends using this inquiry to open the floor to a collaborative, normalizing conversation about what they are experiencing online (see sidebar). He suggests using a simple metaphor to help clients conceptualize the tool, explaining that “these things are like a well-read friend who’s sometimes confidently wrong.” Ultimately, treating AI use like any other unprescribed coping mechanism allows clinicians to understand what a client finds beneficial while safely managing the risks together.
Ultimately, navigating this rapidly shifting landscape requires a commitment that goes far beyond simply ignoring the technology. Even if a clinician wants absolutely nothing to do with AI—and many did not sign up for this and simply want to sit in a room with a human being, Reamer notes—the reality is that professionals can no longer afford to look the other way. He says every single clinical social worker needs to update their consent-to-treat protocols to routinely ask clients about their AI use.
Safely dealing with these tools is going to require a level of consistency that the current NASW Code of Ethics just is not equipped to handle on its own. Sundvall argues that the profession needs a much more robust approach. “I think it’s going to require a very in-depth ethical framework that’s more than just the code of ethics that speaks directly to AI use,” she says.
Because the technology is growing at such a breakneck pace that an accidental, irresponsible error could prove detrimental to individuals, communities and the entire profession, Sundvall points to a clear need for larger-level infrastructure and mandatory continuing education requirements from licensing boards. Even with new rules in place, practitioners must remain fiercely adaptable in their daily work.
Magruder encourages clinicians to weigh the pros and cons while recognizing that where a social worker lands today almost certainly will be obsolete in six months as the technology continues its relentless evolution.
Jaimie Seaton is a New England-based journalist with 30 years of experience. Her work appears in multiple publications, including AARP, National Geographic, Scientific American and Smithsonian Magazine.
Resources:
Study reveals AI chatbots can detect race, but racial bias reduces response empathy, 2024 (MIT): news.mit.edu/2024/study-reveals-ai-chatbots-can-detect-race-but-racial-bias-reduces-response-empathy-1216
Sycophantic AI decreases prosocial intentions and promotes dependence (Science): science.org/doi/10.1126/science.aec8352
AI chatbots perpetuate biases when performing empathy, study finds (UCSC): news.ucsc.edu/2025/03/ai-empathy/
How Bad Are A.I. Delusions? We Asked People Treating Them (NYT): nytimes.com/2026/01/26/us/chatgpt-delusions-psychosis.html
KFF Tracking Poll on Health Information and Trust: Use of AI For Health Information and Advice (KFF): kff.org/public-opinion/kff-tracking-poll-on-health-information-and-trust-use-of-ai-for-health-information-and-advice/
Patients Use AI—Clinicians Should Ask How (Saba, JAMA Psychiatry): jamanetwork.com/journals/jamapsychiatry/article-abstract/2847068