Remember to check the facts
AI chatbots can be wrong and still sound confident
AI chatbots can say things that are not true. Sometimes they show bias and discrimination. They might also agree with you, even if what you have said is wrong.
AI chatbots are built using large amounts of data. They use patterns in this data to work out how to reply to your questions. If the data they were made with contain mistakes, the chatbot might repeat these.
But even if the data were perfect, chatbots can still get things wrong. This is because of how they actually work. They break your question down into small chunks called ‘tokens’. These tokens are often individual words. They look at what tokens have come before, and then work out what the most likely next token would be based on patterns they've seen before.
This means that chatbots don't understand the world the same way a person does. They don’t check facts as they go, they just predict likely patterns of words. Most of the time, this technique produces useful and accurate answers. Other times though it is just wrong. When a chatbot says something that is not true, this is sometimes called a ‘hallucination’.
AI chatbots tend to agree with you, even when they shouldn’t. Researchers at Stanford University found that chatbots do not tell people when they're wrong, and do not give them “tough love”.1 When a chatbot agrees with you even if you are wrong, this is sometimes called being ‘sycophantic’.
The researchers showed that many people prefer it when chatbots agree with them. They also found that using chatbots can persuade people that they are right and other people are wrong.
This means that an AI chatbot might reinforce health beliefs which are not factually correct. It might persuade you to choose treatments which cause you harm.
It can be very hard to tell when an AI chatbot is doing this. Here are some things you could try:
- Tell the chatbot to give the other side of the argument to you. This can help you spot limitations in your thinking.
- Tell it not to always agree with you, and not to use flattering language.
- Before acting on anything important, check the facts against an independent source of trustworthy health information (see more below).
Don't forget, AI chatbots are just computer systems. They cannot be offended if you disagree with them.
Throughout history, some groups of people have experienced discrimination and oppression. In healthcare, this can mean that people get worse care because of incorrect beliefs that we now know are not true.
A 2016 study led by researchers at the University of Virginia, USA, found that many members of the public and doctors incorrectly believed that black people have thicker skin and a higher pain threshold than white people. This is not true.2
These types of incorrect statements can be present in the data which is used to train AI chatbots. A 2023 study led by researchers at Stanford University, USA, found that several AI chatbots produced incorrect statements based on outdated and disproven ideas about racial differences, including skin thickness, lung function, and kidney health.3
The companies which make AI chatbots say they are working hard to reduce these biases, and specific errors like these seem to be corrected once identified. But new research shows bias is still present, and is often subtle.4 The issue of bias in algorithms is an active problem rather than one that has been totally solved.
Some chatbots can provide sources to back up the answers they give. This might appear as a footnote or a reference, like in a scientific study. Other times it might appear like a link to a website.
Seeing these references might be reassuring. It suggests that the chatbot has evidence to support its response. However, research has shown that these can be inaccurate.5 This means that the chatbot's reply might not be supported by the evidence it provides.
You should check these sources yourself before making important decisions about your health.
It would also be a good idea to look for independent sources of trustworthy health information.
Official health services like the NHS, medical charities, and international bodies like the World Health Organization are trustworthy sources of online health information.
You can easily find trustworthy health information by searching the PIF TICK website: https://piftick.org.uk. The PIF TICK is a trust mark run by the Patient Information Forum, a UK charity. Organisations that earn it are called Trusted Information Creators. To get the mark they must meet 10 criteria, and be reassessed every year.
You can also speak to a qualified healthcare professional. They will be able to give you evidence-based information from trustworthy sources.
Misinformation is false information which people share because they think it is true.
Disinformation is false information which is created and shared on purpose to deceive people.
An international team of researchers evaluating five commonly used chatbots showed that chatbots can recite health misinformation in their responses.6
If you are making important decisions about your health, it is important you have access to information which is true. Unfortunately, in recent years there have been increasing amounts of health misinformation and disinformation online.
The World Cancer Research Fund has created the simple TRUST Test to help you avoid being misled. Read more about the TRUST Test.7
The test has 5 steps:
T - Too good to be true? Does it promise unrealistic results or quick fixes?
R - Research-backed? Does it trace back to scientific evidence rather than personal stories or opinions?
U - Understood? Has the person sharing the information understood the dangers and risks, or have they downplayed the harms and promoted unproven remedies over appropriate medical care?
S - Source quality? Does it come from a trusted organisation or scientific source? What do other experts say?
T - Think before you share. If it doesn't pass the TRUST Test, don't pass it on.
This last step protects other people by stopping misinformation spreading online.
- https://news.stanford.edu/stories/2026/03/ai-advice-sycophantic-models-research
https://www.science.org/doi/10.1126/science.aec8352 - https://www.pnas.org/doi/10.1073/pnas.1516047113
- https://www.nature.com/articles/s41746-023-00939-z
- https://www.nature.com/articles/s41746-025-01746-4
https://www.jmir.org/2026/1/e82256
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6053515
https://academic.oup.com/jamia/article/33/9/1822/8707496 - https://bmjopen.bmj.com/content/16/4/e112695
https://www.nature.com/articles/s41598-023-41032-5
https://www.bbc.co.uk/mediacentre/2025/new-ebu-research-ai-assistants-news-content
https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php - https://bmjopen.bmj.com/content/16/4/e112695
- https://www.wcrf.org/living-well/health-guides-cookbooks/trust-test/