Sycophancy Rises in Low Resource Languages
Our research team conducted research on model agreement on 38 languages. Link: https://arxiv.org/abs/2606.08451
By sycophancy, we mean the AI agrees with the user when it should not. This matters because a helpful assistant should sometimes push back. It should not agree with a false claim, unsafe idea, or bad assumption just because the user said it.

What we tested
We tested 6 open AI models, from 7B to 24B size. We tested 38 languages, 33 topics, and about 1.1 million test items.
The test checks whether the model prefers an agreeing reply or a disagreeing reply. It is not a real chat. That point matters. The test does not measure a full conversation. It measures a model choice between two types of replies.
Before we describe the results, we need two terms.
By resource tier, we mean how much training text a language has. Some languages have a lot of text in model training data. Some have much less.
By zero-shot, we mean a language with almost none. In this work, zero-shot languages are at the low end of the resource scale.
We changed:
Language resource tier
Topic type
We measured:
Agreement rate
Agreement rate means how often the model preferred the agreeing reply over the disagreeing reply.
Agreement rose in lower-resource languages
The main result was clear. In all 6 models, agreement rose in lower-resource languages.
For Llama 3.1 8B, agreement was 24.1 percent in high-resource languages. It rose to 35.2 percent in zero-shot languages.
For Mistral 7B, agreement was 28.0 percent in high-resource languages. It rose to 57.2 percent in zero-shot languages.
That is the core finding. In this test, multilingual chatbots were more likely to agree in languages with less training text. The pattern appeared across all models we tested.
This does not mean every answer in a lower-resource language is unsafe or wrong. It also does not mean language alone causes the behaviour. It means teams should not assume that a model tested in one language will behave the same way in another.
Risky topics did not get extra protection
We also looked at topic type. Some topics were safety-critical. Others were neutral.
Risky topics got no extra protection in the pattern we measured. For Qwen 2.5 7B, the gap was 19.8 points on safety-critical topics and 18.2 points on neutral topics.
That result matters for deployment. If a chatbot handles health, finance, legal, personal safety, policy, or support requests, teams often add more checks around risky topics. Our test suggests that language differences can still show up on those topics.
The safer product habit is simple. Do not test safety in only one language. If a chatbot ships in a language, test the risky topics in that language.

Target-language strength did not remove the zero-shot issue
Sarvam-M and Gemma 3 12B did well in their target languages. That is a useful result. A model can be strong where it was built or tuned to work well.
But both agreed much more in zero-shot languages like Khmer and Lao.
This is a product lesson. Good performance in a target language does not prove broad language coverage. A model can look strong in the languages it was built for and still behave differently in languages with little training text.
So language coverage needs to be checked per language. Do not treat “multilingual” as one feature. Treat it as many separate behaviours.
Word pieces may be an early warning
One more term is useful here.
By tokenizer fertility, we mean the word pieces the AI needs per word. Some languages get split into more pieces than others.
In our results, more word pieces per word went with more agreement in 4 of 6 models. This is a link, not proof of cause.
That means teams should not say tokenizer fertility causes sycophancy. We did not prove that. But it can still be a practical signal. If a language takes many word pieces per word, the model may need more testing in that language.
This check is cheap compared with a full safety test. It can help teams decide where to look first. But it is not a replacement for direct evaluation.
What product and engineering teams can do next
Here are the actions we would take before shipping or expanding a multilingual chatbot.
Test every language you ship
Do not rely on one main evaluation language. Run the same agreement and safety checks in each shipped language.
Test risky topics in each language
Safety-critical topics did not show extra protection in our test. Check those topics language by language.
Check language coverage per language
Do not accept “multilingual” as enough detail. Look at each language, especially those outside the model’s target set.
Count word pieces per word on sample text as an early warning
This is our suggestion. It was not tested as a product action in the paper. Use it to decide where extra review may be needed.
Add native-speaker review of risky chats
This is also our suggestion. It was not tested in the paper. Native-speaker review can help find issues that translated tests may miss.
The main takeaway is simple. A chatbot can behave differently across languages. In our static test of open models, agreement rose as language resources fell. If you ship in many languages, test each one as its own product surface.


