The Paris-based start-up Mistral AI launched a competitor to the ChatGPT conversational robot on February 26. Named Le Chat, it's part research assistant, part private secretary. The tool is free to use, but you will be put on a waiting list, which will grow depending on the amount of requests, said a company spokesperson.
Le Monde asked the French chatbot hundreds of questions to compare its skills with those of the pioneering ChatGPT tool. Le Chat's performance came very close to that of the American tool, and even came ahead on several points, including document synthesis. Overall, Le Chat made more errors than ChatGPT in our tests. However, as it has not yet been fully perfected, we can't rule out the possibility of this gap narrowing.
Convincing answers to complex questions
Like ChatGPT, Le Chat gives the impression of great lucidity. It often answers correctly, even remarkably, almost every question put to it, even when the interrogation is complex ("Who invented postmodern dance?", "What is Nabokov's best novel?", "Was the French state in favor of the telephone in the 1870s?").
The unbelievable know-it-all produces answers of comparable quality to both ChatGPT's paid and free versions. Unfortunately, it disappoints in the same way, too. A major error occasionally appears in its answers. For example, Le Chat attributes inventions to Steve Wozniak though he had no part in them, such as the first portable Macintosh. When we tested its competitor Copilot (originally named Bing Chat) just a year ago, it also added inventions to the already considerable list of Wozniak's contributions to the history of computing.
This unpredictable tendency to make mistakes undermines confidence in the system and means we have to check information we can't afford to be wrong about. Le Chat's error rate seemed higher than ChatGPT's, but our tests form a data set that is too narrow to draw any definitive conclusions.
Convincing draft messages
The American ChatGPT robot is capable of writing a draft message based on a brief textual instruction. Its drafts are rarely usable as they stand, but are sometimes sufficiently advanced to save time. For the moment, Le Chat seems to be a little worse at this. Reworking and refining drafts often requires more work.
The French AI is capable of finishing an unfinished text, transforming clumsy French into correct language and then reformulating it in another style (warm or formal, for example). It can also give arguments to convince an interlocutor or counter their attacks. Its performance is passable in this respect, closer to that of ChatGPT version 3.5 (free) than ChatGPT 4 (paid), with its more precise ideas and richer French.
Poorly summarized documents
It's a disappointment: Le Chat rarely succeeds in correctly summarizing a document several dozen pages long, or one with a complex structure, such as a scientific study. It alternates between errors, misunderstandings and omissions, whereas the paid version of ChatGPT generally manages admirably, apart from the odd misunderstanding. It's impossible to draw a comparison with the free version of ChatGPT, which is limited and incapable of summarizing this type of document.
When Le Chat is asked to summarize a simple text of a few pages, its performance is equivalent to that of the paid version of ChatGPT, and even slightly surpasses that of the free version of ChatGPT. However, all three tools make occasional errors which, once again, undermine confidence in them.
A consensual and positive personality
When Le Chat is taken to slippery subjects such as racism, sexism or interpersonal conflict, its bias is clear: It is ethical and progressive. This conversational robot often produces responses particularly similar to those of ChatGPT, often starting with the same words: "I understand your concern." It professes active listening, reinforcement, assertiveness and constructiveness.
When we try to establish an emotional relationship with it, it generally stays in the background, reminding us that it is an emotionless robot. Its sentences often start with the same words as ChatGPT: "I don't have the ability to."
Features to be enhanced
While Le Chat's artificial intelligence is promising, its functionalities are still a work in progress. For example, no smartphone application can be voice-activated while driving or cooking. Also missing are tools reserved for the paid version of ChatGPT, such as image generation, the creation of personalized conversational robots and the ability to import an image for analysis. It's also impossible to import a text document unless you trick it by copying and pasting its web address.
Mistral is working on a paid version of Le Chat with more advanced features, to be launched at an indefinite date and price. A spokesperson for the French company explained to Le Monde that its paid version could then be restricted. It would use its most powerful engine only for the most complex answers, resorting to a less resource-hungry engine for simple questions.