AI Agents, Macnamara blog

AI Chatbots Comparison – Not All The Same

In recent years, AI has transitioned from a term used rarely to hearing it used in almost every conversation. Studies have shown that one in six organisations in the UK use AI in the workplace. This is because AI can automate tasks like drafting emails, summarising meetings and generating report. Some AI models are ‘multimodal’, meaning they can work with text, images, voice and video, while others are ‘unimodal’, built for one type of input like just text.

Forbes: UK Artificial Intelligence (AI) Statistics And Trends – Forbes Advisor UK

It can be difficult to find balance between creativity and accuracy. When the heat (or temperature) setting is increased (for example 0.7 or 0.9), the model becomes more creative, but the accuracy drops and becomes less reliable. On the other hand, when it focuses on accuracy, although it stays factual, the results are less imaginative. Different AI tools balance both differently, which is why their answers don’t always look the same.

The AI companies don’t publish verified information on their ‘heat’ of the model, but we have put together some illustrative examples based on what is available.

*These percentages are illustrative and reflect the typical behaviour of each model. AI companies have not yet published official creativity or accuracy metrics.*

Comparison of AI Chatbots

ChatGPT (OpenAI)

Creativity: 70% | Accuracy: 30%

ChatGPT is probably the most popular chatbot. Known for its creativity, it is the go-to chatbot to generate ideas and problem solve. ChatGPT is bult by Open AI, a reason why it is so popular is because it is so easy to use. The higher the creativity, the more hallucinations that can appear. This highlights that ChatGPT should not be used as a trusted model for accuracy.

Copilot (Microsoft)

Creativity: 45% | Accuracy: 55%

Microsoft Copilot is built directly into Microsoft 365 apps like Word, Excel, Teams and Outlook. It’s designed specifically for business use by helping with tasks like meeting summaries, email drafting and data analysis. This can be very helpful with those who have limited time to make presentations or find excel particularly difficult to navigate. Copilot is quite balanced but is more accurate than creative. It is often a preferred choice in work contexts; it is designed to have less hallucinations than other models.

Claude (Anthropic)

Creativity: 35% | Accuracy: 65%

Claude is designed for long, detailed conversations and can handle very large documents. It was created by Anthropic, an AI company founded by former OpenAI employees. Claude is popular in industries like legal, finance and compliance because it’s great at reading long documents and giving clear responses. This AI chatbot has been created to be strong at reasoning therefore it is more accurate than creative. Even when the temperature is increased, it maintains more stability than other models. Despite the strong levels of accuracy, information should still be reviewed.

Gemini (Google)

Creativity: 55% | Accuracy: 45%

Gemini is Google’s AI assistant. It can read and generate text, images and videos. This makes it ideal for research, creative tasks and anything involving multiple types of media. Similar to Copilot, Gemini is balanced although it is more creative due to its ability to handle text, images and video. At higher temperatures, it can become even more experimental and varied in its ideas while causing a decrease in accuracy

AI Tool	Strengths	Weaknesses	Best For
ChatGPT	High creativity Conversational tone	Hallucinations Lower accuracy Uses data to train	Brainstorming Creative writing
Microsoft Copilot	Microsoft 365 integration Security	Less experimental Dependent on existing data	Writing emails, documents, reports
Claude	Strong reasoning Long document handling	Lower creativity	Legal documents Detailed analysis Long-form content
Gemini	Multimodal Creative exploration	Accuracy drops at higher creativity	Creative projects Visual and media tasks

How We Compared the AI Chatbots

To keep things fair, we gave each chatbot the same business prompt, similar to what a real client might ask:

“Draft a short UK English email (max 120 words) to a business client explaining how deploying an AI assistant can help their team work more efficiently and securely.

Include:

One clear benefit,
Two specific examples of the AI assistant in action,
A polite invitation to discuss further.”

ChatGPT

Copilot

Claude

Gemini

While all the tools have fulfilled the remit of the request, the language they have used and the way they have displayed to results does vary. In all cases, they should probably be seen as a good starting point rather than final version, so you can tweak them to suit your own tone of voice.

CEO Fraud: Don’t Let It Be You

CEO fraud is a highly targeted scam where criminals impersonate senior leaders to pressure staff into making payments or sharing sensitive information. This article explains how these attacks work, the warning signs to look out for, and the practical steps organisations can take to reduce the risk through awareness training, strong approval processes, MFA and regular security reviews.

What Is Patch Tuesday?

If you have ever seen a message asking you to restart your laptop, relaunch Edge, or reopen Teams and thought, “Why now?”, you are not alone.

Getting Started with Copilot Studio

Artificial Intelligence is becoming a major part of modern business, and more organisations are starting to look at AI not only as a trend, but as a real productivity tool. One of Microsoft’s most powerful platforms in this space is Copilot Studio.