Business Users Give Microsoft’s AI Assistant Mixed Reviews

Microsoft’s artificial intelligence (AI) assistant, Copilot for Microsoft 365, has been under scrutiny by testers for over six months, with reviews reportedly a mix of positive and negative.

While the AI upgrade has been deemed useful, concerns about its price and limited functionality have been raised, The Wall Street Journal (WSJ) reported Tuesday (Feb. 13).

Copilot for Microsoft 365 integrates with Microsoft software like Word, Outlook and Teams, using the same technology as OpenAI’s ChatGPT, according to the report. It can summarize emails, generate text and create documents based on natural language prompts.

Initial enthusiasm was reported by companies involved in testing, with employees eager to try the tool, the report said. However, some testers have expressed doubts about its cost and occasional mistakes made by Copilot.

Some testers stated that they are not willing to spend $30 per user for every employee in the company, per the report. Microsoft has not disclosed specific sign-up details, but it said that early demand has been unprecedented.

While some AI upgrades from Microsoft, like GitHub Copilot, have seen success, others, like the AI-powered chatbot in Bing search, have struggled to make an impact, the report said. Bing has gained less than 1 percentage point of market share, falling short of Microsoft’s expectations to gain ground against Google.

Early testers have reported efficiency gains, with meeting summaries saving about 30 minutes per meeting and AI-assisted writing saving six minutes on first drafts, the report said.

While some testers have found the AI add-ons to be useful, others have encountered shortcomings, per the report. Copilot has occasionally provided fabricated responses or made mistakes in meeting summaries. Excel users have been less likely to utilize the AI assistant due to occasional errors in number crunching.

Microsoft acknowledges these issues and plans to address the drop-off in usage by incorporating more alerts and tips to guide users, the report said.

The company reported in January that Copilot is being adopted across industries, with the AI assistant and Microsoft’s other AI offerings driving the company to a record quarter during the three months ended Dec. 31.

“A growing body of evidence makes clear the role AI will play in transforming work,” Microsoft Chairman and CEO Satya Nadella said Jan. 30 during the company’s quarterly earnings call.