Tech Insider: Technology – the portfolio you get demoted to; Survey rates the business usefulness of Microsoft’s Copilot AI

Reti had been in denial about his long-rumoured demotion, according to Herald political editor Claire Trevett.

He was given Science, Innovation and Technology (as well as keeping Pacific Peoples and picking up Statistics and a new Universities role). Outgoing Technology Minister Judith Collins had Public Service Minister added to her mix of portfolios instead.

The portfolio formerly known as Communications and Information Technology was once the preserve of heavy-hitters like Paul Swain, David Cunliffe, Steven Joyce and Amy Adams.

It’s now what you’re allocated after being demoted, or you dwell in Cabinet’s lower ranks. Collins got it after being rolled as leader. Kris Faafoi under the previous Government once he had one foot out the door. Melissa Lee had the carved-out Media Communications role as the new Government came in; it’s now held by sixth-ranked Paul Goldsmith – but Collins pitched her portfolio as the “Technology Minister” role.

“Tech is the new Small Business Minister,” concluded one wag on LinkedIn.

One school of thought is that Cunliffe, Joyce and co were necessary because the role once needed someone with oomph as Telecom was being carved up and the multi-billion UFB established. But with the artificial intelligence (AI) revolution and the rise of Big Tech, and New Zealand’s poor R&D stats, Science, Innovation and Technology is more important than ever.

Technology offers a way – arguably, the only way – for New Zealand to solve its low-standing productivity problem, diversify exports and deliver serious economic growth, which has been recently slowing in the sector according to the annual TIN report (which uses a generous definition of the local tech industry).

On the technology side of her portfolio, Collins did a lot of dismantling and de-funding of various start-ups and venture capital programmes initiated by the previous Government, but had yet to reveal what, if anything, would replace them. She did have plans to add new incentives, including long-promised changes to the rules around employee stock ownership programmes for early-stage companies.

In science and innovation, hundreds of jobs have been culled and Labour’s plan for “science city” hubs in Wellington axed. An extended review of what should come next, led by Sir Peter Gluckman, missed July and October deadlines to deliver its report to cabinet. The Herald understands it will finally arrive later today.

Collins knew the lay of the land. She did a ton of industry networking and her son works in software, helping to give her an appreciation of coalface tech issues.

But she was essentially fighting with one hand behind her back, given her portfolio was starved of funding. “GovtGPT” was delivered by three Callaghan Innovation staff in under 100 hours after no dedicated funding or other resources were allocated to the project. The “AI Activator” programme announced by Collins was a similar story, simply being a repackaging of various existing (modest) Government resources. In terms of ambition, it lags the Australian Government’s A$1.2 billion ($1.3b) Next Generation Technologies Fund by some distance.

Reti will need all his skills if he’s to achieve his boss’ desired “delivery and execution” this time.

Are businesses seeing value from Microsoft’s AI?

A new piece of research finds most early adopters of Microsoft’s generative AI are positive about the product’s potential.

But most are still in the process of establishing its business value – and many have a lot of individual users who don’t use it in their day-to-day work.

Copilot can help with tasks like tweaking the tone of any Outlook email, drafting a document in word, summarising a report, finding trends in an Excel spreadsheet or summarising a Teams meeting, complete with action points.

“Employees are finding value in Microsoft 365 (M365) Copilot but tangible business is elusive,” says Gartner in a new report based on responses from 152 IT leaders worldwide (including 132 responsible for Copilot rollouts; 10 were from Asia-Pacific).

Only 3% of the IT leaders said the generative AI provides “significant value” now, although 62% said it was “somewhat valuable and showed signs of living up to its expectations”.

The IT managers said a majority of their Copilot users “find it difficult to integrate it with their daily routine” and there was frustration with issues including inaccurate results.

“Perhaps the most troublesome challenge is that close to six out of 10 respondents say that user engagement declined quickly.”

There was a mix of unrealistic expectations on the users’ part, and a lack of hand-holding and tracking progress with early rollouts from IT teams.

Users need to be fed with frequent tips and fed, more literally at “lunch-and-learn” sessions, Gartner suggests.

Enablement, training and security was more involved than most had anticipated.

Organisations with less sophisticated in-house IT operations were much more likely to see less value in Copilot, and had more staff who gave it up after a brief try.

Those who are getting more out of Copilot are more like to be setting KPIs (key performance indicators) that a business team monitors each month, with the IT team responsible for Copilot’s “fitness”.

Most respondents came from organisations that had, so far restricted Microsoft’s generative AI to small groups of users – in part because of its US$30 per user per month cost and in part because they were still in a proof-of-concept phase (Microsoft shifted the landscape after Gartner’s report was released by offering a pay-as-you-go option, too).

“M365 Copilot is helping employees to accomplish more things more easily, and is helping to eliminate digital friction,” Gartner’s report says.

“Employees, however, don’t have to worry about the price/value ratio.

“That is the job of technical and business decision-makers, who are looking for solid ROI [return-on-investment] equations or tangible business impact (the ability to have a material impact on KPIs and/or elimination of headcount), which has, so far, been elusive.”

The report adds, “97% said that users saved time using Copilot, with 46% reporting time savings of up to 14 minutes per day. While this has an ROE benefit, it may not necessarily be an ROI benefit.

“Individuals saving time does not mean that teams are collectively saving time, and time saved does not mean that saved time is being used productively. Simply put, time saved does not necessarily equal business value.”

Whether using Copilot or a rival product, a sustained effort around governance and improving the quality of information in your organisation will help, Gartner says. You’re more likely to get inaccurate answers or “redundant, outdated or trivial” responses if Copilot is gummed up with low-grade source material. That mirrors a recent Datacom AI Index survey that found a garbage-in, garbage-out theme with “dirty data” a sticking point for many NZ organisations as they sought to rollout AI.

‘The difference between an Uber and a 36-month car lease’

The Herald asked Gartner research director Larry Cannell about his firm’s AI survey, and Microsoft’s new pay-as-you-go option for M365 Copilot Chat agents.

Herald: Why are employees finding it hard to integrate Copilot – and presumably a lot of its completion – into their daily routine? Is this an inevitable teething issue as white-collar workers adopt AI tools en masse for the first time?

Cannell: Yes, in part. People must change their work habits and routines to get the most out of these tools.

But more importantly, we’ve learned that there is a disconnect between how M365 Copilot delivers features and how users need to use generative AI.

Specifically, M365 Copilot today offers horizontally focused features – write emails, summarise meetings, improve writing etc – but we now know that generative AI tools need to focus on specific use-cases or roles to succeed. In other words, horizontal features don’t directly improve vertical use-cases.

What objection looms largest as companies ponder a wider Copilot rollout?

Cost is the biggest factor in deciding to buy and deploy M365 Copilot. The return on investment for Copilot is difficult to quantify for many of our Gartner clients. At US$30 per user per month, M365 Copilot is expensive to license for an entire company. Some companies buy it for all employees, but most Garter clients buy a small number of licenses for a select group of employees.

Will Microsoft shift the needle with its new pay-as-you-go option with 365 Copilot Chat [announced January 15]?

“This announced change addresses this cost issue by enabling users not licensed for M365 Copilot [on the flat-rate of US$30 per user per month license] to access Copilot services. Instead, companies can create more narrowly focused Copilot agents and pay for how often they are used. If the agent is not valuable, people don’t use it and companies aren’t locked into paying a flat-rate subscription.

Digital workplace leaders often prefer this approach – especially for new products – because costs are better aligned with the value delivered, rather than paying for a license upfront and hoping people use it.

The difference between flat-rate and pay-as-you-go pricing is like comparing taking an Uber to the theatre versus committing to a 36-month automobile lease.

At some point you may save money by buying the flat-rate subscription, but you don’t need to commit to it upfront.

How will 365 Copilot Chat work in everyday scenarios?

Access to Copilot agents is the key improvement in this announcement, specifically access to SharePoint agents.

Our clients tell us that SharePoint agents are the most anticipated new type of Copilot agent.

For example, a user can create a SharePoint agent chatbot grounded in a small number of documents. In effect, the chatbot user then “chats” with the documents to answer questions. Imagine an HR policy and FAQ documents grounding a chatbot to help answer questions from employees.

Chris Keall is an Auckland-based member of the Herald’s business team. He joined the Herald in 2018 and is the technology editor and a senior business writer.