The Real Insight Behind Measuring Copilot Usage Is Microsoft’s Desperation

Opinion The quantum theory of management includes an analogy for the physical law of the observer effect, where observing a system changes its state. When you make a metric a target, it is not useful as a metric. Instead of reflecting whatever underlying behavior it was intended to measure, the metric becomes a measure of how well the benchmark is being gamed.

Thus, Microsoft including Copilot uptake in its Viva Insights corporate surveillance tool as a proxy for productivity can’t be anything of the sort. So what is it?

Feeling lonely? Microsoft Copilot can now listen to your every word, watch your screen

READ MORE

First, it is remarkable at every level. Microsoft says it creates cohorts of employees based on Microsoft’s assessment of region, job function, and manager type – presumably org chart position – to “determine expected values by role.” This is then normalized and compared to others both inside a company and to equivalents in other companies. Yes, Microsoft is collecting your internal company performance data and sending it to your rivals, but don’t worry, it’s all protected by “randomized mathematical models.” Yes, it is of necessity ignoring anything that differentiates the way you work from Microsoft’s idea of a theoretical mean, but that’s how managerialism works.

The Microsoft blog post where all this is announced is masterfully evasive, using undefined terms and lacking any sort of checkable detail, theoretical underpinning, research data, or nuance of any kind. One undefined term gives the game away, where Microsoft says: “The cohort result looks at the role composition of the selected group, and constructs a weighted average expected result based on matching roles across the tenant.” Precisely what “tenant” means is not explained, but it probably shows Microsoft’s thinking of Viva Insights as a multi-tenant platform with each subscribing company being a tenant.

It also reveals Microsoft breaking one of the primary rules of multi-tenant software – that each tenant is invisible to and secure from all the others. But hey, randomized mathematical models.

This sort of unaudited public usage of private data, while being the unregulated plutonium on which the experimental nuclear reactors of AI feed, precisely demands – at the very least – opt-in agreement from the harvested. There is no mention of this in the announcement, so it is safe to assume that there is nothing optional here. Of course, if organizations can exempt themselves from the system, the cross-company comparisons would fail utterly to reflect the “top 25 percent of companies” as it’s quite possible that none of the actual top companies is taking part. It would fatally poison the underlying statistical model, were one to exist.

The next remarkable thing about Copilot adoption metrics in Viva Insights is that it exists at all. Corporate uptake of productivity software has been both a marketing tool and genuine guidance since desktop computing began. Software vendors love saying how popular their products are, while skeptical fans of reality look to third-party analyses to see how a market is really developing. Look at the decadal saga of Windows version adoption for a fine example. Monitoring actual usage and then integrating it into a live managerial insights dashboard is a whole new level, and highly revealing of Microsoft’s internal perception of Copilot’s potential for success.

Microsoft declares bring your Copilot to work day, usurping IT authority

READ MORE

In the same way that public awareness of software uptake is nothing new, the collection of actual usage data by software vendors is long-established and good practice. Telemetry from deployed apps showing what is and isn’t being used, how much it’s used and how successfully, is an essential part of life cycle management. Given the squirrelly state of so much corporate software, this isn’t being used nearly enough.

Yet this only gives a vendor competitive advantages when the data gathered is strictly internal. Microsoft has never seen fit to reveal live cross-corporate usage data for Excel or Visual Studio or Teams, let alone in a gamified leaderboard format. It needs to do it for Copilot, because the actual productivity gains of Copilot are not quantifiable or even visible. Nor are they becoming so.

Microsoft is forced to present Copilot usage by synthetic cohorts through undefined processes for undefined purposes because it is desperate to find a way to make people use the stuff. In general, people use productivity tools to the extent that it makes them more productive. You don’t need to push actual usage after a sale unless there is a crisis in usage after a sale.

That general desperation across the AI industry about adoption is so thick you can virtually smell it, every time a pop-up begs you to try a feature you’ve ignored 30 times this week already. Microsoft is hoping that the magical thinking of managerialism – that all behavior can be quantified and optimized according to rules – will apply to anything it presents as quantifiable and optimized.

Managerialism is notoriously impervious to independent validation or rigorous derivation, which makes it ideal juju for today’s culture of factual superposition: if you can’t actually observe something, it’s safe from analysis that may reveal something dangerous. Present a proxy for such observations, and you can synthesize behavioral metrics that the machinery of management will process like any other.

For this, Microsoft has broken the rules of multi-tenant platform management, the rules of ownership of corporate data, and the rules of consent. It is fitting that this is in the service of AI, which drinks from the same well. It is also a genuine insight into AI uptake and its future path, just not an insight that Microsoft wishes you to have. ®

Original Source