In this article, I touch upon the inherent problems of measuring the success of Product Operations, and how I have addressed this in my guidance (it is not a model or a framework!).
The Problem
The problem we face in measuring Product Operations is simple: How do you measure how happy your colleagues are with your work?
So much of what we do in Product Ops is people-focused - not tech or even process-focused, but people. How can we help others, guide others, enable others, improve the lives of others… This is very difficult to quantify, and so difficult to measure with any accuracy. If you are reading this as a seasoned Product Ops professional, you almost certainly recognise this problem!
I have long said in conversations: this is not unique to Product Ops. For decades, different industries have tried to measure the satisfaction and happiness of end users (in our case, our colleagues), and there is no exact science still. So what chance have we got to solve it quickly?
To that end, I spent several months researching different avenues to collate and measure this area before putting together this guidance, and I am certain this is just the start of a long period of testing, tuning and debate before we are anywhere near using this as a standard. But, we have to start somewhere!
To begin with, I have split things into 2 camps:
KPIs to measure the overall effectiveness of Product Operations within a business, and
OKRs specific to areas of improvement commonly related to the role.
This aligns with the standard uses of KPIs and OKRs, with KPIs looking backwards and measuring what has happened, and OKRs looking forward to plan (and then measure) what is to come.
KPIs
The KPIs are the big ones - measuring our effectiveness in delivering a positive impact to our teams and the business.
To do this, we have to gather feedback, from our partners, our customers - our colleagues.
It will come as little surprise that I have included NPS as part of the criteria. NPS, despite its overuse and inherent simplicity, is very much a standard to measure how promotional an individual is - in this case, the overall work you have done for them (and the business).
I have also incorporated some of the standard Customer Satisfaction (CSAT) scoring - remembering that Product Ops customers are our colleagues for the most part. This includes satisfaction with the efficiency impact on their work, time saved and the value delivered from what you have done/are doing.
Each of these is asked as questions concerning 6 key areas of Product Operations: Communications, Collaboration, Discovery, Data-informed Decision-making, Strategy & Planning & Business-wide support.
6 areas x 4 questions each (3 x CSAT, 1 x NPS) - that is 24 quick scores.
Each of these is scored 1-10, and then calculations are applied. We have some metrics, that can be analysed, averaged, and compared with previous scores.
Example questions and more details can be found in the spreadsheet below.
It is expected that you would survey your colleagues once a quarter/4 times a year, or every twice a year.
OKRs
OKRs are fast becoming the go-to metrics for businesses and teams. I will not explore the rights and wrongs of OKRs nor how they should best be implemented. But OKRs are useful as a goal-setting and goal-monitoring tool, and Product Operations is all about changes big and small.
In most cases, the changes will revolve around the increase or decrease of an event happening - whether this be the use of a form, a page, a product, a dashboard, emails or IMs being sent, time spent on a process or meeting, or even meeting instances happening. What is great about this, is these are numbers already, naturally. We can specify and collect these stats regularly to form our OKRs.
To many, this will not be new, but if you are newer to the world of Product Ops, OKRs linked to your projects can sometimes be difficult to pin down. And so, I have produced a list of common example OKRs that may be appropriate for you to use.
It goes without saying that the OKRs listed in the spreadsheet below are not an exhaustive list. Use them to inspire you or to get you started. Feed back any additional examples you use and the success of them.
I expect too that you would pick the ones most appropriate for the projects you are currently planning to work on and adjust to the project. Do not be led by the OKR/OKR wording as to what to work on!
It is expected that you would collect these metrics either monthly or in line with your business’s OKR guidance.
How to interpret the guidance
This is guidance, and what’s more, it is the first iteration to try to solve this outstanding issue for Product Ops. Please adapt and experiment, and also feed back to me your findings and improvements, to improve measuring of the role for every professional.
You want to use whatever you collect to show improvement and link this with qualitative and anecdotal evidence that you are making a difference and bringing value, ideally across the business. From a career perspective, you can use the stats to confidently claim ‘you increased staff/customer satisfaction by x %, in your time there, in x months etc’. You can also state specifically how much you increased/decreased x value on x system, process or project by way of the Product Ops-specific OKRs suggested.
I have also worked to separate Product Ops KPIs from OKRs which is something I have often found difficult to consistently do, having KPIs focus on the opinions and satisfaction of my work vs the OKRs being system and process-specific. That is not to say that a number of your OKRs might not be ‘to improve the overall satisfaction of your work across the business’, but with a focus on HOW you will do this looking to the future and with a plan of action.
Acknowledgements:
Thanks to all those who I spoke to over the months to hone this project, and specifically the following professionals who took time to review the near-finalised product:
Holly Holbrook | Nerissa Muijs | Miriam Karlin | May Wong | Lisa B
(Any names forgotten do let me know, with apologies!)
The Spreadsheet
Keep reading with a 7-day free trial
Subscribe to Practical Product Ops to keep reading this post and get 7 days of free access to the full post archives.