Job Spotlight: Machine Learning Product Manager
I would like to argue that a certain Job or Role is more important than today’s size of its ranks suggests, and that its importance will even grow. There already exists a confusing plethora of Jobs related to Data and Machine Learning, so the first question is why having another, or how does this one differ?
The motivation behind my thought is that many companies seem to struggle with reaping the benefits of data with machine learning, despite having sizable departments for that purpose, and despite there being many advances in the technical and scientific areas. I could probably cite some Magic Gartner Quadrant Harvard Business Review Study Whitepaper Thingy called Business Not Reaping ML Benefits Enough, but I’m too lazy to search for it, you would be too lazy to read it, and the study would be likely flawed anyway. I mean, anything called Megatrends in the Hype Cycle is kinda sus.
So let’s just stay on the level of pure thought and anecdotal evidence.
Aspects of Machine Learning Roles
Typical Job Requirements for an ML/Data position list proficiency with frameworks for machine learning (Tensorflow, Keras) and data processing (Pandas, Spark), a university degree or self-obtained understanding of the theoretical basis for machine learning, and some generic software engineering skills.
Varyingly, you may have an academic publication track record, focus on certain area (image recognition, reinforcement learning), or proficiency with some cloud tools or deployment frameworks.
During the interview process, the candidate is asked to solve some artificial data problems, and demonstrate technical knowledge. Often, this is accompanied with joking remarks about most of the job being about meetings and data cleaning.
Let’s now put such a Role into the context of Business lifecycle. For simplification, I consider the lifecycle of projects (products) to consist of:
- Ideation — e.g., talking to potential customers, spying competition, thinking (out of the cubicle);
- Exploration — e.g., doing PoCs and mocks, presenting to potential users, doing market research;
- Implementation — e.g., developing actual code;
- Maintenance — e.g., sales, customer support, cloud infrastructure monitoring, bugfixing.
Most of what we understand to be applied ML work falls firmly into the Implementation phase — the data cleaning, model training, model deployment. There is a big chunk of activity in maintenance as well (analysis of performance/concept drift), usually labelled MLOps.
Sometimes, there is activity in Exploration (such as development of a PoC model), but often, there is “not enough data” at first. ML is much easier to ideate and deploy when improving over existing heuristical/human processes.
The ideal task of a ML practitioner comes with a full definition of the task (recognize a hand-written digit), the acceptance criteria (performance metrics bounds) — so that they can then start doing what they were trained for, all the feature engineering, model selection, hyperparameter tuning…
If ML is thus to bring potential not just as an incremental improver of existing well-defined tasks (click-through rate, churn, …), but a disruptive innovation, someone needs to be able to ideate projects that exploit ML potential and formulate them, or their parts, as ML problems. And that person needs to be very business/customer savvy. At times, ML departments are viewed as a black box you pour data in and innovation flows out — yet that is not what, based on my limited experience, happens.
The current ML research probably does not focus much on decision trees or linear regressions, as those are “scientifically boring” already — their potential understood, best practises for their usage identified. A practitioner may even shy away from using them and spending time fine-tweaking them for a particular problem, instead of wielding some mighty generic neural network architecture that generalises to a much higher class of problems “on its own”, or promises a paper publication in addition.
Yet, there likely remain many untaken business opportunities that would make do with just the basic methods. I’ve recently solved two simple-but-important tasks related to electricity consumption time series and classification/forecasts with just observing by eye what features and coefficients could a linear algorithm use — and it was sufficient for the business requirements.
Evolution of a Machine Learning Practitioner
I absolutely don’t claim that pure ML research is not needed, or that everything can be solved using simple methods.
But, with the improvements of ML accessibility, or even the rise of AutoML technologies in offerings such as Google Vertex, it becomes less important for a startup or an innovation department to employ experts with a deep knowledge of various ML methods — a limited intuition of what could/should work, understanding how to verify whether it indeed works, understanding the role of labels, knowing when to call a real ML expert, etc, may be sufficient.
The people doing the Ideation phase, and handing down requirements to Software Engineers to implement, are usually called Product Managers and UX Designers. Requirements for their jobs are vastly different from those of Machine Learning roles — not much technical, but rather about soft skills, presentation, empathy, drive, business understanding, …
Doesn’t it sound naive that we expect a mutual understanding to somehow happen between an ML Scientist and UX Designer? With traditional software development, it somehow did work — yet, it seems to me that in the most efficient cooperations either side did know the basics of the other’s craft.
Yet, with machine learning, it seems to me that all the outsider groups — software developers, product managers, executives — treat it as some arcane black box. Also, with software development — it is not often expected that software developers themselves will bring disruptive ideation and innovation; instead, they are seen at best as partners or constructive opponents in the process.
Different treatment of machine learning is, to a certain extent, justified — because it is not always easy or simple. Therefore, the point of my thinking is that a dedicated role of Machine Learning Product Manager, is to gain importance in near future. I’m of course not coining it — Google Trends is showing some hits already in 2014, with about a tripling in 2018. LinkedIn is offering no such job in my area, but, when looking at California, I’m seeing a bunch of exactly such named positions by all the big players (Meta, Amazon, Google, …). Others wrote articles with such name, e.g., here from August 2020 (and even cites a PWC research — feel free to go read it).
The exact definition is not important, though — regardless of whether the original background would be product management or machine learning, the need to understand both, and the willingness to care about both, is what really matters. I’m using ML Product Manager in this article — but ML Product Designer or ML Business Analyst could have been used as well, whatever. And for specialised domains (health, physics, …), a domain expert is in essence the PM.
Conclusion
If you are in a company that expects to benefit from having a ML department, ask — who is the person bridging the worlds? Who is able to formulate problems as ML ones while at the same time able to evaluate whether a given solution is sufficient / beneficial at all for the business?
If you have to name a combination of a PM person plus an ML person — it may be that your company is missing a link.
Of course, there are other approaches, such as not having a ML department at all, and rather, disperse ML practitioners among product teams, and treat ML just as another problem solving tool in the software development toolbox; and innovate with a tool-free mindset. Identifying which of those two is a better fit for your case ain’t an easy thing, tho.
Maybe the right way for ML is to move its expert out of the general business companies, and turn them into freelancers or ML-focused companies for hire that also work as research labs; and leave only a general “what is the typical power of ML” knowledge among the PMs and executives, and general “how-to train off-the-shelf model” knowledge among the regular SW engineers (it seems quite ambitious that “full-stack dev” has already been coined). In a category-theory sense, this mirrors what happened with PaaS offerings, or what may yet happen with the No Code.
