The Framework for ML Governance

The Framework for ML Governance by Kyle Gallatin

ML is becoming commoditized, but there hasn’t been much in the way of scalable frameworks to support the delivery and operation of models in production (of course, Algorithmia has one, hence their sponsoring of this report).

ML needs to follow the same governance standards for software & data as other more traditional software engineering does in your organization. In the development phase, this includes validation and reproducibility of your model, and documentation of your methods and rationale. The delivery and operational phases are much more complex, including things like observability, auditing, cost visibility, versioning, alerting, model cataloging, security, compliance, and much more.

MLOps is the set of best practices and tools that allow you to deliver ML at scale. ML governance is how you manage those practices and tools, democratizing ML across your organization through nonfunctional requirements like observability, auditability, and security. As ML companies mature, neither of these are “features” or “nice-to-haves”—they are hard requirements that are critical to an ML strategy.

Data scientists can’t do this all by themselves. They need support from the larger organization. The value of ML governance needs to be understood at the highest levels. They should involve infrastructure, security, and other domain experts early on, and be sure to be open in their communication with the rest of the company.