保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Artificial Intelligence

Building Machine Learning Operations for Businesses

A Blueprint for Effective MLOps to Support Your AI Strategy John Adeojo June 20, 20235 min read Image by Author: Generated with Midjourney Image by Author: Generated with Midjourney Pipelines : Kedro structures your data processing as a pipeline of dependent tasks, enforcing a clear code structure and visualising data flow and dependencies. Nodes : In Kedro, a Node is a wrapper for a Python function that names the inputs and outputs of that function, serving as the building blocks of a Kedro pipeline. Configuration : Kedro manages different configurations for various environments (development, production, etc.) without hardcoding any configuration into your code. I/O : In Kedro, I/O operations are abstracted from the actual computation, which increases code testability and modularity and eases switching between different data sources. Modularity and Reusability : Kedro promotes a modular coding style that results in reusable, maintainable and testable code. Testing : Kedro integrates with PyTest, a testing framework in Python, making it easy to write tests for your pipeline. Versioning : Kedro supports versioning for data and code, enabling reproduction of any previous state of your pipeline. Logging : Kedro offers a standardised logging system for tracking events and changes. Hooks and Plugins : Kedro supports hooks and plugins, extending the framework capabilities as per project requirements. Integration with other tools : Kedro can be integrated with various tools like Jupyter notebook, Dask, Apache Spark, and others to facilitate different aspects of a data science workflow. All Kedro projects follow this basic template. Enforcing this standard across your data science teams will enable reproducibility and maintainability. For a more extensive overview of the Kedro framework, please visit these resources:

Registry & Storage - Data Version Control (DVC)

Registry and storage underpin reproducibility in machine learning, something that any business looking to incorporate ML should bear in mind. ML models are essentially composed of code, data, model artefacts, and environment - all of which must be traceable for reproducibility. DVC is a tool that provides version control and tracking for models and data. While GitHub could be an alternative, it's limited in its capacity to store large objects, posing issues for extensive datasets or models. DVC essentially extends Git, offering the same version control capabilities while enabling storage of larger datasets and models in a DVC repo, which can be either local or cloud-based. In commercial settings, there are obvious security benefits to versioning your code in a Git repo, while storing actual model artefacts and data separately in a controlled environment. Remember, model reproducibility will become increasingly important as regulations tighten around the use of AI commercially. Reproducibility facilitates auditability.

Model Pipeline Deployment - Docker

Image by Author: Schematic of Docker deployment of inference pipeline. Note this same approach can be applied to the model monitoring and retraining pipeline Image by Author: Schematic of Docker deployment of inference pipeline. Note this same approach can be applied to the model monitoring and retraining pipeline Deployment isn't merely a single task but rather a meticulously crafted fusion of tools, activities, and processes; Docker ties all these together for model deployment. Crucial for intricate ML applications with numerous dependencies, Docker ensures consistency across any machine by encapsulating the application with its environment. The process begins with a Dockerfile; Docker then uses its commands to construct an image, a ready-packaged model pipeline fit for any Docker-enabled machine. Teamed with Kedro's pipeline functionality, Docker can proficiently deploy both model retraining and inference pipelines, assuring reproducibility across all stages of the ML workflow.

Model Monitoring & Retraining Pipeline - MLflow

Over time, machine learning models suffer from performance deterioration, which can be due to concept drift or data drift. We want to be able to monitor when our models' performance begins to falter and re-train them when necessary. MLflow provides us the ability to do this via its tracking API. The tracking API should be incorporated into the model training and inference pipelines built by the data scientists. Although I have specified MLflow for tracking in the model monitoring and retraining pipeline, tracking can also be done in the model development lab, particularly for experiment tracking.

The Inference Endpoint

Given that the inference pipeline has been encapsulated into a Dockerfile, we can create a Docker image of the pipeline anywhere to be used as an API endpoint for any app. Depending on the use case, we will have to decide where we deploy the Docker image. That, however, is beyond the scope of this article.

Roles & Responsibilities

Assigning distinct roles and responsibilities within MLOps is pivotal to its success. The multifaceted nature of MLOps, which spans across disciplines necessitates a clear demarcation of roles. This ensures that each task is performed efficiently. Further, it fosters accountability, facilitating a quicker resolution of issues. Lastly, clear delegation reduces confusion and overlap, making the team more efficient and helping to maintain a harmonious working environment. It's much like a well-oiled machine, with each cog playing its part to perfection.

Data Scientists

Machine Learning Engineers

Data Governance Professionals

Conclusion

Navigating the realm of MLOps is a task that demands deliberate planning, the right blend of technology and talent, and an organisational culture that endorses change and learning. The journey may appear complex, but by employing a well-designed blueprint and by approaching MLOps as a holistic, iterative process rather than a one-off project, you can derive immense value from your AI strategies. Remember, though, that no single approach fits every scenario. It's crucial to tailor your strategy to your specific needs and to remain agile to changing circumstances.

Follow me on LinkedIn Subscribe to medium to get more insights from me:

Join Medium with my referral link - John Adeojo Should you be interested in integrating AI or data science into your business operations, we invite you to schedule a complimentary initial consultation with us: Book Online | Data-Centric Solutions

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

John Adeojo

See all from John Adeojo Artificial IntelligenceBusiness StrategyData ScienceMachine LearningTechnology Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Applied Time Series Analysis of Bike Accidents in Madrid

Machine Learning You will be safe cycling home Antonio Ramos November 24, 20227 min read

Scalable Serving with Kubernetes and Seldon Core: A tutorial

Machine Learning Learn how to deploy ML models in Kubernetes clusters and to implement autoscaling for your deployment with HPA and KEDA Tin Nguyen January 9, 202311 min read

Exploring a Data-set with Simple Pandas and Plot Visualizations Features

Programming Pandas and Plotly for Initial Data Analysis- A Tennis Example Stephen Fordham September 6, 20196 min read

Cheatsheet for NumPy: Essential and Lesser-Known Functions

Machine Learning Learn various essential functions of NumPy which you might be unaware of (with code) Shantnu & Kartik September 3, 20208 min read

The Most Common Authentication Systems Explained

Artificial Intelligence A novel approach to integrating speech recognition into authentication systems Part 1 Fangyi Yu October 9, 20210 min read

Achieving Greater Self-Consistency in Large Language Models

Deep Learning Artificial intelligence software was used to enhance the grammar, flow, and readability of this article's... Anthony Alcaraz December 1, 20239 min read

Learn Machine Learning Concepts Interactively

Machine Learning Five freely available tools that intuitively break down the complicated machine learning concepts Parul Pandey October 4, 20203 min read

A Quick Introduction to Bag of Words and TF-IDF

Machine Learning How Machine Learning and Natural Language Processing Models Deal with Text Brian Roepke January 21, 20225 min read

Stacked Autoencoders.

Deep Learning Extract important features from data using deep learning. Rajas Bakshi June 28, 20216 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close