The right way to Migrate From MLFlow to neptune.ai

MLflow is a framework extensively used for its experiment-tracking capabilities, however many organizations are trying to find alternate options. It’s usually used as an early monitoring answer that groups can simply undertake as it’s well-known and open-source. Nonetheless, they usually find yourself needing extra superior options and higher help than MLflow can present.

MLflow lacks sturdy safety, collaboration, scaling, and interface modularity options. neptune.ai is a powerful alternative to MLflow because it addresses these limitations. It’s constructed to handle bigger ML initiatives and presents a completely managed service to summary away the necessity for infrastructure upkeep.

This hands-on information will stroll you thru all the mandatory steps to carry out the migration. By the top of this text, you should have transferred your information, tailored your coaching scripts, and have all the data it’s essential accustom your workforce to the brand new UI.

Seven causes for migrating from MLflow to neptune.ai

Earlier than diving into the migration course of, let’s see how MLflow and Neptune differ and why migrating from MLflow to Neptune is definitely worth the effort.

1. Safety and compliance: MLflow has restricted permissions administration and lacks built-in authentication options. Neptune presents superior entry controls and meets strict compliance requirements, together with SOC 2 certification, which is essential for organizations working with delicate information.

2. Consumer entry administration: MLflow doesn’t help the administration of consumer entry and useful resource possession, whereas Neptune does.

3. Crew collaboration: MLflow doesn’t present collaboration options corresponding to reviewing initiatives, sharing information, or creating detailed experiences. Neptune gives utilities that make this course of simpler with customizable workspaces and chronic shareable hyperlinks.

4. Consumer interface: Groups could discover MLflow’s interface much less adaptable for analyzing experiments. Neptune’s interface is richer, customizable, and extra intuitive, making it simpler to handle and evaluate experiments.

5. Infrastructure upkeep: Self-hosting open-source MLflow includes configuring and managing the server supporting the appliance. Then again, Neptune is accessible as a completely managed instrument, saving your workforce members valuable time.

6. Scalability and efficiency: MLflow shouldn’t be designed to deal with more and more giant information volumes produced by your increasing coaching pipelines. Neptune can course of giant streams of logs working 1000s of experiments directly.

7. Help: Whereas MLflow’s neighborhood is supportive, there aren’t any ensures that your query or difficulty might be addressed. Neptune presents devoted consumer help, serving to to unravel points rapidly.

Laptop-vision fashions are an integral a part of Veo’s merchandise. Their sports activities cameras produce 4K video information that’s analyzed to supply insights for coaches and gamers.

Initially, the workforce began with MLflow because the experiment tracker however rapidly discovered it unreliable, particularly beneath heavy computational hundreds. This led to connectivity points and crashes, disrupting their product improvement and typically inflicting them to lose per week’s value of labor.

Switching to Neptune not solely supplied a dependable platform for his or her ML experiments but in addition allowed them to trace and analyze tens of metrics effortlessly.

Challenges when migrating between machine studying experiment trackers

Migrating from one experiment tracker to a different might sound overwhelming at first. Nonetheless, breaking it down into clear steps makes migrating from MLflow to Neptune loads much less daunting.

These are the important duties you have to to do:

1. Exporting coaching metadata: Step one is to export your present coaching logs from MLflow to Neptune. Neptune created the neptune-mlflow plugin to deal with this course of for you.

2. Copy dashboards: To take care of your experiment-tracking workflow and information visibility, you have to to copy your MLflow dashboards within the new instrument. Neptune presents no-code dashboard features. You may add pre-built widgets with drag-and-drop and resize to prepare your information visualization as you want.

3. Adapting coaching code: You need to modify your present MLflow coaching scripts to redirect run logs to the brand new instrument. You may both redirect your MLflow logs to Neptune utilizing the neptune-mlflow plugin or carry out a full migration to Neptune’s native shopper.

4. Organising consumer accounts: To handle collaboration on the brand new instrument, you have to to create accounts and arrange consumer permissions. Neptune’s documentation presents steerage on methods to add collaborators to your workspace and manage user roles.

5. Onboarding: You’ll have to practice your workforce to make use of the brand new instrument successfully.

Within the the rest of this text, we’ll work via these steps intimately.

The right way to migrate from MLflow to Neptune: step-by-step information

Listed below are the steps we are going to carry out collectively:

1. Prepare for migration: We are going to guarantee now we have entry to the mandatory sources to carry out the migration.

2. Create a neptune.ai account and project: We are going to join Neptune, entry our workspace, and create a undertaking to prepare our experiments.

3. Set up a local Python environment: We’re going to create a Python setting with the mandatory packages to carry out the migration.
4. Configure the Neptune client: We are going to retrieve an API key and go it to the shopper together with the undertaking identify.

5. Export MLflow logs to neptune.ai: We are going to migrate your present MLflow run logs and fashions to the Neptune experiment tracker.
6. Inspect the imported data: After the export has completed, we are going to confirm that each one information has been transferred and will be accessed in Neptune.

7. Adapt your training scripts: We are going to adapt your coaching scripts to ship experiment metadata to Neptune by including just some traces of Python code.

8. Optional: Migrate from the MLflow client to Neptune’s native client: We’re going to stroll via the refactoring of an MLflow-instrumented coaching script to make use of Neptune’s native shopper.

Put together for migration

Earlier than you begin, guarantee that:

• You might have a pc with Python and pip put in. (We developed this tutorial utilizing Python 3.10.12.) You’ll need to have the ability to create new Python environments and obtain software program packages from the web.

• Your MLflow tracking server is working and you realize its URI. (That is the deal with you go to your MLflow shopper by way of mlflow.set_tracking_uri() or the MLFLOW_TRACKING_URI setting variable.)
• You may entry the MLflow API from the pc you’re engaged on. (When you can log information to MLflow out of your laptop, that is fulfilled.)

Create a Neptune account and undertaking

When you wouldn’t have a Neptune account but, create one on the neptune.ai registration page.

As soon as your account is created, entry your workspace by logging into the Neptune app and create a undertaking as follows:

Click on on All initiatives on the high left. Then click on on Create undertaking on the high proper.

The Create new undertaking type will seem as proven beneath:

Select a reputation in your undertaking corresponding to mlflow-to-neptune, and a undertaking key with higher case letters like MIGRATION. This key might be used to determine runs and different Neptune objects.

Lastly, click on the Create button on the backside proper. You now have a brand new, empty Neptune undertaking to switch your MLflow information to.

Arrange an area Python setting

To carry out the migration, we might want to set up the Neptune and MLflow shoppers in a Python setting. We are going to construct the setting utilizing virtualenv to isolate it out of your different Python initiatives. (When you’re already utilizing a distinct Python setting supervisor, be at liberty to make use of it as a substitute.)

First, set up virtualenv utilizing pip:

When you retailer your MLflow information domestically, navigate to the foundation listing of your MLflow repository:

In all different instances, create a brand new listing (e.g., mlflow_migration).

Create a brand new digital setting:

Activate the setting you simply created:

You now have a practical Python setting.

Let’s proceed by putting in the mandatory packages to carry out the migration utilizing pip:

Configure the Neptune shopper

Subsequent, we have to configure the Neptune shopper. It wants the identify and an entry token of the undertaking to which it ought to ship information.

In your Neptune workspace, entry your Neptune API token as proven within the screenshot beneath:

Within the bottom-left nook of your display screen, click on in your username after which on Get your API token.

The next pop-up will present up:

Click on the copy to clipboard icon on the left to repeat your API token.

Then, set the NEPTUNE_API_TOKEN setting variable in your terminal:

Now, it’s essential discover the exact identifier of your Neptune undertaking. On the primary web page of your workspace, click on in your undertaking. Then, click on on the Edit undertaking particulars possibility within the menu within the high proper:

The Edit undertaking particulars web page will seem.

Copy your undertaking identifier by clicking on the copy to clipboard icon:

Then, set the NEPTUNE_PROJECT variable to this identifier:

These two setting variables might be picked up by the Neptune shopper and permit it to entry your newly created undertaking.

Export MLflow logs to Neptune

Relying on whether or not you retailer your MLflow information domestically or work with a distant monitoring server, the export works barely in a different way.

Native MLflow

In your terminal, navigate to the listing containing your MLflow logs and run:

When you don’t run the MLflow monitoring server at its default deal with, you’ll should go it by way of the –mlflow-tracking-uri command-line possibility:

When you encounter a NeptuneSynchronizationAlreadyStopped exception whereas working neptune mlflow, don’t panic! You’ll merely must run the synchronization manually:

Distant MLflow

When you retailer the MLflow information remotely, e.g., utilizing a centralized MLflow monitoring server, it’s essential go the monitoring URI to the export command:

The export plugin makes use of the MLflow monitoring shopper internally, so you may go any form of URI supported by MLflow.

Examine the imported information

In your Neptune undertaking, it is best to see all of the experiments beneath the Run tab. On the left, you will discover a listing of the experiments migrated from MLflow. 

You may discover that the ID of the exported runs doesn’t match the run names in MLflow.

To match the run names, carry out these steps as proven within the screenshot above:

1. Click on on Runs desk on the high of the online web page.
2. Click on on Add column within the top-right nook.
3. Write run_data/tags/mlflow.runName within the search bar.
4. Click on on the merchandise run_data/tags/mlflow.runName beneath the search bar.

You now have a column in your customized view that shows the unique run names from MLflow:

Adapt your coaching scripts

We will use the identical neptune-mlflow plugin that we used for information migration in our coaching script to instruct the MLflow shopper to log all information to Neptune as a substitute of MLflow.

This lets you begin utilizing Neptune for all of your experiment monitoring rapidly, with out having to interchange the MLflow-specific features corresponding to mlflow.log_metric() or mlflow.log_params().

Right here’s an instance of methods to create a Neptune monitoring URI and go it to MLflow:

Non-compulsory: Migrate from the MLflow shopper to Neptune’s native shopper

Adapting your MLflow scripts utilizing the Neptune MLflow plugin is handy, because it redirects your logs to Neptune with minimal effort. Nonetheless, in the long term, you’ll most definitely wish to swap to the Neptune shopper to get probably the most out of Neptune and its versatile logging capabilities centered across the Run object.

MLflow makes use of a number of distinct features to arrange experiment monitoring. In distinction, Neptune’s init_run() perform returns a Run object. This object is central to Neptune’s monitoring system and gives all of the strategies wanted to log data.

Step-by-step instance: changing MLflow’s shopper with Neptune’s shopper

To information you thru the method of migrating to the Neptune shopper, we are going to refactor the MLflow quickstart script utilizing the neptune.ai API.

After working the refactored script, we are able to entry all of the uploaded metrics, parameters, and mannequin weights within the Neptune UI:

Cheat Sheet: Changing coaching scripts from the MLflow shopper to Neptune’s shopper

For extra data, learn the Neptune API and the MLflow API documentation.

Navigating Neptune as an MLflow consumer

Experiment trackers are fairly related, and switching from MLflow to Neptune is simpler than you may assume.

Let’s stroll via Neptune’s interface, specializing in the similarities and variations to MLflow’s core features.

Experiments and initiatives

MLflow aggregates mannequin runs in Experiments. In Neptune, these are referred to as Tasks.

While you log into Neptune, your initiatives are within the middle of the workspace. That is much like how MLflow shows experiments within the left-hand sidebar:

Run checklist view

Neptune and MLflow show the run checklist view within the middle of the UI. That is the place you see all of your experiment runs at a look:

Neptune gives extra customization relating to which data to show:

Run particulars view

For each instruments, clicking on a row within the run checklist takes you to the run particulars web page. That is the place you see all the data associated to a selected experiment run:

Neptune presents a multi-pane view that shows the run checklist, the logs, and the visualizations on one single web page. MLflow is much less versatile right here and solely shows the important thing data on the run particulars web page. To entry the visualizations, it’s essential click on on a metric, which is able to open a brand new web page.

Mannequin checklist view

In MLflow, you entry the mannequin view from the top-left nook. In Neptune, you attain this view from the left-hand sidebar:

MLflow combines all of the fashions from varied experiments. This setup will be complicated when many groups use the identical experiment tracker as a result of everybody’s information get combined collectively. Neptune organizes fashions into totally different initiatives, making it simpler to view solely your personal fashions.

Mannequin particulars view

For each trackers, clicking on a row within the mannequin checklist view brings you to the mannequin particulars view:

That is the place you may view the data associated to a selected mannequin model.

Neptune helps logging detailed data for every mannequin model. You may log coaching metrics and even the inference velocity on particular {hardware}. As a substitute, MLflow solely reveals primary data, such because the enter and output schema, tags, and outline.

Conclusion

Migrating from MLflow to neptune.ai is simpler than it appears. Neptune presents all of the options, documentation, and instruments to simplify transferring your experiment logs, adapting your coaching scripts, managing consumer accounts, and onboarding your workforce.

Beginning the migration is easy. You simply want a pc with Python and guarantee you may hook up with your MLflow monitoring server by way of the MLflow API.

Right here’s a abstract of the primary steps to carry out the migration:

Utilizing Neptune as an MLflow consumer is easy, as each instruments share many similarities. Adapting your experiment workflows requires little effort and yields nice advantages because of Neptune’s in depth enhancements.

We hope this information has helped you execute or plan your migration. When you’ve got any additional questions or suggestions, please don’t hesitate to reach out. We’re right here to help you in each step of the method.

Was the article helpful?

Sure

No

Thanks in your suggestions!

Thanks in your vote! It has been famous. | What matters you wish to see in your subsequent learn?

Thanks in your vote! It has been famous. | Tell us what needs to be improved.

Thanks! Your recommendations have been forwarded to our editors