Python
An open source AutoML toolkit for neural architecture search, model compression and hyper-parameter tuning.

简体中文

NNI (Neural Network Intelligence) is a lightweight but powerful toolkit to help users automate Feature Engineering, Neural Architecture Search, Hyperparameter Tuning and Model Compression.

The tool manages automated machine learning (AutoML) experiments, dispatches and runs experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in different training environments like Local Machine, Remote Servers, OpenPAI, Kubeflow, FrameworkController on K8S (AKS etc.) and other cloud options.

Who should consider using NNI

• Those who want to try different AutoML algorithms in their training code/model.
• Those who want to run AutoML trial jobs in different environments to speed up search.
• Researchers and data scientists who want to easily implement and experiement new AutoML algorithms, may it be: hyperparameter tuning algorithm, neural architect search algorithm or model compression algorithm.
• ML Platform owners who want to support AutoML in their platform.

NNI v1.2 has been released!  

NNI capabilities in a glance

NNI provides CommandLine Tool as well as an user friendly WebUI to manage training experiements. With the extensible API, you can customize your own AutoML algorithms and training services. To make it easy for new users, NNI also provides a set of build-in stat-of-the-art AutoML algorithms and out of box support for popular training platforms.

Within the following table, we summarized the current NNI capabilities, we are gradually adding new capabilities and we'd love to have your contribution.

	Frameworks & Libraries	Algorithms	Training Services
Built-in	Supported Frameworks PyTorch Keras TensorFlow MXNet Caffe2 More... Supported Libraries Scikit-learn XGBoost LightGBM More... Examples MNIST-pytorch MNIST-tensorflow MNIST-keras Auto-gbdt Cifar10-pytorch Scikit-learn More...	Hyperparameter Tuning Exhaustive search Random Search Grid Search Batch Heuristic search Naïve Evolution Anneal Hyperband Bayesian optimization BOHB TPE SMAC Metis Tuner GP Tuner RL Based PPO Tuner Neural Architecture Search ENAS DARTS P-DARTS Network Morphism Model Compression Pruning AGP Pruner Slim Pruner FPGM Pruner Quantization QAT Quantizer DoReFa Quantizer Feature Engineering (Beta) GradientFeatureSelector GBDTSelector Early Stop Algorithms Median Stop Curve Fitting	Local Machine Remote Servers Kubernetes based services OpenPAI Kubeflow FrameworkController on K8S (AKS etc.)
References	Python API NNI Annotation Supported OS	CustomizeTuner CustomizeAssessor	Support TrainingService Implement TrainingService

Install & Verify

Install through pip

• We support Linux, MacOS and Windows (local, remote and pai mode) in current stage, Ubuntu 16.04 or higher, MacOS 10.14.1 along with Windows 10.1809 are tested and supported. Simply run the following pip install in an environment that has python >= 3.5.

Linux and MacOS

python3 -m pip install --upgrade nni

Windows

python -m pip install --upgrade nni

Note:

• --user can be added if you want to install NNI in your home directory, which does not require any special privileges.
• Currently NNI on Windows support local, remote and pai mode. Anaconda or Miniconda is highly recommended to install NNI on Windows.
• If there is any error like Segmentation fault, please refer to FAQ

Install through source code

• We support Linux (Ubuntu 16.04 or higher), MacOS (10.14.1) and Windows (10.1809) in our current stage.

Linux and MacOS

• Run the following commands in an environment that has python >= 3.5, git and wget.

    git clone -b v1.2 https://github.com/Microsoft/nni.git
    cd nni
    source install.sh

Windows

• Run the following commands in an environment that has python >=3.5, git and PowerShell

  git clone -b v1.2 https://github.com/Microsoft/nni.git
  cd nni
  powershell -ExecutionPolicy Bypass -file install.ps1

For the system requirements of NNI, please refer to Install NNI

For NNI on Windows, please refer to NNI on Windows

Verify install

The following example is an experiment built on TensorFlow. Make sure you have TensorFlow 1.x installed before running it. Note that currently Tensorflow 2.0 is NOT supported.

• Download the examples via clone the source code.

    git clone -b v1.2 https://github.com/Microsoft/nni.git

Linux and MacOS

• Run the MNIST example.

    nnictl create --config nni/examples/trials/mnist-tfv1/config.yml

Windows

• Run the MNIST example.

    nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml

• Wait for the message INFO: Successfully started experiment! in the command line. This message indicates that your experiment has been successfully started. You can explore the experiment using the Web UI url.

INFO: Starting restful server...
INFO: Successfully started Restful server!
INFO: Setting local config...
INFO: Successfully set local config!
INFO: Starting experiment...
INFO: Successfully started experiment!
-----------------------------------------------------------------------
The experiment id is egchD4qy
The Web UI urls are: http://223.255.255.1:8080   http://127.0.0.1:8080
-----------------------------------------------------------------------

You can use these commands to get more information about the experiment
-----------------------------------------------------------------------
         commands                       description
1. nnictl experiment show        show the information of experiments
2. nnictl trial ls               list all of trial jobs
3. nnictl top                    monitor the status of running experiments
4. nnictl log stderr             show stderr log content
5. nnictl log stdout             show stdout log content
6. nnictl stop                   stop an experiment
7. nnictl trial kill             kill a trial job by id
8. nnictl --help                 get help information about nnictl
-----------------------------------------------------------------------

• Open the Web UI url in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. Here are more Web UI pages.

Documentation

• To learn about what's NNI, read the NNI Overview.
• To get yourself familiar with how to use NNI, read the documentation.
• To get started and install NNI on your system, please refer to Install NNI.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

After getting familiar with contribution agreements, you are ready to create your first PR =), follow the NNI developer tutorials to get start:

• We recommend new contributors to start with 'good first issue' or 'help-wanted', these issues are simple and easy to start.
• NNI developer environment installation tutorial
• How to debug
• Customize your own Tuner
• Implement customized TrainingService
• Implement a new NAS trainer on NNI
• Customize your own Advisor

External Repositories and References

With authors' permission, we listed a set of NNI usage examples and relevant articles.

• External Repositories

  • Run ENAS with NNI
  • Run Neural Network Architecture Search with NNI
  • Automatic Feature Engineering with NNI
  • Hyperparameter Tuning for Matrix Factorization with NNI
  • scikit-nni Hyper-parameter search for scikit-learn pipelines using NNI

• Relevant Articles

  • Hyper Parameter Optimization Comparison
  • Neural Architecture Search Comparison
  • Parallelizing a Sequential Algorithm TPE
  • Automatically tuning SVD with NNI
  • Automatically tuning SPTAG with NNI
  • Find thy hyper-parameters for scikit-learn pipelines using Microsoft NNI
  • Blog (in Chinese) - AutoML tools (Advisor, NNI and Google Vizier) comparison by @gaocegege - 总结与分析 section of design and implementation of kubeflow/katib

Feedback

• Discuss on the NNI Gitter in NNI.
• File an issue on GitHub.
• Ask a question with NNI tags on Stack Overflow.

Related Projects

Targeting at openness and advancing state-of-art technology, Microsoft Research (MSR) had also released few other open source projects.

• OpenPAI : an open source platform that provides complete AI model training and resource management capabilities, it is easy to extend and supports on-premise, cloud and hybrid environments in various scale.
• FrameworkController : an open source general-purpose Kubernetes Pod Controller that orchestrate all kinds of applications on Kubernetes by a single controller.
• MMdnn : A comprehensive, cross-framework solution to convert, visualize and diagnose deep neural network models. The "MM" in MMdnn stands for model management and "dnn" is an acronym for deep neural network.
• SPTAG : Space Partition Tree And Graph (SPTAG) is an open source library for large scale vector approximate nearest neighbor search scenario.

We encourage researchers and students leverage these projects to accelerate the AI development and research.

License

The entire codebase is under MIT license