https://github.com/IntelLabs/hpat

HPAT

A compiler-based framework for big data in Python

High Performance Analytics Toolkit (HPAT) is a compiler-based framework for big data analytics and machine learning on cluster/cloud environments that is both easy to use and extremely fast; it is orders of magnitude faster than alternatives like Apache Spark.

HPAT automatically or semi-automatically parallelizes analytics tasks written in Numpy and pandas and generates efficient MPI code using Numba and LLVM. These academic papers describe the underlying methods in HPAT:

• HPAT paper at ICS'17
• HPAT at HotOS'17
• HiFrames on arxiv

HPAT's documentation can be found here.