github

Tencent / plato

  • вторник, 19 ноября 2019 г. в 00:30:10
https://github.com/Tencent/plato

C++
腾讯高性能图计算框架Plato



Plato(柏拉图)

A framework for distributed graph computation and machine learning at wechat scale, for more details, see 柏拉图简介 | Plato Introduction.

Authors(In alphabetical order): Benli Li, Conghui He, Donghai Yu, Pin Gao, Shijie Sun, Wenqiang Wu, Wanjing Wei, Xing Huang, Xiaogang Tu, Yongan Li.

Contact: plato@tencent.com

Special thanks to Xiaowei Zhu and many for their work Gemini. Several basic utility functions in Plato is derived from Gemini, the design principle of some dual-mode based algorithms in Plato is also heavily influenced by Gemini's dualmode-engine. Thanks to Ke Yang and many for their work KnightKing which served as foundation of plato's walk-engine.

Dependencies

To simplify installation, Plato currently downloads and builds most of its required dependencies by calling 3rdtools.sh. You should call it at least once before any build operations.

There are however, a few dependencies which must be manually satisfied.

  • GCC
    • At least 4.8.5 for C++11 support.
  • MPICH-3
    • Required for compiling and run Plato.
  • OpenMP
    • Required for compiling and run Plato.
  • Bazel-0.26
    • Required for compiling.

Environment

Plato was developed and tested on x86_64 cluster and Centos 7.0. Theoretically, it can be ported to other Linux distribution easily.

Build

BAZEL_LINKOPTS=-static-libstdc++ CC=/your_mpi_location/mpicxx bazel build example/...

Test

BAZEL_LINKOPTS=-static-libstdc++ CC=/your_mpi_location/mpicxx LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${PWD}/3rd/hadoop2/lib bazel test --test_env=LD_LIBRARY_PATH plato/...

Run

Prerequisite:

  1. A cluster which can submit MPI programs(Hydra is a feasible solution).
  2. An accessible HDFS where Plato can find its input and put output on it.

A sample submit script was locate in here, modify it based on your cluster's environment and run.

./scripts/run_pagerank.sh

Documents