github

Byaidu / PDFMathTranslate

  • суббота, 14 декабря 2024 г. в 00:00:01
https://github.com/Byaidu/PDFMathTranslate

PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译,支持 Google/DeepL/Ollama/OpenAI 等服务,提供 CLI/GUI/Docker



English | 简体中文

PDF2ZH

PDFMathTranslate

PDF scientific paper translation and bilingual comparison.

Feel free to provide feedback in GitHub Issues, Telegram Group or QQ Group.

Updates

  • [Nov. 26 2024] CLI now supports online file(s) (by @reycn)
  • [Nov. 24 2024] ONNX support to reduce dependency sizes (by @Wybxc)
  • [Nov. 23 2024] 🌟 Public Service online! (by @Byaidu)
  • [Nov. 23 2024] Firewall for preventing web bots (by @Byaidu)
  • [Nov. 22 2024] GUI now supports Italian, and has been improved (by @Byaidu, @reycn)
  • [Nov. 22 2024] You can now share your deployed service to others (by @Zxis233)
  • [Nov. 22 2024] Now supports Tencent Translation (by @hellofinch)
  • [Nov. 21 2024] GUI now supports downloading dual-document (by @reycn)
  • [Nov. 20 2024] 🌟 Demo online! (by @reycn)

Preview

Public Service 🌟

Free Service (https://pdf2zh.com/)

You can try our public service online without installation.

Hugging Face Demo

You can try our demo on HuggingFace without installation. Note that the computing resources of the demo are limited, so please avoid abusing them.

Installation and Usage

We provide three methods for using this project: Commandline, Portable, GUI, and Docker.

Method I. Commandline

  1. Python installed (3.8 <= version <= 3.12)

  2. Install our package:

    pip install pdf2zh
  3. Execute translation, files generated in current working directory:

    pdf2zh document.pdf

Method II. Portable

No need to pre-install Python environment

Download setup.bat and double-click to run

Method III. GUI

  1. Python installed (3.8 <= version <= 3.12)

  2. Install our package:

    pip install pdf2zh
  3. Start using in browser:

    pdf2zh -i
  4. If your browswer has not been started automatically, goto

    http://localhost:7860/

See documentation for GUI for more details.

Method IV. Docker

  1. Pull and run:

    docker pull byaidu/pdf2zh
    docker run -d -p 7860:7860 byaidu/pdf2zh
  2. Open in browser:

    http://localhost:7860/
    

For docker deployment on cloud service:

Advanced Options

Execute the translation command in the command line to generate the translated document example-mono.pdf and the bilingual document example-dual.pdf in the current working directory. Use Google as the default translation service.

cmd

In the following table, we list all advanced options for reference:

Option Function Example
files Local files pdf2zh ~/local.pdf
links Online files pdf2zh http://arxiv.org/paper.pdf
-i Enter GUI pdf2zh -i
-p Partial document translation pdf2zh example.pdf -p 1
-li Source language pdf2zh example.pdf -li en
-lo Target language pdf2zh example.pdf -lo zh
-s Translation service pdf2zh example.pdf -s deepl
-t Multi-threads pdf2zh example.pdf -t 1
-o Output dir pdf2zh example.pdf -o output
-f, -c Exceptions pdf2zh example.pdf -f "(MS.*)"
--share get gradio public link pdf2zh -i --share

Full / partial document translation

  • Entire document

    pdf2zh example.pdf
  • Part of the document

    pdf2zh example.pdf -p 1-3,5

Specify source and target languages

See Google Languages Codes, DeepL Languages Codes

pdf2zh example.pdf -li en -lo ja

Translate with Different Services

The table below outlines the required environment variables for each translation service. Make sure to set them before using the respective service.

Translator Service Environment Variables Default Values Notes
Google (Default) google None N/A None
Bing bing None N/A None
DeepL deepl DEEPL_SERVER_URL,DEEPL_AUTH_KEY https://api.deepl.com, [Your Key] See DeepL
DeepLX deeplx DEEPLX_ENDPOINT https://api.deepl.com/translate See DeepLX
Ollama ollama OLLAMA_HOST, OLLAMA_MODEL http://127.0.0.1:11434, gemma2 See Ollama
OpenAI openai OPENAI_BASE_URL, OPENAI_API_KEY, OPENAI_MODEL https://api.openai.com/v1, [Your Key], gpt-4o-mini See OpenAI
AzureOpenAI azure-openai AZURE_OPENAI_BASE_URL, AZURE_OPENAI_API_KEY, AZURE_OPENAI_MODEL [Your Endpoint], [Your Key], gpt-4o-mini See Azure OpenAI
Zhipu zhipu ZHIPU_API_KEY, ZHIPU_MODEL [Your Key], glm-4-flash See Zhipu
Silicon silicon SILICON_API_KEY, SILICON_MODEL [Your Key], Qwen/Qwen2.5-7B-Instruct See SiliconCloud
Azure azure AZURE_ENDPOINT, AZURE_API_KEY https://api.translator.azure.cn, [Your Key] See Azure
Tencent tencent TENCENTCLOUD_SECRET_ID, TENCENTCLOUD_SECRET_KEY [Your ID], [Your Key] See Tencent

Use -s service or -s service:model to specify service:

pdf2zh example.pdf -s openai:gpt-4o-mini

Or specify model with environment variables:

set OPENAI_MODEL=gpt-4o-mini
pdf2zh example.pdf -s openai

Translate wih exceptions

Use regex to specify formula fonts and characters that need to be preserved:

pdf2zh example.pdf -f "(CM[^RT].*|MS.*|.*Ital)" -c "(\(|\||\)|\+|=|\d|[\u0080-\ufaff])"

Specify threads

Use -t to specify how many threads to use in translation:

pdf2zh example.pdf -t 1

API

Python

from pdf2zh import translate, translate_stream

params = {"lang_in": "en", "lang_out": "zh", "service": "google", "thread": 4}
file_mono, file_dual = translate(files=["example.pdf"], **params)[0]
with open("example.pdf", "rb") as f:
    stream_mono, stream_dual = translate_stream(stream=f.read(), **params)

HTTP

pip install pdf2zh[backend]
pdf2zh --flask
pdf2zh --celery worker
curl http://localhost:11008/v1/translate -F "file=@example.pdf" -F "data={\"lang_in\":\"en\",\"l
ang_out\":\"zh\",\"service\":\"google\",\"thread\":4}"
{"id":"d9894125-2f4e-45ea-9d93-1a9068d2045a"}

curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a
{"info":{"n":13,"total":506},"state":"PROGRESS"}

curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a
{"state":"SUCCESS"}

curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a/mono --output example-mono.pdf

curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a/dual --output example-dual.pdf

curl http://localhost:11008/v1/translate/d9894125-2f4e-45ea-9d93-1a9068d2045a -X DELETE

TODO

  • Parse layout with DocLayNet based models, PaddleX, PaperMage, SAM2

  • Fix page rotation, table of contents, format of lists

  • Fix pixel formula in old papers

  • Async retry except KeyboardInterrupt

  • Knuth–Plass algorithm for western languages

  • Support non-PDF/A files

  • Plugins of Zotero and Obsidian

Acknowledgements

Contributors

Alt

Star History

Star History Chart