Python
Javascript for refined palates: a Python 3 to ES6 Javascript translator

javascripthon: a Python 3 to ES6 JavaScript translator

author:	Alberto Berti
contact:	alberto@metapensiero.it
license:	GNU General Public License version 3 or later

It is based on previous work by Andrew Schaaf.

Goal

JavaScripthon is a small and simple Python 3.5+ translator to JavaScript which aims to be able to translate most of the Python's core semantics without providing a full python-in-js environment, as most existing translators do. It tries to emit code which is simple to read and check and it does so by switching to ES6 construct when necessary. This allows to simplify the needs of polyfills for many of the expected Python behaviors.

The interface with the js world is completely flat, import the modules or use the expected globals (window, document, etc...) as you would do in JavaScript.

The ES6 code is then converted (if requested) to ES5 code with the aid of the popular BabelJS library together with the fantastic dukpy embedded js interpreter.

Another goal is to just convert single modules or entire dir tree structures without emitting concatenated or minified files. This is left to the Javascript tooling of your choice. I use webpack which has BabelJS integration to getting this job done. Check out the bundled example.

JavaScripthon also generates SourceMap files with the higher detail possible in order to aid development. This means that while you are debugging some piece of translated JavaScript with the browser's tools, you can actually choose to follow the flow of the code on the original Pyhton 3 source.

This project is far from complete, but it has achieved a good deal of features, please have a look at tests/test_evaljs.py file for the currently supported ones.

Installation

Python 3.5 is required because Python's ast has changed between 3.4 and 3.5 and as of now supporting multiple Python versions is not one of my priorities.

To install the package execute the following command:

$ pip install javascripthon

Usage

To compile or transpile a python source module, use the commandline:

$ python -m metapensiero.pj source.py

or:

$ python -m metapensiero.pj -5 source.py

to transpile.

A pj console script is also automatically installed:

$ pj --help
usage: pj [-h] [--disable-es6] [--disable-stage3] [-5] [-o OUTPUT] [-d]
          [--pdb]
          file [file ...]

A Python 3.5+ to ES6 JavaScript compiler

positional arguments:
  file                  Python source file(s) or directory(ies) to convert.
                        When it is a directory it will be converted
                        recursively

optional arguments:
  -h, --help            show this help message and exit
  --disable-es6         Disable ES6 features during conversion (Ignored if
                        --es5 is specified)
  --disable-stage3      Disable ES7 stage3 features during conversion
  -5, --es5             Also transpile to ES5 using BabelJS.
  -o OUTPUT, --output OUTPUT
                        Output file/directory where to save the generated code
  -d, --debug           Enable error reporting
  --pdb                 Enter post-mortem debug when an error occurs

Conversions Rosetta Stone

Here are brief list of examples of the conversions the tool applies, just some, but not all.

Simple stuff

x < y <= z < 5

def foo():
    return [True, False, None, 1729,
            "foo", r"foo\bar", {}]


while len(foo) > 0:
    print(foo.pop())


if foo > 0:
    ....
elif foo < 0:
    ....
else:
    ....

((x < y) && (y <= z) && (z < 5))

function foo() {
    return [true, false, null, 1729,
            "foo", "foo\\bar", {}];
}

while ((foo.length > 0)) {
    console.log(foo.pop());
}

if ((foo > 0)) {
    ....
} else {
    if ((foo < 0)) {
        ....
    } else {
        ....
    }
}

Then there are special cases. Here you can see some of these conversions. Javascripthon cannot do a full trace of the sources, so some shortcuts are taken about the conversion of some core, specific Python's semantics. For example Python's self is always converted to JavaScript's this, no matter where it's found. Or len(foo) is always translated to foo.length. Albeit this an api specific of just some objects (Strings, Arrays, etc...), it considered wide adopted and something the user may consider obvious.

The rules of thumb to treat things especially are:

• Is it possible to think of a conversion that covers most of the use cases?
• It's possible to find a convention widely used on the Python world to express this special case?

==
!=
2**3
'docstring'

self
len(...)
print(...)
isinstance(x, y)
typeof(x)

FirstCharCapitalized(...)















foo in bar

===
!==
Math.pow(2, 3)
/* docstring */

this
(...).length
console.log(...)
(x instanceof y)
(typeof x)

new FirstCharCapitalized(...)

var _pj;
function _pj_snippets(container) {
    function _in(left, right) {
        if (((right instanceof Array) || ((typeof right) === "string"))) {
            return (right.indexOf(left) > (- 1));
        } else {
            return (left in right);
        }
    }
    container["_in"] = _in;
    return container;
}
_pj = {};
_pj_snippets(_pj);
_pj._in(foo, bar);

for statement

The for statement by default is translated as if the object of the cycle is a list but has two special cases:

for el in dict(a_dict):
    print(el)





for el in an_array:
    print(el)




for i in range(5):
    print(i)

var _pj_a = a_dict;
for (var el in _pj_a) {
    if (_pj_a.hasOwnProperty(el)) {
        console.log(el);
    }
}

for (var el, _pj_c = 0, _pj_a = an_array, _pj_b = _pj_a.length;
      (_pj_c < _pj_b); _pj_c += 1) {
    el = _pj_a[_pj_c];
    console.log(el);
}

for (var i = 0, _pj_a = 5; (i < _pj_a); i += 1) {
    console.log(i);
}

Classes

Classes with single inheritance are translated to ES6 classes, they can have only function members for now, with no class or method decorators, because the ES7 spec for them is being rediscussed.

Methods can be functions or async-functions.

Python`s super() calls are converted accordingly to the type of their surrounding method: super().__init__(foo) becomes super(foo) in constructors.

Functions inside methods are translated to arrow functions so that they keep the this of the surrounding method.

Arrow method expression to retain the this at method level aren't implemented yet.

class Foo(bar):
    def __init__(self, zoo):
        super().__init__(zoo)

    def meth(self, zoo):
        super().meth(zoo)
        def cool(a, b, c):
            print(self.zoo)

    async def something(self, a_promise):
        result = await a_promise

class Foo extends bar {
    constructor(zoo) {
        super(zoo);
    }

    meth(zoo) {
        super.meth(zoo);
        var cool;
        cool = (a, b, c) => {
            console.log(this.zoo);
        };
    }

    async something(a_promise) {
        var result;
        result = await a_promise;
    }
}

Only direct descendants of Exception are threated especially, but just for them to be meaningful in js-land and to be detectable with instanceof in catch statements.

class MyError(Exception):
    pass

raise MyError("An error occurred")

function MyError(message) {
    this.name = "MyError";
    this.message = (message || "Custom error MyError");
    if (((typeof Error.captureStackTrace) === "function")) {
        Error.captureStackTrace(this, this.constructor);
    } else {
        this.stack = new Error(message).stack;
    }
}
MyError.prototype = Object.create(Error.prototype);
MyError.prototype.constructor = MyError;
throw new MyError("An error occurred");

try...except...finally statement

The conversion of this statement is mostly obvious with the only exception of the except part: it translates to a catch part containing one if statement for each non catchall except. If a catchall except is present, the error will be re-thrown, to mimic Python's behavior.

try:
    foo.bar()
except MyError:
    recover()
except MyOtherError:
    recover_bad()
finally:
    foo.on_end()

try {
    foo.bar();
} catch(e) {
    if ((e instanceof MyError)) {
        recover();
    } else {
        if ((e instanceof MyOtherError)) {
            recover_bad()
        } else {
            throw e;
        }
    }
} finally {
    foo.on_end();
}

import statements

import and from ... import statements are converted to ES6 imports, and the declaration of an __all__ member on the module top level is translated to ES6 exports.

import foo, bar
import foo.bar as b
from foo.bar import hello as h, bye as bb
from ..foo.zoo import bar
from . import foo
from .foo import bar

from __globals__ import test_name

# this should not trigger variable definition
test_name = 2

# this instead should do it
test_foo = True

__all__ = ['test_name', 'test_foo']

var test_foo;

import * as foo from 'foo';
import * as bar from 'bar';
import * as b from 'foo/bar';
import {hello as h, bye as bb} from 'foo/bar';
import {bar} from '../../foo/zoo';
import * as foo from './foo';
import {bar} from './foo';

test_name = 2;
test_foo = true;

export {test_name};
export {test_foo};

function's args and call parameters

Parmeters defaults and keyword parameters are supported and so is *foo accumulator, which is translated into the ES6 rest expression (...foo).

The only caveat is that really JS support for keyword args sucks, so you will have to remember to fill in all the arguments before specifying keywords.

def foo(a=2, b=3, *args):
    pass

def bar(c, d, *, zoo=2):
    pass


foo(5, *a_list)

bar('a', 'b', zoo=5, another='c')

function foo(a = 2, b = 3, ...args) {
}
function bar(c, d, {zoo = 2}={}) {
}
foo(5, ...a_list);
bar("a", "b", {zoo: 5, another: "c"});

Examples

Execute make inside the examples directory.

Testing

To run the tests you should run the following at the package root:

python setup.py test

Build status

Contributing

Any contribution is welcome, drop me a line or file a pull request.

Todo

This is a brief list of what needs to be done:

• refactor the comprehensions conversion to use the snippets facility;
• refactor snippets rendering to write them as a module and import them in the module when tree conversion is enabled;
• convert dict() calls to ES6 Map object creation. Also, update "foo in bar" to use bar.has(foo) for maps;
• convert set literals to ES6 Set objects. Also, update "foo in bar" to use bar.has(foo) for sets;
• multi-line strings to ES6 template strings (does this make any sense?);
• properties to ES6 properties (getter and setter);
• class and method decorators to ES7 class and method decorators;
• implement yield and generator functions;
• take advantage of new duckpy features to use a JS execution context that lasts multiple calls. This way the BabelJS bootstrap affects only the initial execution;

Done

Stuff that was previously in the todo:

• translate import statements to ES6;
• translate __all__ definition to ES6 module exports;
• write a command line interface to expose the api;
• make try...except work again and implement try...finally;
• convert async and await to the same proposed features for js (see BabelJS documentation);
• convert argument defaults on functions to ES6;
• convert call keyword arguments;
• convert *iterable syntax to ES6 destructuring;
• use arrow functions for functions created in functions;

External documentation

A good documentation and explanation of ES6 features can be found on the book Exploring ES6 by Axel Rauschmayer (donate if you can).

An extensive documentation about Python's ast objects, very handy.

Tools

Have a look at ECMAScript 6 Tools by Addy Osmani.

To debug source maps have a look at source-map-visualization and its package on npm.

Still i found these links to be helpful:

• BASE64 VLQ CODEC (COder/DECoder)
• Testing Source Maps

Here is an example of the latter tool showing code generated by JavaScripthon, have fun!

Notes

• A post about proposed solutions to use ES6 classes with Backbone. See also the bug open on github.

• A benchmark of ES6 features and discussion about it on hacker's news.

• A compatibility table of ES6 features showing completeness of support feature by feature.