Performance Comparison of SWC and Babel

January 31st, 2020 by DongYoon Kang

JavaScript is single-threaded. The JS thread is not a good place to do heavy computation. Let's talk about babel and swc, which are both computation-heavy.

Synchronous Benchmark

Let's do a benchmark for the single-core workload. Note this uses transformSync, which is rarely useful in the wild.

[transform]
  swc (es3) x 616 ops/sec ±4.36% (88 runs sampled)
  swc (es2015) x 677 ops/sec ±2.01% (90 runs sampled)
  swc (es2016) x 1,963 ops/sec ±0.45% (93 runs sampled)
  swc (es2017) x 1,971 ops/sec ±0.35% (94 runs sampled)
  swc (es2018) x 2,555 ops/sec ±0.35% (93 runs sampled)
  swc-optimize (es3) x 645 ops/sec ±0.40% (90 runs sampled)
  babel (es5) x 34.05 ops/sec ±1.15% (58 runs sampled)

SWC is very fast. Although swc (es3) does more work than babel (es5), swc (es3) is faster than babel (es5).

Real-World Benchmark

transformSync and transformFileSync are rarely used in the real world, as it blocks the current thread. await Promise.all() is frequently used as is better than:

for (const promise of promises) {
  await promise;
}

Let's create a benchmark for actual real-world usage using Promise.all().

Ideal Case

First, I created a benchmark for the ideal case. It invokes [n] promises at once where n is the number of physical CPU cores. See node-swc repository (opens in a new tab) for the full code.

const os = require("os");
const cpuCount = os.cpus().length;
 
const SOURCE = `
  // See the link above
`;
 
const SUITES = [
  // ...
  // See the link above
];
 
const arr = [];
for (let i = 0; i < cpuCount / 2; i++) {
  arr.push(0);
}
 
console.info(`CPU Core: ${cpuCount}; Parallelism: ${arr.length}`);
console.info(
  `Note that output of this benchmark should be multiplied by ${arr.length} as this test uses Promise.all`
);
 
SUITES.map(args => {
  const [name, requirePath, fn] = args;
  const func = fn.bind(null, require(requirePath));
  bench(name, async done => {
    await Promise.all(arr.map(v => func()));
    done();
  });
});

I ran benchmarks on my old desktop. It has E3-v1275 and 24GB of ram. The output below is copied as-is from benchmark output.

CPU Core: 8; Parallelism: 4
Note that output of this benchmark should be multiplied by 4 as this test uses Promise.all
[multicore]
swc (es3) x 426 ops/sec ±3.75% (73 runs sampled)
swc (es2015) x 422 ops/sec ±3.57% (74 runs sampled)
swc (es2016) x 987 ops/sec ±2.53% (75 runs sampled)
swc (es2017) x 987 ops/sec ±3.44% (75 runs sampled)
swc (es2018) x 1,221 ops/sec ±2.46% (77 runs sampled)
swc-optimize (es3) x 429 ops/sec ±1.94% (82 runs sampled)
babel (es5) x 6.82 ops/sec ±17.18% (40 runs sampled)

Now, we need to multiply it by 4, as we do 4 operations per iteration.

swc (es3) x 1704 ops/sec ±3.75% (73 runs sampled)
swc (es2015) x 1688 ops/sec ±3.57% (74 runs sampled)
swc (es2016) x 3948 ops/sec ±2.53% (75 runs sampled)
swc (es2017) x 3948 ops/sec ±3.44% (75 runs sampled)
swc (es2018) x 4884 ops/sec ±2.46% (77 runs sampled)
swc-optimize (es3) x 1716 ops/sec ±1.94% (82 runs sampled)
babel (es5) x 27.28 ops/sec ±17.18% (40 runs sampled)

This is an actual result.

The performance of babel (es5) is dropped. Async is not free. Even though, 34.05 ops/sec => 27.28 ops/sec is much better than I expected.

Benchmark for Many Operations

I modified the benchmark file slightly to make it creates 100 promises per iteration.

CPU Core: 8; Parallelism: 100
Note that output of this benchmark should be multiplied by 100 as this test uses Promise.all
[multicore]
  swc (es3) x 21.99 ops/sec ±1.83% (54 runs sampled)
  swc (es2015) x 19.11 ops/sec ±3.39% (48 runs sampled)
  swc (es2016) x 55.80 ops/sec ±6.97% (71 runs sampled)
  swc (es2017) x 62.59 ops/sec ±2.12% (74 runs sampled)
  swc (es2018) x 81.08 ops/sec ±5.22% (75 runs sampled)
  swc-optimize (es3) x 18.60 ops/sec ±2.13% (50 runs sampled)
  babel (es5) x 0.32 ops/sec ±19.10% (6 runs sampled)

It must be multiplied by 100 as above.

  swc (es3) x 2199 ops/sec ±1.83% (54 runs sampled)
  swc (es2015) x 1911 ops/sec ±3.39% (48 runs sampled)
  swc (es2016) x 5580 ops/sec ±6.97% (71 runs sampled)
  swc (es2017) x 6259 ops/sec ±2.12% (74 runs sampled)
  swc (es2018) x 8108 ops/sec ±5.22% (75 runs sampled)
  swc-optimize (es3) x 1860 ops/sec ±2.13% (50 runs sampled)
  babel (es5) x 32 ops/sec ±19.10% (6 runs sampled)

Why does the performance of SWC not drop drastically? The secret is Node.js. Node.js internally manages a worker thread pool, and SWC runs on it. Thus, even though you create 100 promises at once, the number of worker threads is much smaller than it.

Conclusion

name	1 core, sync	4 promises	100 promises
swc (es3)	616 ops/sec	1704 ops/sec	2199 ops/sec
swc (es2015)	677 ops/sec	1688 ops/sec	1911 ops/sec
swc (es2016)	1963 ops/sec	3948 op s/sec	5580 ops/sec
swc (es2017)	1971 ops/sec	3948 ops/sec	6259 ops/sec
swc (es2018)	2555 ops/sec	4884 ops/sec	8108 ops/sec
swc-optimize (es3)	645 ops/sec	1716 ops/sec	1860 ops/sec
babel (es5)	34.05 ops/sec	27.28 ops/sec	32 ops/sec

swc scales well, as it does almost all work in the worker thread. From the fact that the throughput of 100 promises was better than 4 promises, we can conclude that the worker thread pool of Node.js utilizes hyperthreading.

swc scales up with the number of cpu cores. Promise.all is enough for scaling.

Try SWC Introducing SWC 1.0