How to run ES packages in CommonJS modules

We recently got a lot of headache caused by migration of some popular dependency packages in our stack from CommonJS to ES modules. A good example of that trend is the unified eco-system which recently moved hundreds of packages into ES without providing CommonJS builds.

As CommonJS modules cannot simply require() ES packages, we faced the following choices:

1. keep old versions of dependencies. That was OK short term but as the whole stack gets updated, with moving to newer node version, sooner or later something would break and it would be impossible to keep that approach.

2. move entire application to ES - we don't have unlimited resources and spending time refactoring code without any functional gains is last on our list of priorities.

3. tactically integrate ES modules into CommonJS code. This is what we went for.

The import() function

You can import ES modules to CommonJS modules using async import() function. The trick is that you need to write code around the async function to accommodate the asynchronous flow.

Here is a quick example of how we achieved that with a piece of code build on unified modules to parse markdown files.

Step 1 - move ES dependency modules into a standalone internal ES module

Here we simply take the "new" ES dependencies and assemble a new unified.mjs ES internal module with them:

import { unified } from 'unified'
import markdown from 'remark-parse'
import remarkGfm from 'remark-gfm'
import remarkRehype from 'remark-rehype'
import rehypeRaw from 'rehype-raw'
import hrehypeStringify from 'rehype-stringify'
import highlight from 'rehype-highlight'
import langHttp from 'highlight.js/lib/languages/http'
import langNginx from 'highlight.js/lib/languages/nginx'

const languages = {
  http: langHttp,
  nginx: langNginx
}

export const processor = unified()
  .use(markdown)
  .use(remarkGfm)
  .use(remarkRehype, { allowDangerousHtml: true })
  .use(rehypeRaw)
  .use(highlight, { languages: languages })
  .use(hrehypeStringify)

Note the .mjs extension - Node 14+ will automatically know to treat this file as an ES module.

Step 2 - use async import() in the legacy CommonJS code

Now we can async import() our internal ES module to the CommonJS module like this:

async function getProcessor () {
  const { processor } = await import('./unified.mjs')
  return processor
}

// we have an express app
router.get('/', async function (req, res, next) {
...
  const processor = await getProcessor()
...
})

It is a bit painful compared to simple require or import directives in CommonJS and ES but it works and it allows you to use ES dependencies in old legacy CommonJS modules.

We would prefer, of course, to fetch CommonJS dependencies directly, but unfortunately many package maintainers are moving to ES without providing secondary builds. As we are not ready to migrate our entire codebase to ES, we are stuck with this tactical workaround for now.