"Unexpected token" error - Drop Esprima dependency?

Technically this is an Esprima bug, but I do report this here because Esprima seems to be outdated in general and does not support modern JavaScript. I’ve heard about the Fork Espree, but thats just a suggestion.

Expected Behavior

ui5 build should compile modern JavaScript code.

Current Behavior

The build fails if it encounters a spread operator.

Steps to reproduce the issue

1. Copy the following snippet in any UI5 project

const foo = {
  1: 1, 2: 2, 3: 3
}
const bar = {
  ...foo
}

1. run ui5 build
2. It fails with the log below. The process returns 0 and states “Build succeeded”, but the parsing error happens and the WebIDE cancels the build process.

Context

• UI5 Module Version (output of ui5 --version when using the CLI): 2.4.2
• Node.js Version: v12.10.0
• npm Version: 6.13.4
• OS/Platform: Windows and WebIDE

Affected components (if known)

• ui5-builder
• ui5-server
• ui5-cli
• ui5-fs
• ui5-project
• ui5-logger

Log Output / Stack Trace

info normalizer:translators:ui5Framework Using OpenUI5 version: 1.81.2
info builder:builder Building project openui5-sample-app not including dependencies...
info builder:builder �   (1/1) Building project openui5-sample-app
info builder:builder application openui5-sample-app � (1/8) Running task escapeNonAsciiCharacters...
info builder:builder application openui5-sample-app � (2/8) Running task replaceCopyright...
info builder:builder application openui5-sample-app � (3/8) Running task replaceVersion...
info builder:builder application openui5-sample-app � (4/8) Running task generateFlexChangesBundle...
info builder:builder application openui5-sample-app � (5/8) Running task generateComponentPreload...
ERR! lbt:resources:ResourcePool Error: Line 17: Unexpected token ...
ERR! lbt:resources:ResourcePool     at ErrorHandler.constructError (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:5012:22)
ERR! lbt:resources:ResourcePool     at ErrorHandler.createError (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:5028:27)
ERR! lbt:resources:ResourcePool     at Parser.unexpectedTokenError (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:1985:39)
ERR! lbt:resources:ResourcePool     at Parser.throwUnexpectedToken (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:1995:21)
ERR! lbt:resources:ResourcePool     at Parser.parseObjectPropertyKey (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2499:33)
ERR! lbt:resources:ResourcePool     at Parser.parseObjectProperty (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2534:25)
ERR! lbt:resources:ResourcePool     at Parser.parseObjectInitializer (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2602:35)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parsePrimaryExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2354:38)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parseLeftHandSideExpressionAllowCall (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2899:26)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parseUpdateExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:3004:26)
ERR! lbt:resources:ResourcePool     at Parser.parseUnaryExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:3048:26)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parseExponentiationExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:3054:26)
ERR! lbt:resources:ResourcePool  failed to parse or analyze %s: sap/ui/demo/todo/controller/App.controller.js Error: Line 17: Unexpected token ...
ERR! lbt:resources:ResourcePool     at ErrorHandler.constructError (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:5012:22)
ERR! lbt:resources:ResourcePool     at ErrorHandler.createError (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:5028:27)
ERR! lbt:resources:ResourcePool     at Parser.unexpectedTokenError (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:1985:39)
ERR! lbt:resources:ResourcePool     at Parser.throwUnexpectedToken (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:1995:21)
ERR! lbt:resources:ResourcePool     at Parser.parseObjectPropertyKey (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2499:33)
ERR! lbt:resources:ResourcePool     at Parser.parseObjectProperty (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2534:25)
ERR! lbt:resources:ResourcePool     at Parser.parseObjectInitializer (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2602:35)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parsePrimaryExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2354:38)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parseLeftHandSideExpressionAllowCall (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2899:26)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parseUpdateExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:3004:26)
ERR! lbt:resources:ResourcePool     at Parser.parseUnaryExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:3048:26)
ERR! lbt:resources:ResourcePool     at Parser.inheritCoverGrammar (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:2285:37)
ERR! lbt:resources:ResourcePool     at Parser.parseExponentiationExpression (xxxnode_modules\@ui5\cli\node_modules\esprima\dist\esprima.js:3054:26) {
ERR! lbt:resources:ResourcePool   index: 372,
ERR! lbt:resources:ResourcePool   lineNumber: 17,
ERR! lbt:resources:ResourcePool   description: 'Unexpected token ...',
ERR! lbt:resources:ResourcePool   stack: 'Error: Line 17: Unexpected token ...\n' +
ERR! lbt:resources:ResourcePool     '    at ErrorHandler.constructError (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:5012:22)\n' +
ERR! lbt:resources:ResourcePool     '    at ErrorHandler.createError (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:5028:27)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.unexpectedTokenError (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:1985:39)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.throwUnexpectedToken (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:1995:21)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parseObjectPropertyKey (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2499:33)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parseObjectProperty (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2534:25)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parseObjectInitializer (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2602:35)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.inheritCoverGrammar (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2285:37)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parsePrimaryExpression (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2354:38)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.inheritCoverGrammar (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2285:37)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parseLeftHandSideExpressionAllowCall (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2899:26)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.inheritCoverGrammar (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2285:37)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parseUpdateExpression (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:3004:26)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parseUnaryExpression (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:3048:26)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.inheritCoverGrammar (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:2285:37)\n' +
ERR! lbt:resources:ResourcePool     '    at Parser.parseExponentiationExpression (xxxnode_modules\\@ui5\\cli\\node_modules\\esprima\\dist\\esprima.js:3054:26)'
ERR! lbt:resources:ResourcePool }
info builder:builder application openui5-sample-app � (6/8) Running task createDebugFiles...
info builder:builder application openui5-sample-app � (7/8) Running task uglify...
info builder:builder application openui5-sample-app � (8/8) Running task generateVersionInfo...
info builder:builder Build succeeded in 748 ms
info builder:builder Executing cleanup tasks...

Process finished with exit code 0

[RandomByte]

Thanks for reporting. Yes, with the recent terser upgrade, Esprima (used in our preload generation) is currently the only piece that is still lacking ES6 support.

A switch to Espree seems reasonable. We’ll discuss this in the team. I would be surprised if @codeworrior hasn’t already looked into the differences between Esprima and Espree and the possible issues these could cause in our current implementation 😉

[RandomByte]

We have switched parsers to Espree v6. We can’t use any newer versions, because they require a higher version of Node.js than what UI5 Tooling currently supports.

This means some ES6 features are still not supported:

• 75e80bc Update: support ?? operator, import.meta, and export * as ns
• 872645c Update: support optional chaining
• bd0a405 Update: support logical assignment and numeric separators

We will probably add support for those by upgrading Espree with the UI5 Tooling 3.0 release.

However, the “Unexpected token” error for spread operators reported in this issue is now resolved with UI5 CLI v2.11.2 🎉