question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

LUA script doesn't return evaljs content on Ubuntu 16.04.

See original GitHub issue

I am trying to scrap the HTML and the Window object from site using Splash V.3.3.1. Locally on OSX environment it works perfectly but when I deploy to Ubuntu 16.04 production server LUA scripts return the object only with ‘html’ key.

def parse(self, response):
    print(response.data)

Result:

{'html': 'ALL HTML'}

Script:

    def start_requests(self):
        """Starting place for request."""
        script = """
        function main(splash, args)
          assert(splash:go(args.url))
          assert(splash:wait(0.5))
          local data = splash:evaljs("window.typeOfObject")
          return {
            js = data,
            html = splash:html()
          }
        end
        """
        yield SplashRequest(self.url, self.parse, endpoint='execute', args={
            'wait': 1,
            'proxy': 'http://localhost:24000',
            'lua_source': script
        })

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:9 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
GaganBhatcommented, Oct 7, 2020

Did you ever find a solution @laranicolas ? Having the same exact issue here.

0reactions
laranicolascommented, May 23, 2019

When looking more specific under window js object

script = """
        function main(splash, args)
          assert(splash:go(args.url))
          assert(splash:wait(0.5))
          local data = splash:evaljs("window. typeOfObject.data.overview)
          return {
            js = data,
            html = splash:html()
          }
        end
        """

it raises undefined error:

2019-05-23 19:10:30 [scrapy_splash.middleware] WARNING: Bad request to Splash: {'info': {'line_number': 5, 'error': 'JS error: "TypeError: undefined is not an object (evaluating \'window. typeOfObject.data.overview\')"', 'message': '[string "..."]:5: JS error: "TypeError: undefined is not an object (evaluating \'window. typeOfObject.data.overview\')"', 'source': '[string "..."]', 'js_error': "TypeError: undefined is not an object (evaluating 'window. typeOfObject.data.overview')", 'splash_method': 'evaljs', 'type': 'JS_ERROR', 'js_error_type': 'TypeError', 'js_error_message': "undefined is not an object (evaluating 'window. typeOfObject.data.overview')"}, 'description': 'Error happened while executing Lua script', 'error': 400, 'type': 'ScriptError'}

but if I did on site it show the information using same js variable, also if I run locally it works…, really I don’t know what could be?

Read more comments on GitHub >

github_iconTop Results From Across the Web

eval() - JavaScript - MDN Web Docs - Mozilla
The eval() function evaluates JavaScript code represented as a string and returns its completion value. The source is parsed as a script.
Read more >
Splash Scripts Reference — Splash 1.0 documentation
The rule of thumb: if an argument or a return value can be serialized via JSON, then it is fine. If a JavaScript...
Read more >
Why is using the JavaScript eval function a bad idea?
1. While this is true -- if your content isn't dynamic, what reason is there to use eval for it at all? You...
Read more >
openresty/lua-nginx-module: Embed the Power of ... - GitHub
At that time ngx_lua can calculate the total length of the body and construct a proper Content-Length header to return to the HTTP...
Read more >
Stealthily Backdooring CMS Through Redis' Memory Space ...
Redis can execute sandboxed Lua scripts through the “EVAL” command. dofile() is a command that can be used to enumerate files and directories....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found