Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Both capture and capture_or_cache fail and return errors

See original GitHub issue

Both capture and capture_or_cache fail and return errors.

# savepagenow 0.0.13
$ eval "python3 -c 'from savepagenow import*;"{capture,capture_or_cache}"(\"http://www.example.com/\")';"|&cat>>log
$ cat log

capture:

Traceback (most recent call last):
  File "/home/eggplants/.local/lib/python3.6/site-packages/savepagenow/api.py", line 50, in capture
    archive_id = response.headers['Content-Location']
  File "/home/eggplants/.local/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__
    return self._store[key.lower()][1]
KeyError: 'content-location'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/eggplants/.local/lib/python3.6/site-packages/savepagenow/api.py", line 53, in capture
    raise WaybackRuntimeError(dict(status_code=response.status_code, headers=response.headers))
savepagenow.api.WaybackRuntimeError:

{
  "status_code":200,
  "headers":{
    "Server":"nginx/1.15.8",
    "Date":"Sat, 11 Jul 2020 19:04:09 GMT",
    "Content-Type":"text/html; charset=UTF-8",
    "Transfer-Encoding":"chunked",
    "Connection":"keep-alive",
    "X-Archive-Orig-Content-Encoding":"gzip",
    "X-Archive-Orig-Age":"439714",
    "X-Archive-Orig-Cache-Control":"max-age=604800",
    "X-Archive-Orig-Date":"Sat, 11 Jul 2020 18:58:06 GMT",
    "X-Archive-Orig-Etag":"\"3147526947+gzip\"",
    "X-Archive-Orig-Expires":"Sat, 18 Jul 2020 18:58:06 GMT",
    "X-Archive-Orig-Last-Modified":"Thu, 17 Oct 2019 07:18:26 GMT",
    "X-Archive-Orig-Server":"ECS (sjc/4E76)",
    "X-Archive-Orig-Vary":"Accept-Encoding",
    "X-Archive-Orig-X-Cache":"HIT",
    "X-Archive-Orig-Content-Length":"648",
    "X-Archive-Guessed-Content-Type":"text/html",
    "X-Archive-Guessed-Charset":"utf-8",
    "Memento-Datetime":"Sat, 11 Jul 2020 18:58:06 GMT",
    "Link":"<http://www.example.com/>; rel=\"original\", <https://web.archive.org/web/timemap/link/http://www.example.com/>; rel=\"timemap\"; type=\"application/link-format\", <https://web.archive.org/web/http://www.example.com/>; rel=\"timegate\", <https://web.archive.org/web/20020120142510/http://example.com:80/>; rel=\"first memento\"; datetime=\"Sun, 20 Jan 2002 14:25:10 GMT\", <https://web.archive.org/web/20200711183038/http://example.com/>; rel=\"prev memento\"; datetime=\"Sat, 11 Jul 2020 18:30:38 GMT\", <https://web.archive.org/web/20200711185806/http://www.example.com/>; rel=\"memento\"; datetime=\"Sat, 11 Jul 2020 18:58:06 GMT\", <https://web.archive.org/web/20200711185806/http://www.example.com/>; rel=\"last memento\"; datetime=\"Sat, 11 Jul 2020 18:58:06 GMT\"",
    "Content-Security-Policy":"default-src 'self' 'unsafe-eval' 'unsafe-inline' data: blob: archive.org web.archive.org analytics.archive.org pragma.archivelab.org",
    "X-Archive-Src":"spn2-20200711181600-wwwb-spn14.us.archive.org-8002.warc.gz",
    "Server-Timing":"load_resource;dur=89.916878, exclusion.robots;dur=1.158545, exclusion.robots.policy;dur=1.071748, CDXLines.iter;dur=150.374492, RedisCDXSource;dur=59.431015, LoadShardBlock;dur=617.530887, esindex;dur=0.083691, PetaboxLoader3.resolve;dur=56.383628, PetaboxLoader3.datanode;dur=482.086999",
    "X-App-Server":"wwwb-app29",
    "X-ts":"200",
    "X-location":"All",
    "X-Cache-Key":"httpsweb.archive.org/web/20200711185806/http://www.example.com/JP",
    "X-Page-Cache":"HIT",
    "Content-Encoding":"gzip"
  }
}

capture_or_cache:

Traceback (most recent call last):
  File "/home/eggplants/.local/lib/python3.6/site-packages/savepagenow/api.py", line 50, in capture
    archive_id = response.headers['Content-Location']
  File "/home/eggplants/.local/lib/python3.6/site-packages/requests/structures.py", line 54, in __getitem__
    return self._store[key.lower()][1]
KeyError: 'content-location'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/eggplants/.local/lib/python3.6/site-packages/savepagenow/api.py", line 84, in capture_or_cache
    return capture(target_url, user_agent=user_agent, accept_cache=False), True
  File "/home/eggplants/.local/lib/python3.6/site-packages/savepagenow/api.py", line 53, in capture
    raise WaybackRuntimeError(dict(status_code=response.status_code, headers=response.headers))
savepagenow.api.WaybackRuntimeError:

{
  "status_code":200,
  "headers":{
    "Server":"nginx/1.15.8",
    "Date":"Sat, 11 Jul 2020 19:04:32 GMT",
    "Content-Type":"text/html; charset=UTF-8",
    "Transfer-Encoding":"chunked",
    "Connection":"keep-alive",
    "X-Archive-Orig-Content-Encoding":"gzip",
    "X-Archive-Orig-Age":"439714",
    "X-Archive-Orig-Cache-Control":"max-age=604800",
    "X-Archive-Orig-Date":"Sat, 11 Jul 2020 18:58:06 GMT",
    "X-Archive-Orig-Etag":"\"3147526947+gzip\"",
    "X-Archive-Orig-Expires":"Sat, 18 Jul 2020 18:58:06 GMT",
    "X-Archive-Orig-Last-Modified":"Thu, 17 Oct 2019 07:18:26 GMT",
    "X-Archive-Orig-Server":"ECS (sjc/4E76)",
    "X-Archive-Orig-Vary":"Accept-Encoding",
    "X-Archive-Orig-X-Cache":"HIT",
    "X-Archive-Orig-Content-Length":"648",
    "X-Archive-Guessed-Content-Type":"text/html",
    "X-Archive-Guessed-Charset":"utf-8",
    "Memento-Datetime":"Sat, 11 Jul 2020 18:58:06 GMT",
    "Link":"<http://www.example.com/>; rel=\"original\", <https://web.archive.org/web/timemap/link/http://www.example.com/>; rel=\"timemap\"; type=\"application/link-format\", <https://web.archive.org/web/http://www.example.com/>; rel=\"timegate\", <https://web.archive.org/web/20020120142510/http://example.com:80/>; rel=\"first memento\"; datetime=\"Sun, 20 Jan 2002 14:25:10 GMT\", <https://web.archive.org/web/20200711183038/http://example.com/>; rel=\"prev memento\"; datetime=\"Sat, 11 Jul 2020 18:30:38 GMT\", <https://web.archive.org/web/20200711185806/http://www.example.com/>; rel=\"memento\"; datetime=\"Sat, 11 Jul 2020 18:58:06 GMT\", <https://web.archive.org/web/20200711185806/http://www.example.com/>; rel=\"last memento\"; datetime=\"Sat, 11 Jul 2020 18:58:06 GMT\"",
    "Content-Security-Policy":"default-src 'self' 'unsafe-eval' 'unsafe-inline' data: blob: archive.org web.archive.org analytics.archive.org pragma.archivelab.org",
    "X-Archive-Src":"spn2-20200711181600-wwwb-spn14.us.archive.org-8002.warc.gz",
    "Server-Timing":"load_resource;dur=89.916878, exclusion.robots;dur=1.158545, exclusion.robots.policy;dur=1.071748, CDXLines.iter;dur=150.374492, RedisCDXSource;dur=59.431015, LoadShardBlock;dur=617.530887, esindex;dur=0.083691, PetaboxLoader3.resolve;dur=56.383628, PetaboxLoader3.datanode;dur=482.086999",
    "X-App-Server":"wwwb-app29",
    "X-ts":"200",
    "X-location":"All",
    "X-Cache-Key":"httpsweb.archive.org/web/20200711185806/http://www.example.com/JP",
    "X-Page-Cache":"HIT",
    "Content-Encoding":"gzip"
  }
}

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:13 (5 by maintainers)

github_iconTop GitHub Comments

3reactions
palewirecommented, Jul 12, 2020

New version is out with the fix. Let me know if it doesn’t work for you.

https://pypi.org/project/savepagenow/1.0.0/

3reactions
palewirecommented, Jul 12, 2020

Okay. I’ve rewritten the core capture function to use the Link header and it appears to be working in my local branch.

https://github.com/pastpages/savepagenow/blob/0422475e509dbc9b254a820534d6f2c9cf77824e/savepagenow/api.py#L54-L76

I am going to do a little more testing, drop Python 2 support, tidy up a couple other things. If it all goes well I’ll try to release a new version to PyPI this morning.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Handling operation errors - Apollo GraphQL Docs
graphQLErrors is not populated), and any returned data is cached and rendered as if no errors occurred. all, Both data and error.graphQLErrors are...
Read more >
Error handling with DynamoDB - AWS Documentation
If the request is unsuccessful, DynamoDB returns an error. ... In this code example, the try-catch construct handles two different kinds of exceptions:....
Read more >
8. Errors and Exceptions — Python 3.11.1 documentation
Exceptions¶. Even if a statement or expression is syntactically correct, it may cause an error when an attempt is made to execute it....
Read more >
Error handling, "try...catch" - The Modern JavaScript Tutorial
Both measurements are done correctly. In other words, the function may finish with return or throw , that doesn't matter. The finally clause ......
Read more >
Error Handling — The Swift Programming Language (Swift 5.7)
You can propagate the error from a function to the code that calls that function, handle the error using a do - catch...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

KeyError: 'link' in capture

Next page

'content-location' error