question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Please add context action for creating archives (tar, zip, maybe jar?)

See original GitHub issue

Creating archives is a common task of Skylark rules, but today it’s difficult to do correctly. Bazel itself packages a pkg_tar rule that (1) depends on Python and (2) generates non-deterministic output (https://github.com/bazelbuild/bazel/issues/1844). This has downstream effects on every rule built on pkg_tar (see pkg_deb bug https://github.com/bazelbuild/bazel/issues/3723), and complicates life for rule authors (see timestamp hoop-jumping in rules_docker).

Bazel already depends on Apache Commons Compress, which supports creating various types of archive. It should be straightforward to wrap their tar/zip logic with a repository action. Also useful is .ar for pkg_deb, and .xz because the XZ utilities are not installed by default on Mac OS.

Example of use:

def _some_rule(ctx):
  entry_a = ctx.actions.declare_archive_entry(
    input = ctx.file.some_file,
  )
  entry_b = ctx.actions.declare_archive_entry(
    input = ctx.file.some_other_file,
  )
  archive = ctx.actions.declare_file("big_blob_of_bytes.tar.gz")
  ctx.actions.write_archive(
    output = archive,
    inputs = [entry_a, entry_b],
  )

The declare_archive_entry method would have optional parameters to override the path, permissions, metadata, and so on.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
aiutocommented, Jul 19, 2019

I don’t know if having a built-in action is going to win. If we have remote execution, the action is running on the remote machine, so we would have to provide a binary portable to the execution environment. The blaze binary is not running there.

What I do support is adding a set of rules to rules_pkg that make defining complex package structures easier. At the bottom they can call tar or zip or a python equivalent for now. Over time we can replace these with faster (and more portable) compression tools. For reference, to any ex-Googlers, what I would like to do is export the pkgfilegroup() rule, but with a cleaned up design.

0reactions
jmillikincommented, Aug 8, 2019

Since Bazel’s built-in rules for C++/Java/etc are being moved out to their own rule sets, I agree that building this functionality as a Starlark rule instead of ctx.actions makes sense.

I’ll close this since https://github.com/bazelbuild/bazel/issues/8857 covers it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

7zip Add Operation Not Supported? - Super User
Use "Add to archive..." context menu from Windows Explorer, or open the archive in 7-Zip and then drag files in from Windows Explorer....
Read more >
Use tar to combine multiple files into an archive file - IU KB
In Unix and Unix-like operating systems (such as Linux), you can use the tar command (short for "tape archiving") to combine multiple files...
Read more >
i can't solve maven building error failure - Stack Overflow
I have fixed this issue by deleting local .m2/repository folder. The build is successful now.
Read more >
Python's zipfile: Manipulate Your ZIP Files Efficiently
In this guided tutorial, you'll learn how to manipulate ZIP files using Python's zipfile module from the standard library.
Read more >
peazip_help.pdf
like 7Z, RAR (extraction), TAR, ZIP and many other ones, see Supported formats ... On Windows platform, the installer takes care to create...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found