question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

🐞 Bug

Balance by time didn’t ignore the time for CUDA initialization. I think adding a warm-up run can eliminate the effect of that. Balance by size actually accumulate the size so that the latter layer always consumes more memory than the previous layers.

Code that reproduces

Change the test to the following can reproduce for balance by size.

def test_balance_by_size():
    class Expand(nn.Module):
        def __init__(self, times):
            super().__init__()
            self.times = times

        def forward(self, x):
            for i in range(self.times):
                x = x + torch.rand_like(x, requires_grad=True)
            return x

    model = nn.Sequential(*[Expand(i) for i in [6, 5, 4, 3, 2, 1]])
    sample = torch.rand(10, 100, 100)
    balance = balance_by_size(model, sample, partitions=2, device='cuda')
    assert balance == [2, 4]

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
842974287commented, Aug 7, 2019

That makes sense, thanks!

0reactions
subleecommented, Aug 13, 2019

I fixed this issue with 3011ff0613fc91606c637d5df139e44b2d3bfaa5. If you still have the same issue, please reopen it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Balanced module - Wikipedia
In the subfield of abstract algebra known as module theory, a right R module M is called a balanced module if every endomorphism...
Read more >
Balance Module for (3.2V) LiFePO4 Battery Pack with ...
The balance module will keep each LFP cell's voltage at 3.60V peak during charging by draining excessive voltage with selectable discharge current (0.65A/ ......
Read more >
Tutorial to Interface HX711 Balance Module With Load Cell
1. In this tutorial, you will need : 1. Arduino Uno Board and USB 2. HX711 Balance Sensor 3. Load Cell (can be any weight of...
Read more >
Battery Charging Module, 3S 11.1V 18650 Li-ion 12.6V Li ...
Features: This balancing module is used for lithium battery charging which has no balancing. Balancing charging would ensure each battery cell being fully ......
Read more >
Steam Workshop::Faction Enhancer - Econ Balance Module
This is the economy balance module. Features include: Weapon Components production increased by 20%; Fixed Lasertower Mk 1 resource costs (were the same...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found