Balance Module
See original GitHub issue🐞 Bug
Balance by time didn’t ignore the time for CUDA initialization. I think adding a warm-up run can eliminate the effect of that. Balance by size actually accumulate the size so that the latter layer always consumes more memory than the previous layers.
Code that reproduces
Change the test to the following can reproduce for balance by size.
def test_balance_by_size():
class Expand(nn.Module):
def __init__(self, times):
super().__init__()
self.times = times
def forward(self, x):
for i in range(self.times):
x = x + torch.rand_like(x, requires_grad=True)
return x
model = nn.Sequential(*[Expand(i) for i in [6, 5, 4, 3, 2, 1]])
sample = torch.rand(10, 100, 100)
balance = balance_by_size(model, sample, partitions=2, device='cuda')
assert balance == [2, 4]
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Balanced module - Wikipedia
In the subfield of abstract algebra known as module theory, a right R module M is called a balanced module if every endomorphism...
Read more >Balance Module for (3.2V) LiFePO4 Battery Pack with ...
The balance module will keep each LFP cell's voltage at 3.60V peak during charging by draining excessive voltage with selectable discharge current (0.65A/ ......
Read more >Tutorial to Interface HX711 Balance Module With Load Cell
1. In this tutorial, you will need :
1. Arduino Uno Board and USB
2. HX711 Balance Sensor
3. Load Cell (can be any weight of...
Read more >Battery Charging Module, 3S 11.1V 18650 Li-ion 12.6V Li ...
Features: This balancing module is used for lithium battery charging which has no balancing. Balancing charging would ensure each battery cell being fully ......
Read more >Steam Workshop::Faction Enhancer - Econ Balance Module
This is the economy balance module. Features include: Weapon Components production increased by 20%; Fixed Lasertower Mk 1 resource costs (were the same...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
That makes sense, thanks!
I fixed this issue with 3011ff0613fc91606c637d5df139e44b2d3bfaa5. If you still have the same issue, please reopen it.