question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

REFACTOR: remove code duplication for `PandasOnRayDataframePartitionManager`

See original GitHub issue

In the current implementation, the following methods of PandasOnRayDataframePartitionManager class (child class of PandasDataframePartitionManager) are overridden just to apply the progress_bar_wrapper decorator:

  • map_partitions
  • lazy_map_partitions
  • map_axis_partitions
  • _apply_func_to_list_of_partitions
  • apply_func_to_select_indices
  • apply_func_to_select_indices_along_full_axis
  • apply_func_to_indices_both_axis
  • binary_operation

Moving the decorator to the base class will remove the duplication of functions.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
vnlitvinovcommented, Aug 30, 2022

This doesn’t work because classmethod decorator should be applied last (#4895).

Okay, so then do just

    lazy_map_partitions = classmethod(progress_bar_wrapper(GenericRayDataframePartitionManager.lazy_map_partitions.__func__))

maybe even defining a small helper method like

class RayDataframePartitionManager:
    # do things

def _make_wrapped_method(name):
    setattr(
        RayDataframePartitionManager, 
        name,
        classmethod(
            progress_bar_wrapper(
                getattr(GenericRayDataframePartitionManager, name).__func__)
            )
        )
    )

for method in ('lazy_map_partitions', ........):
    _make_wrapped_method(method)
0reactions
anmyachevcommented, Aug 30, 2022

@anmyachev @vnlitvinov can we use inherit_doctsrings to inherit the parent class docstrings?

I think we can, but this only solves part of the issue of code duplication.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Duplicate Code - Refactoring.Guru
If the duplicate code is inside a constructor, use Pull Up Constructor Body. If the duplicate code is similar but not completely identical,...
Read more >
Eliminating code duplication in a single file - Stack Overflow
This project is an ASP.NET/C# project, but I work with a variety of languages including Java; I'm interested in what tools are best...
Read more >
The Art of Refactoring: Removing Code Duplications
You can fix this by moving the log message into a parameter passed to the method to be created.
Read more >
Releases · modin-project/modin - GitHub
Modin: Scale your Pandas workflows by changing a single line of code - Releases ... Remove code duplication for PandasOnRayDataframePartitionManager (#4895) ...
Read more >
How To Remove Duplicate Code Properly From Your App
Removing duplicate code is a simple concept to understand yet it has profound results in regards to code quality and scalability.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found