ProgressBar for `.apply` when looping over rows

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20.04.4 LTS
• Modin version (modin.__version__): 0.14.1
• Python version: 3.8
• Code we can use to reproduce:

import modin.pandas as pd
import tqdm
from modin.config import ProgressBar
ProgressBar.enable()

X = pd.DataFrame([[1,2,3]]*100, columns=["a", "b", "c"]) # be wary that this produces its own progress bar
X.apply(lambda x: 1, axis=1) # does not produce progress bar; if axis=0, progress bar is produced

Describe the problem

The ProgressBar seems to not appear when apply is used with axis=1 (iterating over rows). It does appear for axis=0, but oddly not if a Series is returned (common pattern for constructing new dataframes). If running the example, be careful to disregard the progress bar generated by the pd.Dataframe command.

Source code / logs

[mvashishtha]

@Bonnevie I would also like to add to @jeffreykennethli 's response:

• Issue #4167 says that we’re missing some functions at the internal level of the Modin dataframe. Among those are the ones we would need to give an estimate of apply progress. I think just making apply work shouldn’t be too much effort. I mentioned it in a comment on #4167 and we will try to address the feature soon.
• Each progress bar should tell users which frame-level operation it’s for. In the case of axis=0 apply that you show here, it should explain that it’s showing the progress of the transpose. I have filed #4447 for that new feature.

[mvashishtha]

@Bonnevie I’m closing this issue. If you have any more concerns, please comment here. We’ll continue tracking work on the followups: https://github.com/modin-project/modin/issues/4443#issuecomment-1121221578