Fillna doesn't work when series object is an argument

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
• Modin version (modin.__version__):
• Python version:
• Code we can use to reproduce:

import modin.pandas as pd

df = pd.DataFrame({'a': ['a'], 'b': ['b'],}, index=['row1'])
df['c'] = pd.NA
df['c'] = df['c'].fillna(df['b'])   # df['c'].fillna('c') works fine
print(df)

Describe the problem

when fillna is called with series object as argument seems nothing happens while pandas does actual na replacement

Source code / logs

[noloerino]

As far as I can tell, it seems like something’s going awry in broadcast_axis_partitions, on this line: the value in the partition seems to be changed when the dataframe is reindexed. I’ll let someone more familiar with the codebase weigh in on why this is the case.

Abridged stack trace:

-> print(df['c'].fillna(df['b']))  # df['c'].fillna('c') works fine
  modin/modin/logging/logger_decorator.py(128)run_and_log()
-> return obj(*args, **kwargs)
  modin/modin/pandas/series.py(993)fillna()
-> return super(Series, self)._fillna(
  modin/modin/logging/logger_decorator.py(128)run_and_log()
-> return obj(*args, **kwargs)
  modin/modin/pandas/base.py(1525)_fillna()
-> new_query_compiler = self._query_compiler.fillna(
  modin/modin/logging/logger_decorator.py(128)run_and_log()
-> return obj(*args, **kwargs)
  modin/modin/core/storage_formats/pandas/query_compiler.py(1882)fillna()
-> new_modin_frame = self._modin_frame.binary_op(
  modin/modin/logging/logger_decorator.py(128)run_and_log()
-> return obj(*args, **kwargs)
  modin/modin/core/dataframe/pandas/dataframe/dataframe.py(115)run_f_on_minimally_updated_metadata()
-> result = f(self, *args, **kwargs)
  modin/modin/core/dataframe/pandas/dataframe/dataframe.py(2526)binary_op()
-> ) = new_left_frame._copartition(1, new_right_frame, join_type, sort=True)
  modin/modin/logging/logger_decorator.py(128)run_and_log()
-> return obj(*args, **kwargs)
  modin/modin/core/dataframe/pandas/dataframe/dataframe.py(2471)_copartition()
-> reindexed_other_list[i] = other_frames[
  modin/modin/core/dataframe/pandas/partitioning/partition_manager.py(573)map_axis_partitions()
-> return cls.broadcast_axis_partitions(
> modin/modin/core/dataframe/pandas/partitioning/partition_manager.py(476)broadcast_axis_partitions()

[anmyachev]

Could you please describe what “reindexing” do you mean?

_copartition calls _join_index_objects, which finds the common index to cast the left and right operand to, but for fillna doesn’t need to do this (this is True for column reindexing). You can see my changes in https://github.com/modin-project/modin/pull/4674.