PERF: slow `concat`

Measured on master (https://github.com/modin-project/modin/commit/cfafbb254c221dd4f739a9cf5af17c9e8cdf13c3), Ray, 8 cores.

Problem: too much time is spent doing concat, it can be much faster. Pandas vs Modin: 0.89 sec vs 4,5 sec.

Possible solution: compute new_widths via _column_widths_cache where possible.

Script:

import modin.pandas as pd
import numpy as np
from time import time

random_state = np.random.RandomState(seed=42)
array = random_state.rand(10**6, 35)

df1 = pd.DataFrame(array)
df2 = pd.DataFrame(array)

df1 = df1 - 1
df2 = df2 - 2

start = time()
df = pd.concat([df1, df2], axis=1, copy=False)
print(f"concat time: {time()-start}") <-- 4,5 sec

[pyrito]

@anmyachev yeah of course! I was a bit puzzled to see modin finish the concat quickly in BenchmarkMode, but then I recall that benchmark mode resolves computations immediately, so it didn’t wait on the binary operation for df1 and df2.

[pyrito]

Nevertheless, I think your PR addresses this @anmyachev