[FEATURE REQUEST]: Spark 3.0 Readiness

APIs

SparkSession

• csharp public static void SetActiveSession(SparkSession session) (#641)
• csharp public static void ClearActiveSession() (#641)
• csharp public static SparkSession GetActiveSession() (#641)
• csharp public DataFrame ExecuteCommand(string runner, string command, Dictionary<string, string> options) (#647)

DataFrame

• csharp public DataFrame Transform(Func<DataFrame, DataFrame> func) (#688)
• csharp public IEnumerable<Row> Tail(int n) (#647)
• csharp public void PrintSchema(int level) (#647)
• csharp public void Explain(string mode) (#647)
• csharp public DataFrame Observe(string name, Column expr, params Column[] exprs) (#647)
• csharp WriteTo(string table) (#677)

DataFrameStatFunctions

• csharp public DataFrame SampleBy<T>(Column column, IDictionary<T, double> fractions, long seed) (#647)

DataFrameWriterV2

• csharp public DataFrameWriterV2 Using(string provider) (#677)
• csharp public DataFrameWriterV2 Option(string key, string value) (#677)
• csharp public DataFrameWriterV2 Option(string key, bool value) (#677)
• csharp public DataFrameWriterV2 Option(string key, long value) (#677)
• csharp public DataFrameWriterV2 Option(string key, double value) (#677)
• csharp public DataFrameWriterV2 Options(Dictionary<string, string> options) (#677)
• csharp public DataFrameWriterV2 TableProperty(string property, string value) (#677)
• csharp public DataFrameWriterV2 PartitionedBy(Column column, params Column[] columns) (#677)
• csharp public void Create() (#677)
• csharp public void Replace() (#677)
• csharp public void CreateOrReplace() (#677)
• csharp public void Append() (#677)
• csharp public void Overwrite(Column condition) (#677)
• csharp public void OverwritePartitions() (#677)

RelationalGroupedDataset

• scala def as[K: Encoder, T: Encoder]: KeyValueGroupedDataset[K, T]

Functions

• csharp public static Column XXHash64(params Column[] columns) (#649)
• csharp public static Column Split(Column column, string pattern, int limit) (#649)
• csharp public static Column Overlay(Column src, Column replace, Column pos, Column len) (#649)
• csharp public static Column Overlay(Column src, Column replace, Column pos) (#649)
• csharp public static Column AddMonths(Column startDate, Column numMonths) (#649)
• csharp public static Column DateAdd(Column start, Column days) (#649)
• csharp public static Column DateSub(Column start, Column days) (#649)
• scala def transform(column: Column, f: Column => Column): Column Unsupported: passing function as parameter
• scala def transform(column: Column, f: (Column, Column) => Column): Column Unsupported: passing function as parameter
• scala def exists(column: Column, f: Column => Column): Column Unsupported: passing function as parameter
• scala def forall(column: Column, f: Column => Column): Column Unsupported: passing function as parameter
• scala def filter(column: Column, f: Column => Column): Column Unsupported: passing function as parameter
• scala def filter(column: Column, f: (Column, Column) => Column): Column Unsupported: passing function as parameter
• scala def aggregate(expr: Column, initialValue: Column, merge: (Column, Column) => Column, finish: Column => Column): Column Unsupported: passing function as parameter
• scala def aggregate(expr: Column, initialValue: Column, merge: (Column, Column) => Column): Column Unsupported: passing function as parameter
• scala def zip_with(left: Column, right: Column, f: (Column, Column) => Column): Column Unsupported: passing function as parameter
• scala def transform_keys(expr: Column, f: (Column, Column) => Column): Column Unsupported: passing function as parameter
• scala def transform_values(expr: Column, f: (Column, Column) => Column): Column Unsupported: passing function as parameter
• scala def map_filter(expr: Column, f: (Column, Column) => Column): Column Unsupported: passing function as parameter
• scala def map_zip_with(left: Column, right: Column, f: (Column, Column, Column) => Column): Column Unsupported: passing function as parameter
• csharp public static Column SchemaOfJson(Column json, Dictionary<string, string> options) (#649)
• csharp public static Column MapEntries(Column column) (#649)
• csharp public static Column FromCsv(Column column, StructType schema, Dictionary<string, string> options) (#649)
• csharp public static Column FromCsv(Column column, Column schema, Dictionary<string, string> options) (#649)
• csharp public static Column SchemaOfCsv(string csv) (#649)
• csharp public static Column SchemaOfCsv(Column csv) (#649)
• csharp public static Column SchemaOfCsv(Column csv, Dictionary<string, string> options) (#649)
• csharp public static Column ToCsv(Column column, Dictionary<string, string> options) (#649)
• csharp public static Column ToCsv(Column column) (#649)
• csharp public static Column Years(Column column) (#649)
• csharp public static Column Months(Column column) (#649)
• csharp public static Column Days(Column column) (#649)
• csharp public static Column Hours(Column column) (#649)
• csharp public static Column Bucket(Column numBuckets, Column column) (#649)
• csharp public static Column Bucket(int numBuckets, Column column) (#649)

[rrekapalli]

@GoEddie , I am new to Spark and just trying out this library for one of my use case. However, I can definitely give it a try from September 2nd week.

[Niharikadutta]

All features in this issue have been merged, this can be closed.