Efficiency of run length encoding

Version: 3.9.1

Runtime Version: .NET 5

Expected behavior

Remove repetitions in columns.

Actual behavior

The file produced by the sample below contains a number of bytes that repeat - 00 FE 04 00 FE 04 00 FE 04 etc. When I compress this file with 7-zip, it becomes almost 200 times smaller.

Is this expected behavior?

Code snippet reproducing the behavior

			var buf = new int[1024 * 1024];
			for (var i = 0; i < buf.Length; i++)
			{
				buf[i] = 1;
			}

			using (var stream = File.Create("_test.parquet"))
			{
				var field = new DataField<int>("Val");
				var schema = new Schema(field);
				using var parquetWriter = new ParquetWriter(schema, stream);
				using var groupWriter = parquetWriter.CreateRowGroup();
				var column = new DataColumn(field, buf);
				groupWriter.WriteColumn(column);
			}

[houseofcat]

@aloneguid not that I have much time to work on someone else’s projects.

Plenty of time to benefit from some one else’s work though.

[fandrei]

@aloneguid well, it’s your project. And RLE is one of important features of Parquet. If you decided to not make a quality implementation, it’s up to you 😃