Hand over data stream as upload stream throws exception

FTP Server OS: Unix

FTP Server Type: Vsftpd

Client Computer OS: Windows

FluentFTP Version: 42.1.0

Framework: .NET 6

We want to provide data via ftp upload to our clients. The data are requested via Http request from an internal service and the stream is directly forwarded to the FTP upload. To avoid huge in memory data allocation we use the HttpCompletionOption ResponseHeadersRead to hand over the data immediately piece by piece.

Unfortunately, FluentFTP throws an error since it tries to get the file size, that is not known in this scenario since the data are not fully loaded into memory at any time.

We could use HttpCompletionOption ResponseContentRead but then we don’t have the performance advantage and we need to provide huge memory resources.

Is there any possibility to cope our use case with Fluent FTP? Is it maybe a bug?

Code to reproduce:

 var client = new AsyncFtpClient("ftp1", new NetworkCredential("user1", "password123"));
 await client.AutoConnect();
 
var testClient = new HttpClient();
var response = await testClient.GetAsync(
	"https://file-examples.com/storage/fee589dbcc6394c129ba7e9/2017/10/file-sample_150kB.pdf",
	HttpCompletionOption.ResponseHeadersRead);

var stream = await response.Content.ReadAsStreamAsync();

// here the exception throws
await client.UploadStream(stream, "/upload/test/file/test_doc.pdf", FtpRemoteExists.Overwrite, createRemoteDir: true);

Logs from FluentFTP:

# AutoConnect()

# AutoDetect(True, False)

# ConnectAsync()
Status:   FluentFTP 42.1.0.0
Status:   Connecting to IP #1= ***:21
Status:   Waiting for a response
Response: 220 (vsFTPd 3.0.3)
Status:   Detected FTP server: VsFTPd
Command:  AUTH TLS
Status:   Waiting for response to: AUTH TLS
Response: 530 Please login with USER and PASS.
Command:  USER ***
Status:   Waiting for response to: USER ***
Response: 331 Please specify the password.
Command:  PASS ***
Status:   Waiting for response to: PASS ***
Response: 230 Login successful.
Command:  FEAT
Status:   Waiting for response to: FEAT
Response: 211-Features:
Response: EPRT
Response: EPSV
Response: MDTM
Response: PASV
Response: REST STREAM
Response: SIZE
Response: TVFS
Response: 211 End
Status:   Text encoding: System.Text.UTF8Encoding+UTF8EncodingSealed
Command:  OPTS UTF8 ON
Status:   Waiting for response to: OPTS UTF8 ON
Response: 200 Always in UTF8 mode.
Command:  SYST
Status:   Waiting for response to: SYST
Response: 215 UNIX Type: L8
Status:   Listing parser set to: Unix
Command:  PWD
Status:   Waiting for response to: PWD
Response: 257 "/" is the current directory

# UploadStream("/upload/test/file/test_doc.pdf", Overwrite, True)

# FileExists("/upload/test/file/test_doc.pdf")
Command:  SIZE /upload/test/file/test_doc.pdf
Status:   Waiting for response to: SIZE /upload/test/file/test_doc.pdf
Response: 550 Could not get file size.

# DirectoryExists("/upload/test/file")
Command:  CWD /upload/test/file
Status:   Waiting for response to: CWD /upload/test/file
Response: 250 Directory successfully changed.
Command:  CWD /
Status:   Waiting for response to: CWD /
Response: 250 Directory successfully changed.
FluentFTP.FtpException: Error while uploading the file to the server. See InnerException for more info.
 ---> System.NotSupportedException: Specified method is not supported.
   at System.Net.Http.HttpBaseStream.get_Length()
   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress) in C:\Data\src\Demo\FluentFTP\FluentFTP\Client\AsyncClient\UploadFileInternal.cs:line 106
   --- End of inner exception stack trace ---
   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress) in C:\Data\src\Demo\FluentFTP\FluentFTP\Client\AsyncClient\UploadFileInternal.cs:line 286
   at FluentFTP.AsyncFtpClient.UploadStream(Stream fileStream, String remotePath, FtpRemoteExists existsMode, Boolean createRemoteDir, IProgress`1 progress, CancellationToken token) in C:\Data\src\Demo\FluentFTP\FluentFTP\Client\AsyncClient\UploadStream.cs:line 43
   at Program.<Main>$(String[] args) in C:\Data\src\Demo\FluentFtpBug\Program.cs:line 146
Unhandled exception. FluentFTP.FtpException: Error while uploading the file to the server. See InnerException for more info.
 ---> System.NotSupportedException: Specified method is not supported.
   at System.Net.Http.HttpBaseStream.get_Length()
   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress) in C:\Data\src\Demo\FluentFTP\FluentFTP\Client\AsyncClient\UploadFileInternal.cs:line 106
   --- End of inner exception stack trace ---
   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress) in C:\Data\src\Demo\FluentFTP\FluentFTP\Client\AsyncClient\UploadFileInternal.cs:line 286
   at FluentFTP.AsyncFtpClient.UploadStream(Stream fileStream, String remotePath, FtpRemoteExists existsMode, Boolean createRemoteDir, IProgress`1 progress, CancellationToken token) in C:\Data\src\Demo\FluentFTP\FluentFTP\Client\AsyncClient\UploadStream.cs:line 43
   at Program.<Main>$(String[] args) in C:\Data\src\Demo\FluentFtpBug\Program.cs:line 146
   at Program.<Main>(String[] args)

[MurmelNash]

A quick test was successful.

[FanDjango]

Ok, you may try again. Meanwhile I will run some other tests on the new change.