从FTP服务器下载大文件(~150MB)挂起
我试图使用以下代码从ftp服务器下载文件:
using (System.IO.FileStream fileStream = System.IO.File.OpenWrite(filePath)) { byte[] buffer = new byte[4096]; int bytesRead = responseStream.Read(buffer, 0, 4096); while (bytesRead > 0) { fileStream.Write(buffer, 0, bytesRead); bytesRead = responseStream.Read(buffer, 0, 4096); } }
responseStream的创建:
System.IO.Stream responseStream = GetFileAsStream(url, username, password, false); public static System.IO.Stream GetFileAsStream(string ftpUrl, string username, string password, bool usePassive) { System.Net.FtpWebRequest request = (System.Net.FtpWebRequest)System.Net.WebRequest.Create(ftpUrl); request.KeepAlive = false; request.ReadWriteTimeout = 120000; request.Timeout = -1; request.UsePassive = usePassive; request.Credentials = new System.Net.NetworkCredential(username, password); request.Method = System.Net.WebRequestMethods.Ftp.DownloadFile; System.IO.Stream fileResponseStream; System.Net.FtpWebResponse fileResponse = (System.Net.FtpWebResponse)request.GetResponse(); fileResponseStream = fileResponse.GetResponseStream(); return fileResponseStream; }
它适用于较小的文件,但当文件较大(例如150mb)时,该过程会挂起。 由于某种原因,程序不理解它已完成下载并仍尝试读取更多字节。
我更喜欢不包括使用外部库的答案。 谢谢
我已经使用您的代码成功下载了超过150MB的多个文件。 正如其他人所建议的那样,您的FTP服务器可能存在问题。
我通过引入请求超时解决了我的问题 – 如果达到,则使程序抛出WebException。 在这种情况下,程序将从其剩余的位置恢复下载。 这是我的代码:
这是一个方法,如果文件被下载则返回true,否则返回false
Digitalez.DirectoryUtil.EnsureDirectoryExists(relativePath); string filePath = System.IO.Path.Combine(relativePath, fileInfo.Name); long length = Digitalez.FtpUtil.GetFileLength(fileInfo.FullPath, userName, password, usePassive); long offset = 0; int retryCount = 10; int? readTimeout = 5 * 60 * 1000; //five minutes // if the file exists, do not download it if (System.IO.File.Exists(filePath)) { return false; } while (retryCount > 0) { using (System.IO.Stream responseStream = Captator.Eifos.Net.FtpUtil.GetFileAsStream(fileInfo.FullPath, userName, password, usePassive, offset, requestTimeout: readTimeout != null ? readTimeout.Value : System.Threading.Timeout.Infinite)) { using (System.IO.FileStream fileStream = new System.IO.FileStream(filePath, System.IO.FileMode.Append)) { byte[] buffer = new byte[4096]; try { int bytesRead = responseStream.Read(buffer, 0, buffer.Length); while (bytesRead > 0) { fileStream.Write(buffer, 0, bytesRead); bytesRead = responseStream.Read(buffer, 0, buffer.Length); } return true; } catch (System.Net.WebException) { // Do nothing - consume this exception to force a new read of the rest of the file } } if (System.IO.File.Exists(filePath)) { offset = new System.IO.FileInfo(filePath).Length; } else { offset = 0; } retryCount--; if (offset == length) { return true; } } }
Digitalez.FtpUtil:
public static System.IO.Stream GetFileAsStream(string ftpUrl, string username, string password, bool usePassive, long offset, int requestTimeout) { System.Net.FtpWebRequest request = (System.Net.FtpWebRequest)System.Net.WebRequest.Create(ftpUrl); request.KeepAlive = false; request.ReadWriteTimeout = requestTimeout; request.Timeout = requestTimeout; request.ContentOffset = offset; request.UsePassive = usePassive; request.UseBinary = true; request.Credentials = new System.Net.NetworkCredential(username, password); request.Method = System.Net.WebRequestMethods.Ftp.DownloadFile; System.IO.Stream fileResponseStream; System.Net.FtpWebResponse fileResponse = (System.Net.FtpWebResponse)request.GetResponse(); fileResponseStream = fileResponse.GetResponseStream(); return fileResponseStream; } public static long GetFileLength(string ftpUrl, string username, string password, bool usePassive) { System.Net.FtpWebRequest request = (System.Net.FtpWebRequest)System.Net.WebRequest.Create(ftpUrl); request.KeepAlive = false; request.UsePassive = usePassive; request.Credentials = new System.Net.NetworkCredential(username, password); request.Method = System.Net.WebRequestMethods.Ftp.GetFileSize; System.Net.FtpWebResponse lengthResponse = (System.Net.FtpWebResponse)request.GetResponse(); long length = lengthResponse.ContentLength; lengthResponse.Close(); return length; }
我没有尝试过其他服务器,但这确实可以解决问题。