异步并行下载文件

编辑

我已经改变了问题的标题，以反映我的问题，但也回答了如何轻松实现这一目标。

---

我试图让第二种方法返回Task而不是第一种方法中的Task ，但是由于尝试修复它，我得到了一连串的错误。

• 我在await body(partition.Current);之前添加了return await body(partition.Current);
• 反过来它要求我在下面添加一个return语句，所以我在下面添加了return null
• 但是现在select语句抱怨它无法从查询中推断出类型参数
• 我将Task.Run更改为Task.Run但没有成功。

我该如何解决？

第一种方法来自http://blogs.msdn.com/b/pfxteam/archive/2012/03/05/10278165.aspx ，第二种方法是我正在尝试创建的重载。

 public static class Extensions { public static Task ForEachAsync(this IEnumerable source, int dop, Func body) { return Task.WhenAll( from partition in Partitioner.Create(source).GetPartitions(dop) select Task.Run(async delegate { using (partition) while (partition.MoveNext()) await body(partition.Current); })); } public static Task ForEachAsync(this IEnumerable source, int dop, Func<T, Task> body) { return Task.WhenAll( from partition in Partitioner.Create(source).GetPartitions(dop) select Task.Run(async delegate { using (partition) while (partition.MoveNext()) await body(partition.Current); })); } } 

用法示例：

使用这种方法，我想并行和异步下载多个文件：

 private async void MainWindow_Loaded(object sender, RoutedEventArgs e) { Artist artist = await GetArtist(); IEnumerable enumerable = artist.Reviews.Select(s => s.ImageUrl); string[] downloadFile = await DownloadFiles(enumerable); } public static async Task DownloadFiles(IEnumerable enumerable) { if (enumerable == null) throw new ArgumentNullException("enumerable"); await enumerable.ForEachAsync(5, s => DownloadFile(s)); // Incomplete, the above statement is void and can't be returned } public static async Task DownloadFile(string address) { /* Download a file from specified address, * return destination file name on success or null on failure */ if (address == null) { return null; } Uri result; if (!Uri.TryCreate(address, UriKind.Absolute, out result)) { Debug.WriteLine(string.Format("Couldn't create URI from specified address: {0}", address)); return null; } try { using (var client = new WebClient()) { string fileName = Path.GetTempFileName(); await client.DownloadFileTaskAsync(address, fileName); Debug.WriteLine(string.Format("Downloaded file saved to: {0} ({1})", fileName, address)); return fileName; } } catch (WebException webException) { Debug.WriteLine(string.Format("Couldn't download file from specified address: {0}", webException.Message)); return null; } } 

Answer 1

我解决了它并在这里发布，可能会帮助任何人有同样的问题。

我最初的需求是一个小帮手，可以快速下载图像，但如果服务器没有快速响应，也只是放弃连接，所有这些都是并行和异步的 。

这个帮助器将返回一个元组，其中包含远程路径，本地路径和exception（如果发生）; 非常有用，因为知道为什么错误的下载出现故障总是很好。 我想我没有忘记下载的任何情况，但欢迎你发表评论。

• 您指定要下载的URL列表
• 您可以指定保存它的本地文件名，如果没有为您生成一个文件名
• 可选择取消下载的持续时间（方便慢速或无法访问的服务器）

您可以使用DownloadFileTaskAsync本身或使用ForEachAsync帮助程序进行并行和异步下载。

代码以及如何使用它的示例：

 private async void MainWindow_Loaded(object sender, RoutedEventArgs e) { IEnumerable enumerable = your urls here; var results = new List>(); await enumerable.ForEachAsync(s => DownloadFileTaskAsync(s, null, 1000), (url, t) => results.Add(t)); } /// 
 /// Downloads a file from a specified Internet address. /// 
 /// Internet address of the file to download. ///  /// Local file name where to store the content of the download, if null a temporary file name will /// be generated. ///  /// Duration in miliseconds before cancelling the operation. /// A tuple containing the remote path, the local path and an exception if one occurred. private static async Task> DownloadFileTaskAsync(string remotePath, string localPath = null, int timeOut = 3000) { try { if (remotePath == null) { Debug.WriteLine("DownloadFileTaskAsync (null remote path): skipping"); throw new ArgumentNullException("remotePath"); } if (localPath == null) { Debug.WriteLine( string.Format( "DownloadFileTaskAsync (null local path): generating a temporary file name for {0}", remotePath)); localPath = Path.GetTempFileName(); } using (var client = new WebClient()) { TimerCallback timerCallback = c => { var webClient = (WebClient) c; if (!webClient.IsBusy) return; webClient.CancelAsync(); Debug.WriteLine(string.Format("DownloadFileTaskAsync (time out due): {0}", remotePath)); }; using (var timer = new Timer(timerCallback, client, timeOut, Timeout.Infinite)) { await client.DownloadFileTaskAsync(remotePath, localPath); } Debug.WriteLine(string.Format("DownloadFileTaskAsync (downloaded): {0}", remotePath)); return new Tuple(remotePath, localPath, null); } } catch (Exception ex) { return new Tuple(remotePath, null, ex); } } public static class Extensions { public static Task ForEachAsync( this IEnumerable source, Func> taskSelector, Action resultProcessor) { var oneAtATime = new SemaphoreSlim(5, 10); return Task.WhenAll( from item in source select ProcessAsync(item, taskSelector, resultProcessor, oneAtATime)); } private static async Task ProcessAsync( TSource item, Func> taskSelector, Action resultProcessor, SemaphoreSlim oneAtATime) { TResult result = await taskSelector(item); await oneAtATime.WaitAsync(); try { resultProcessor(item, result); } finally { oneAtATime.Release(); } } } 

我没有更改ForEachAsync的签名来选择并行级别，我会让你按照自己的意愿调整它。

输出示例：

 DownloadFileTaskAsync (null local path): generating a temporary file name for http://sofzh.miximages.com/c%23/main_OTR_Britney480.jpg DownloadFileTaskAsync (null local path): generating a temporary file name for http://sofzh.miximages.com/c%23/britneyspears_femmefatale_cd.jpg DownloadFileTaskAsync (null local path): generating a temporary file name for http://sofzh.miximages.com/c%23/albumreviewsuk-526650850-1301400550.jpg?ymm_1xEDE5bu0tMi DownloadFileTaskAsync (null remote path): skipping DownloadFileTaskAsync (time out due): http://hangout.altsounds.com/geek/gars/images/3/9/8/5/2375.jpg DownloadFileTaskAsync (time out due): http://sofzh.miximages.com/c%23/britney-spears-femme-fatale.jpg DownloadFileTaskAsync (time out due): http://sofzh.miximages.com/c%23/main_OTR_Britney480.jpg DownloadFileTaskAsync (downloaded): http://sofzh.miximages.com/c%23/britneyspears1.jpg DownloadFileTaskAsync (downloaded): http://sofzh.miximages.com/c%23/britneyspears1.jpg DownloadFileTaskAsync (downloaded): http://sofzh.miximages.com/c%23/Femme-Fatale.jpg DownloadFileTaskAsync (downloaded): http://sofzh.miximages.com/c%23/72328.jpg 

过去需要花费1分钟的时间，相同的结果几乎不需要10秒:)

非常感谢这两篇文章的作者：

http://blogs.msdn.com/b/pfxteam/archive/2012/03/05/10278165.aspx

http://blogs.msdn.com/b/pfxteam/archive/2012/03/04/10277325.aspx