如何在超时期限后取消任务等待

我正在使用此方法以编程方式实例化Web浏览器,导航到URL并在文档完成时返回结果。

如果文档加载时间超过5秒,我将如何能够停止Task并让GetFinalUrl()返回null

我见过很多使用TaskFactory例子,但是我无法将它应用于这段代码。

  private Uri GetFinalUrl(PortalMerchant portalMerchant) { SetBrowserFeatureControl(); Uri finalUri = null; if (string.IsNullOrEmpty(portalMerchant.Url)) { return null; } Uri trackingUrl = new Uri(portalMerchant.Url); var task = MessageLoopWorker.Run(DoWorkAsync, trackingUrl); task.Wait(); if (!String.IsNullOrEmpty(task.Result.ToString())) { return new Uri(task.Result.ToString()); } else { throw new Exception("Parsing Failed"); } } // by Noseratio - http://stackoverflow.com/users/1768303/noseratio static async Task DoWorkAsync(object[] args) { _threadCount++; Console.WriteLine("Thread count:" + _threadCount); Uri retVal = null; var wb = new WebBrowser(); wb.ScriptErrorsSuppressed = true; TaskCompletionSource tcs = null; WebBrowserDocumentCompletedEventHandler documentCompletedHandler = (s, e) => tcs.TrySetResult(true); foreach (var url in args) { tcs = new TaskCompletionSource(); wb.DocumentCompleted += documentCompletedHandler; try { wb.Navigate(url.ToString()); await tcs.Task; } finally { wb.DocumentCompleted -= documentCompletedHandler; } retVal = wb.Url; wb.Dispose(); return retVal; } return null; } public static class MessageLoopWorker { #region Public static methods public static async Task Run(Func<object[], Task> worker, params object[] args) { var tcs = new TaskCompletionSource(); var thread = new Thread(() => { EventHandler idleHandler = null; idleHandler = async (s, e) => { // handle Application.Idle just once Application.Idle -= idleHandler; // return to the message loop await Task.Yield(); // and continue asynchronously // propogate the result or exception try { var result = await worker(args); tcs.SetResult(result); } catch (Exception ex) { tcs.SetException(ex); } // signal to exit the message loop // Application.Run will exit at this point Application.ExitThread(); }; // handle Application.Idle just once // to make sure we're inside the message loop // and SynchronizationContext has been correctly installed Application.Idle += idleHandler; Application.Run(); }); // set STA model for the new thread thread.SetApartmentState(ApartmentState.STA); // start the thread and await for the task thread.Start(); try { return await tcs.Task; } finally { thread.Join(); } } #endregion } 

更新 :可以在Github上找到基于WebBrowser的控制台Web剪贴板的最新版本。

更新 :为多个并行下载添加WebBrowser对象池 。

您是否有任何机会在控制台应用程序中执行此操作的示例? 此外,我不认为webBrowser可以是一个类变量,因为我在每个平行运行整个事物,迭代数千个URL

下面是或多或少通用的基于WebBrowser的web scrapper的实现 ,它作为控制台应用程序。 它是我之前与WebBrowser相关的一些工作的合并,包括问题中引用的代码:

  • 使用不透明度捕获网页图像

  • 加载包含动态AJAX内容的页面

  • WebBrowser创建STA消息循环线程

  • 一个接一个地加载一组URL

  • 使用WebBrowser打印一组URL

  • 网页UI自动化

几点:

  • 可重用的MessageLoopApartment类用于启动和运行具有自己的消息泵的WinForms STA线程。 它可以在控制台应用程序中使用 ,如下所示。 此类公开TPL任务计划程序( FromCurrentSynchronizationContext )和一组Task.Factory.StartNew包装器以使用此任务计划程序。

  • 这使得async/await成为在单独的STA线程上运行WebBrowser导航任务的一个很好的工具。 这样, WebBrowser对象就会在该线程上创建,导航和销毁。 虽然, MessageLoopApartment并没有专门与WebBrowser捆绑在一起。

  • 使用浏览器function控件启用HTML5渲染非常重要,否则默认情况下WebBrowser对象将以IE7仿真模式运行。 这就是SetFeatureBrowserEmulation在下面所做的事情。

  • 可能无法始终确定网页何时以100%的概率完成渲染。 有些页面非常复杂,并且使用连续的AJAX更新。 然而,通过首先处理DocumentCompleted事件,然后轮询页面的当前HTML快照以进行更改并检查WebBrowser.IsBusy属性,我们可以非常接近。 这就是NavigateAsync在下面所做的事情。

  • 如果页面呈现永无止境,则在上面提供超时逻辑(注意CancellationTokenSourceCreateLinkedTokenSource )。

 using Microsoft.Win32; using System; using System.Threading; using System.Threading.Tasks; using System.Windows.Forms; namespace Console_22239357 { class Program { // by Noseratio - https://stackoverflow.com/a/22262976/1768303 // main logic static async Task ScrapSitesAsync(string[] urls, CancellationToken token) { using (var apartment = new MessageLoopApartment()) { // create WebBrowser inside MessageLoopApartment var webBrowser = apartment.Invoke(() => new WebBrowser()); try { foreach (var url in urls) { Console.WriteLine("URL:\n" + url); // cancel in 30s or when the main token is signalled var navigationCts = CancellationTokenSource.CreateLinkedTokenSource(token); navigationCts.CancelAfter((int)TimeSpan.FromSeconds(30).TotalMilliseconds); var navigationToken = navigationCts.Token; // run the navigation task inside MessageLoopApartment string html = await apartment.Run(() => webBrowser.NavigateAsync(url, navigationToken), navigationToken); Console.WriteLine("HTML:\n" + html); } } finally { // dispose of WebBrowser inside MessageLoopApartment apartment.Invoke(() => webBrowser.Dispose()); } } } // entry point static void Main(string[] args) { try { WebBrowserExt.SetFeatureBrowserEmulation(); // enable HTML5 var cts = new CancellationTokenSource((int)TimeSpan.FromMinutes(3).TotalMilliseconds); var task = ScrapSitesAsync( new[] { "http://example.com", "http://example.org", "http://example.net" }, cts.Token); task.Wait(); Console.WriteLine("Press Enter to exit..."); Console.ReadLine(); } catch (Exception ex) { while (ex is AggregateException && ex.InnerException != null) ex = ex.InnerException; Console.WriteLine(ex.Message); Environment.Exit(-1); } } } ///  /// WebBrowserExt - WebBrowser extensions /// by Noseratio - https://stackoverflow.com/a/22262976/1768303 ///  public static class WebBrowserExt { const int POLL_DELAY = 500; // navigate and download public static async Task NavigateAsync(this WebBrowser webBrowser, string url, CancellationToken token) { // navigate and await DocumentCompleted var tcs = new TaskCompletionSource(); WebBrowserDocumentCompletedEventHandler handler = (s, arg) => tcs.TrySetResult(true); using (token.Register(() => tcs.TrySetCanceled(), useSynchronizationContext: true)) { webBrowser.DocumentCompleted += handler; try { webBrowser.Navigate(url); await tcs.Task; // wait for DocumentCompleted } finally { webBrowser.DocumentCompleted -= handler; } } // get the root element var documentElement = webBrowser.Document.GetElementsByTagName("html")[0]; // poll the current HTML for changes asynchronosly var html = documentElement.OuterHtml; while (true) { // wait asynchronously, this will throw if cancellation requested await Task.Delay(POLL_DELAY, token); // continue polling if the WebBrowser is still busy if (webBrowser.IsBusy) continue; var htmlNow = documentElement.OuterHtml; if (html == htmlNow) break; // no changes detected, end the poll loop html = htmlNow; } // consider the page fully rendered token.ThrowIfCancellationRequested(); return html; } // enable HTML5 (assuming we're running IE10+) // more info: https://stackoverflow.com/a/18333982/1768303 public static void SetFeatureBrowserEmulation() { if (System.ComponentModel.LicenseManager.UsageMode != System.ComponentModel.LicenseUsageMode.Runtime) return; var appName = System.IO.Path.GetFileName(System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName); Registry.SetValue(@"HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION", appName, 10000, RegistryValueKind.DWord); } } ///  /// MessageLoopApartment /// STA thread with message pump for serial execution of tasks /// by Noseratio - https://stackoverflow.com/a/22262976/1768303 ///  public class MessageLoopApartment : IDisposable { Thread _thread; // the STA thread TaskScheduler _taskScheduler; // the STA thread's task scheduler public TaskScheduler TaskScheduler { get { return _taskScheduler; } } /// MessageLoopApartment constructor public MessageLoopApartment() { var tcs = new TaskCompletionSource(); // start an STA thread and gets a task scheduler _thread = new Thread(startArg => { EventHandler idleHandler = null; idleHandler = (s, e) => { // handle Application.Idle just once Application.Idle -= idleHandler; // return the task scheduler tcs.SetResult(TaskScheduler.FromCurrentSynchronizationContext()); }; // handle Application.Idle just once // to make sure we're inside the message loop // and SynchronizationContext has been correctly installed Application.Idle += idleHandler; Application.Run(); }); _thread.SetApartmentState(ApartmentState.STA); _thread.IsBackground = true; _thread.Start(); _taskScheduler = tcs.Task.Result; } /// shutdown the STA thread public void Dispose() { if (_taskScheduler != null) { var taskScheduler = _taskScheduler; _taskScheduler = null; // execute Application.ExitThread() on the STA thread Task.Factory.StartNew( () => Application.ExitThread(), CancellationToken.None, TaskCreationOptions.None, taskScheduler).Wait(); _thread.Join(); _thread = null; } } /// Task.Factory.StartNew wrappers public void Invoke(Action action) { Task.Factory.StartNew(action, CancellationToken.None, TaskCreationOptions.None, _taskScheduler).Wait(); } public TResult Invoke(Func action) { return Task.Factory.StartNew(action, CancellationToken.None, TaskCreationOptions.None, _taskScheduler).Result; } public Task Run(Action action, CancellationToken token) { return Task.Factory.StartNew(action, token, TaskCreationOptions.None, _taskScheduler); } public Task Run(Func action, CancellationToken token) { return Task.Factory.StartNew(action, token, TaskCreationOptions.None, _taskScheduler); } public Task Run(Func action, CancellationToken token) { return Task.Factory.StartNew(action, token, TaskCreationOptions.None, _taskScheduler).Unwrap(); } public Task Run(Func> action, CancellationToken token) { return Task.Factory.StartNew(action, token, TaskCreationOptions.None, _taskScheduler).Unwrap(); } } } 

我怀疑在另一个线程上运行处理循环不会很好,因为WebBrowser是一个承载ActiveX控件的UI组件。

当您通过EAP包装器编写TAP时 ,我建议使用扩展方法来保持代码清洁:

 public static Task NavigateAsync(this WebBrowser @this, string url) { var tcs = new TaskCompletionSource(); WebBrowserDocumentCompletedEventHandler subscription = null; subscription = (_, args) => { @this.DocumentCompleted -= subscription; tcs.TrySetResult(args.Url.ToString()); }; @this.DocumentCompleted += subscription; @this.Navigate(url); return tcs.Task; } 

现在您的代码可以轻松应用超时:

 async Task GetUrlAsync(string url) { using (var wb = new WebBrowser()) { var navigate = wb.NavigateAsync(url); var timeout = Task.Delay(TimeSpan.FromSeconds(5)); var completed = await Task.WhenAny(navigate, timeout); if (completed == navigate) return await navigate; return null; } } 

可以这样消费:

 private async Task GetFinalUrlAsync(PortalMerchant portalMerchant) { SetBrowserFeatureControl(); if (string.IsNullOrEmpty(portalMerchant.Url)) return null; var result = await GetUrlAsync(portalMerchant.Url); if (!String.IsNullOrEmpty(result)) return new Uri(result); throw new Exception("Parsing Failed"); } 

我正试图从Noseratio的解决方案中受益,并遵循Stephen Cleary的建议。

这是我更新的代码,包括来自Stephen的代码,来自Noseratio的关于AJAX技巧的代码。

第一部分:斯蒂芬建议的Task NavigateAsync

 public static Task NavigateAsync(this WebBrowser @this, string url) { var tcs = new TaskCompletionSource(); WebBrowserDocumentCompletedEventHandler subscription = null; subscription = (_, args) => { @this.DocumentCompleted -= subscription; tcs.TrySetResult(args.Url.ToString()); }; @this.DocumentCompleted += subscription; @this.Navigate(url); return tcs.Task; } 

第二部分:一个新的Task NavAjaxAsync来运行AJAX的提示(基于Noseratio的代码)

 public static async Task NavAjaxAsync(this WebBrowser @this) { // get the root element var documentElement = @this.Document.GetElementsByTagName("html")[0]; // poll the current HTML for changes asynchronosly var html = documentElement.OuterHtml; while (true) { // wait asynchronously await Task.Delay(POLL_DELAY); // continue polling if the WebBrowser is still busy if (webBrowser.IsBusy) continue; var htmlNow = documentElement.OuterHtml; if (html == htmlNow) break; // no changes detected, end the poll loop html = htmlNow; } return @this.Document.Url.ToString(); } 

第三部分:一个新的Task NavAndAjaxAsync来获取导航和AJAX

 public static async Task NavAndAjaxAsync(this WebBrowser @this, string url) { await @this.NavigateAsync(url); await @this.NavAjaxAsync(); } 

第四部分和最后一部分:来自Stephen的更新Task GetUrlAsync和Noseratio的AJAX代码

 async Task GetUrlAsync(string url) { using (var wb = new WebBrowser()) { var navigate = wb.NavAndAjaxAsync(url); var timeout = Task.Delay(TimeSpan.FromSeconds(5)); var completed = await Task.WhenAny(navigate, timeout); if (completed == navigate) return await navigate; return null; } } 

我想知道这是否是正确的方法。