首页 > 代码库 > tornado的非异步阻塞模式
tornado的非异步阻塞模式
【优化tornado阻塞任务的三个选择】
1、优化阻塞的任务,使其执行时间更快。经常由于是一个DB的慢查询,或者复杂的上层模板导致的,这个时候首要的是加速这些任务,而不是优化复杂的webserver。可以提升99%的效率。
2、开启一个单独的线程或者进程执行耗时任务。这意味着对于IOLoop来说,可以开启另一个线程(或进程)处理off-loading任务,这样它就可以再接收其他请求了,而不是阻塞住。
3、使用异步的驱动或者库函数来执行任务,例如gevent , motor。
【例子1】
import timeimport tornado.ioloopimport tornado.webclass MainHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world %s" % time.time())class SleepHandler(tornado.web.RequestHandler): def get(self, n): time.sleep(float(n)) self.write("Awake! %s" % time.time())application = tornado.web.Application([ (r"/", MainHandler), (r"/sleep/(\d+)", SleepHandler),])if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start()
这样在一个tab页中开启http://localhost:8888/sleep/10,同时在另一个tab页访问http://localhost:8888/,会发现没有打印"Hello World"直到第一个页面完成为止。实际上,第一个调用将IOLoop阻塞住了,导致其无法响应第二个请求。
【例子2——非阻塞模式】
from concurrent.futures import ThreadPoolExecutorfrom functools import partial, wrapsimport tornado.ioloopimport tornado.webEXECUTOR = ThreadPoolExecutor(max_workers=4)def unblock(f): @tornado.web.asynchronous @wraps(f) def wrapper(*args, **kwargs): self = args[0] def callback(future): self.write(future.result()) self.finish() EXECUTOR.submit( partial(f, *args, **kwargs) ).add_done_callback( lambda future: tornado.ioloop.IOLoop.instance().add_callback( partial(callback, future))) return wrapperclass SleepHandler(tornado.web.RequestHandler): @unblock def get(self, n): time.sleep(float(n)) return "Awake! %s" % time.time()
unblock修饰器将被修饰函数提交给线程池,返回一个future。在future中添加一个callback函数,并将控制权交给IOLoop。
这个回调函数最终将调用self.finish,并结束此次请求。
Note:这个修饰器函数本身还需要被tornado.web.asynchronous修饰,为了是避免调用self.finish太快。
self.write不是线程安全(thread-safe)的,因此避免在主线程中处理future的结果。
当你使用@tornado.web.asynchonous装饰器时,Tornado永远不会自己关闭连接,需要显式的self.finish()关闭
【完整的demo】
from concurrent.futures import ThreadPoolExecutorfrom functools import partial, wrapsimport time import tornado.ioloopimport tornado.web EXECUTOR = ThreadPoolExecutor(max_workers=4) def unblock(f): @tornado.web.asynchronous @wraps(f) def wrapper(*args, **kwargs): self = args[0] def callback(future): self.write(future.result()) self.finish() EXECUTOR.submit( partial(f, *args, **kwargs) ).add_done_callback( lambda future: tornado.ioloop.IOLoop.instance().add_callback( partial(callback, future))) return wrapper class MainHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world %s" % time.time()) class SleepHandler(tornado.web.RequestHandler): @unblock def get(self, n): time.sleep(float(n)) return "Awake! %s" % time.time() class SleepAsyncHandler(tornado.web.RequestHandler): @tornado.web.asynchronous def get(self, n): def callback(future): self.write(future.result()) self.finish() EXECUTOR.submit( partial(self.get_, n) ).add_done_callback( lambda future: tornado.ioloop.IOLoop.instance().add_callback( partial(callback, future))) def get_(self, n): time.sleep(float(n)) return "Awake! %s" % time.time() application = tornado.web.Application([ (r"/", MainHandler), (r"/sleep/(\d+)", SleepHandler), (r"/sleep_async/(\d+)", SleepAsyncHandler),]) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start()
【ThreadPoolExecutor】
上面涉及到ThreadPoolExecutor两个方法,初始化以及submit,查看帮助
class ThreadPoolExecutor(concurrent.futures._base.Executor) | Method resolution order: | ThreadPoolExecutor | concurrent.futures._base.Executor | __builtin__.object | | Methods defined here: | | __init__(self, max_workers) | Initializes a new ThreadPoolExecutor instance. | | Args: | max_workers: The maximum number of threads that can be used to | execute the given calls. | | submit(self, fn, *args, **kwargs) | Submits a callable to be executed with the given arguments. | | Schedules the callable to be executed as fn(*args, **kwargs) and returns | a Future instance representing the execution of the callable. | | Returns: | A Future representing the given call.
1、max_workers可以处理给定calls的最大线程数目,如果超过这个数目会怎么样呢??
2、submit调用fn(*args, **kwargs),返回一个Future的实例
【Future】
Help on class Future in module concurrent.futures._base:class Future(__builtin__.object) | Represents the result of an asynchronous computation. | | Methods defined here: | | __init__(self) | Initializes the future. Should not be called by clients. | | __repr__(self) | | add_done_callback(self, fn) | Attaches a callable that will be called when the future finishes. | | Args: | fn: A callable that will be called with this future as its only | argument when the future completes or is cancelled. The callable | will always be called by a thread in the same process in which | it was added. If the future has already completed or been | cancelled then the callable will be called immediately. These | callables are called in the order that they were added.
【参考文献】
1、http://lbolla.info/blog/2013/01/22/blocking-tornado
2、http://www.tuicool.com/articles/36ZzA3