Requests is a great library for performing HTTP related request and response.
As it’s description, it supports connection pool automatically.
Requests takes all of the work out of Python HTTP/1.1 — making your integration with web services seamless. There’s no need to manually add query strings to your URLs, or to form-encode your POST data. Keep-alive and HTTP connection pooling are 100% automatic, powered by urllib3, which is embedded within Requests.
Actually, the following snippet will not use connection pool provided from urllib3.
requests.get('http://developer.github.com/v3/') requests.get('http://developer.github.com/v3/media/')
Requests use urllib3.PoolManager
in requests.adapters.HTTPAdapter
. Which is mounted in requests.Session.__init__
. Thus, for each request
def request(method, url, **kwargs): session = sessions.Session() return session.request(method=method, url=url, **kwargs)
will create new session, which make requests.request won’t gain any benefit from connection pool. Alternatively, you can use requests.Session to make Requests use connection pool powered by urllib3.
session = requests.Session() session.get('http://developer.github.com/v3/') session.get('http://developer.github.com/v3/media/')
I rewrite a benchmark_request.py from urllib3 benchmark.py.
The result
Completed requests_session_get in 7.067s
Completed requests_get in 9.162s