poolmanager

Module Contents

Classes

PoolManager(self,num_pools=10,headers=None,**connection_pool_kw) Allows for arbitrary requests while transparently keeping track of
ProxyManager(self,proxy_url,num_pools=10,headers=None,proxy_headers=None,**connection_pool_kw) Behaves just like PoolManager, but sends all requests through

Functions

proxy_from_url(url,**kw)
class PoolManager(num_pools=10, headers=None, **connection_pool_kw)

Allows for arbitrary requests while transparently keeping track of necessary connection pools for you.

Parameters:
  • num_pools – Number of connection pools to cache before discarding the least recently used pool.
  • headers – Headers to include with all requests, unless other headers are given explicitly.
  • **connection_pool_kw – Additional parameters are used to create fresh urllib3.connectionpool.ConnectionPool instances.

Example:

>>> manager = PoolManager(num_pools=2)
>>> r = manager.request('GET', 'http://google.com/')
>>> r = manager.request('GET', 'http://google.com/mail')
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> len(manager.pools)
2
__init__(num_pools=10, headers=None, **connection_pool_kw)
_new_pool(scheme, host, port)

Create a new ConnectionPool based on host, port and scheme.

This method is used to actually create the connection pools handed out by connection_from_url() and companion methods. It is intended to be overridden for customization.

clear()

Empty our store of pools and direct them all to close.

This will not affect in-flight connections, but they will not be re-used after completion.

connection_from_host(host, port=None, scheme="http")

Get a ConnectionPool based on the host, port, and scheme.

If port isn’t given, it will be derived from the scheme using urllib3.connectionpool.port_by_scheme.

connection_from_url(url)

Similar to urllib3.connectionpool.connection_from_url() but doesn’t pass any additional parameters to the urllib3.connectionpool.ConnectionPool constructor.

Additional parameters are taken from the PoolManager constructor.

urlopen(method, url, redirect=True, **kw)

Same as urllib3.connectionpool.HTTPConnectionPool.urlopen() with custom cross-host redirect logic and only sends the request-uri portion of the url.

The given url parameter must be absolute, such that an appropriate urllib3.connectionpool.ConnectionPool can be chosen for it.

class ProxyManager(proxy_url, num_pools=10, headers=None, proxy_headers=None, **connection_pool_kw)

Behaves just like PoolManager, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs.

Parameters:
  • proxy_url – The URL of the proxy to be used.
  • proxy_headers – A dictionary contaning headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication.
Example:
>>> proxy = urllib3.ProxyManager('http://localhost:3128/')
>>> r1 = proxy.request('GET', 'http://google.com/')
>>> r2 = proxy.request('GET', 'http://httpbin.org/')
>>> len(proxy.pools)
1
>>> r3 = proxy.request('GET', 'https://httpbin.org/')
>>> r4 = proxy.request('GET', 'https://twitter.com/')
>>> len(proxy.pools)
3
__init__(proxy_url, num_pools=10, headers=None, proxy_headers=None, **connection_pool_kw)
connection_from_host(host, port=None, scheme="http")
_set_proxy_headers(url, headers=None)

Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user.

urlopen(method, url, redirect=True, **kw)

“Same as HTTP(S)ConnectionPool.urlopen, url must be absolute.”

proxy_from_url(url, **kw)