Tools¶

Utility for managing HTTP headers for both clients and servers.

It has a dictionary like interface with few extra functions to facilitate the insertion of multiple header values. Header fields are case insensitive, therefore doing:

>>> h = Headers()
>>> h['Content-Length'] = '1050'

is equivalent to

>>> h['content-length'] = '1050'

This Headers container maintains an ordering as suggested by http://www.w3.org/Protocols/rfc2616/rfc2616.html:

The order in which header fields with differing field names are received is not significant. However, it is “good practice” to send general-header fields first, followed by request-header or response-header fields, and ending with the entity-header fields.

—rfc2616 section 4.2

The strict parameter is rarely used and it forces the omission on non-standard header fields.

update(*args, **kwargs)[source]¶

Extend the headers with an iterable.

Parameters:	iterable – a dictionary or an iterable over keys, values tuples.

override(iterable)[source]¶

Extend headers by overriding fields form iterable.

Parameters:	iterable – a dictionary or an iterable over keys, values tuples.

get(key, default=None)[source]¶

Get the field value at key as comma separated values.

For example:

>>> from pulsar.utils.httpurl import Headers
>>> h = Headers(kind='client')
>>> h.add_header('accept-encoding', 'gzip')
>>> h.add_header('accept-encoding', 'deflate')
>>> h.get('accept-encoding')

results in:

'gzip, deflate'

get_all(key, default=None)[source]¶

Get the values at header key as a list rather than a string separated by comma (which is returned by the get() method).

For example:

>>> from pulsar.utils.httpurl import Headers
>>> h = Headers(kind='client')
>>> h.add_header('accept-encoding', 'gzip')
>>> h.add_header('accept-encoding', 'deflate')
>>> h.get_all('accept-encoding')

results in:

['gzip', 'deflate']

has(field, value)[source]¶

Check if value is available in header field.

clear()[source]¶

Same as dict.clear(), it removes all headers.

getheaders(key)[source]¶

Required by cookielib in python 2.

If the key is not available, it returns an empty list.

add_header(key, values)[source]¶

Add values to key header.

If the header is already available, append the value to the list.

Parameters:	key – header name values – a string value or a list/tuple of strings values for header key

remove_header(key, value=None)[source]¶

Remove the header at key.

If value is provided, it removes only that value if found.

flat(version, status)[source]¶

Full headers bytes representation

A python HTTP parser.

Original code from https://github.com/benoitc/http-parser

2011 (c) Benoit Chesneau <benoitc@e-engura.org>

recv_body()[source]¶

return last chunk of the parsed body

is_headers_complete()[source]¶

return True if all headers have been parsed.

is_partial_body()[source]¶

return True if a chunk of body have been parsed

is_message_begin()[source]¶

return True if the parsing start

is_message_complete()[source]¶

return True if the parsing is done (we get EOF)

is_chunked()[source]¶

return True if Transfer-Encoding header value is chunked

Decoder and encoder for the websocket protocol.

version¶

Optional protocol version (Default 13).

kind¶

• 0 for parsing client’s frames and sending server frames (to be used in the server)
• 1 for parsing server frames and sending client frames (to be used by the client)
• 2 Assumes always unmasked data

ping(body=None)[source]¶

return a ping Frame.

pong(body=None)[source]¶

return a pong Frame.

close(code=None)[source]¶

return a close Frame.

continuation(body=None, final=True)[source]¶

return a continuation Frame.

encode(message, final=True, masking_key=None, opcode=None, rsv1=0, rsv2=0, rsv3=0)[source]¶

Encode a message for writing into the wire.

To produce several frames for a given large message use multi_encode() method.

multi_encode(message, masking_key=None, opcode=None, rsv1=0, rsv2=0, rsv3=0, max_payload=0)[source]¶

Encode a message into several frames depending on size.

Returns a generator of bytes to be sent over the wire.

A dictionary-like container of Setting parameters for fine tuning pulsar servers.

It provides easy access to Setting.value attribute by exposing the Setting.name as attribute.

settings¶

Dictionary of all Setting instances available in this Config container.

Keys are given by the Setting.name attribute.

params¶

Dictionary of additional parameters which cannot be parsed on the command line

update(data, default=False)[source]¶

Update this Config with data.

Parameters:	data – must be a Mapping like object exposing the item method for iterating through key-value pairs. default – if True the updated settings will also set their default attribute with the updating value (provided it is a valid one).

copy_globals(cfg)[source]¶

Copy global settings from cfg to this config.

The settings are copied only if they were not already modified.

get(name, default=None)[source]¶

Get the value at name for this Config container

The returned value is obtained from:

• the value at name in the settings dictionary if available.
• the value at name in the params dictionary if available.
• the default value.

set(name, value, default=False, imported=False)[source]¶

Set the Setting at name with a new value.

If default is True, the Setting.default is also set.

parser()[source]¶

Create the argparser for this configuration by adding all settings via the Setting.add_argument() method.

Return type:	an instance of ArgumentParser.

add_to_parser(parser)[source]¶

Add this container settings to an existing parser.

parse_command_line(argv=None)[source]¶

Parse the command line

on_start()[source]¶

Invoked by a Application just before starting.

workers¶

Number of workers

address¶

An address to bind to, only available if a bind setting has been added to this Config container.

copy(name=None, prefix=None)[source]¶

A copy of this Config container.

If prefix is given, it prefixes all non global settings with it. Used when multiple applications are loaded.

configured_logger(name=None)[source]¶

Configured logger.

Class for creating pulsar settings.

Most parameters can be specified on the command line, all of them on a config file.

virtual = None¶

If set to True the settings won’t be loaded.

It can be only used as base class for other settings.

imported = False¶

The actual value for this setting.

const = None¶

A constant value required by some action and nargs selections

choices = None¶

Restrict the argument to the choices provided.

type = None¶

The type to which the command-line argument should be converted

is_global = False¶

True only for global settings.

app = None¶

Setting for a specific Application.

default = None¶

The default value for this setting.

flags = None¶

List of options strings, e.g. [-f, --foo].

action = None¶

The basic type of action to be taken when this argument is encountered at the command line

meta = None¶

Same usage as metavar in the python argparse module. It is the name for the argument in usage message.

name = None¶

The key to access this setting in a Config container.

nargs = None¶

The number of command-line arguments that should be consumed

short = None¶

Optional shot description string

desc = None¶

Description string

validator = None¶

A validating function for this setting.

It provided it must be a function accepting one positional argument, the value to validate.

section = None¶

Setting section, used for creating the settings documentation.

on_start()[source]¶

Called when pulsar server starts.

It can be used to perform custom initialization for this Setting.

get()[source]¶

Returns value

set(val, default=False, imported=False)[source]¶

Set val as the value for this Setting.

If default is True set also the default value.

add_argument(parser, set_default=False)[source]¶

Add this Setting to the parser.

The operation is carried out only if flags or nargs and name are defined.

copy(name=None, prefix=None)[source]¶

Copy this SettingBase

A subclass of dictionary customized to handle multiple values for the same key.

getlist(key)[source]¶

Returns the list of values for the passed key.

extend(key, values)[source]¶

Appends an item to the internal list associated with key.

items()[source]¶

Returns a generator of (key, value) pairs.

lists()[source]¶

Returns a list of (key, list) pairs.

values()[source]¶

Returns a list of the last value on every key list.

A Mapping structures which exposes keys as attributes.

A dictionary which cannot be changed once initialised.

Sorted collection supporting O(log n) insertion, removal, and lookup by rank.

clear()[source]¶

Clear the container from all data.

extend(iterable)[source]¶

Extend this skiplist with an iterable over score, value pairs.

update(iterable)¶

Extend this skiplist with an iterable over score, value pairs.

rank(score)[source]¶

Return the 0-based index (rank) of score.

If the score is not available it returns a negative integer which absolute score is the right most closest index with score less than score.

remove_range(start, end, callback=None)[source]¶

Remove a range by rank.

This is equivalent to perform:

del l[start:end]

on a python list. It returns the number of element removed.

remove_range_by_score(minval, maxval, include_min=True, include_max=True, callback=None)[source]¶

Remove a range with scores between minval and maxval.

Parameters:	minval – the start value of the range to remove maxval – the end value of the range to remove include_min – whether or not to include minval in the values to remove include_max – whether or not to include maxval in the scores to to remove callback – optional callback function invoked for each score, value pair removed.
Returns:	the number of elements removed.

count(minval, maxval, include_min=True, include_max=True)[source]¶

Returns the number of elements in the skiplist with a score between min and max.

Ordered-set equivalent of redis zset.

items()[source]¶

Iterable over ordered score, value pairs of this zset

score(member, default=None)[source]¶

The score of a given member

update(score_vals)[source]¶

Update the zset with an iterable over pairs of scores and values.

remove(item)[source]¶

Remove item for the zset it it exists. If found it returns the score of the item removed.

remove_range(start, end)[source]¶

Remove a range by score.

remove_range_by_score(minval, maxval, include_min=True, include_max=True)[source]¶

Remove a range by score.

clear()[source]¶

Clear this zset.

rank(item)[source]¶

Return the rank (index) of item in this zset.