utils.json_stream

Module Contents

Functions

stream_as_text(stream) Given a stream of bytes or text, if any of the items in the stream
json_splitter(buffer) Attempt to parse a json object from a buffer. If there is at least one
json_stream(stream) Given a stream of text, return a stream of json objects.
line_splitter(buffer,separator=” “)
split_buffer(stream,splitter=None,decoder=None) Given a generator which yields strings and a splitter function,
stream_as_text(stream)

Given a stream of bytes or text, if any of the items in the stream are bytes convert them to text. This function can be removed once we return text streams instead of byte streams.

json_splitter(buffer)

Attempt to parse a json object from a buffer. If there is at least one object, return it and the rest of the buffer, otherwise return None.

json_stream(stream)

Given a stream of text, return a stream of json objects. This handles streams which are inconsistently buffered (some entries may be newline delimited, and others are not).

line_splitter(buffer,separator="

“)

split_buffer(stream, splitter=None, decoder=None)

Given a generator which yields strings and a splitter function, joins all input, splits on the separator and yields each chunk. Unlike string.split(), each chunk includes the trailing separator, except for the last one if none was found on the end of the input.