Skip to content

This Python/Django library that provides utilities for safe reading large file-like objects.

License

Notifications You must be signed in to change notification settings

centrumholdings/yowie

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

=============
Yowie library
=============

This Python/Django library provides utilities for safe reading large
file-like objects.


-----------------------------
yowie.open_url(url, **kwargs)
-----------------------------

This function open url and return file object. Url attribute can be local
file path or full url path. Allowed protocols are local file path, file,
http and ftp. kwargs are additional attributes according to protocol -
'mode' for local path and file protocol, 'proxy', 'data' and 'timeout'
(Python >= 2.6) for http and ftp protocols.

Examples:

open_url('/home/praetorian/secret.txt')
open_url('file:///home/praetorian/secret.txt', mode='r')
open_url('http://domain.tld/secret.txt', proxy='172:16:1:100:8000')
open_url('ftp://domain.tld/secret.txt')

----------------------------------------------------
class yowie.LimitedFile(file_descriptor, limit=None)
----------------------------------------------------

File-like object wrapper which raise LimitedFileSizeOverflow exception
if file content length is greater then limit. Default limit is
settings.LIMITEDFILE_MAX_SIZE bytes or 10MB, if setting doesn't exist.
Override read([size]), readline([size]) and readlines([sizehint]) methods.
Sum of data, which has been read, is stored in the total attribute.

-----------------------------------------------------
yowie.LimitedFile.open_url(url, limit=None, **kwargs)
-----------------------------------------------------

Static method, shortcut for:

fd = open_url('my_big_file.txt')
lf = LimitedFile(fd, 1024*1024)

or

lf = LimitedFile.open_url('my_big_file.txt', 1024*1024)

---------------------------------------
exception yowie.LimitedFileSizeOverflow
---------------------------------------

LimitedFile class raise this exception if file content length is greater
then limit.


Example:
--------

>>> from yowie import LimitedFile, open_url

>>> fd = open_url('my_big_file.txt')
>>> lf = LimitedFile(fd, 1024*1024)

>>> content = ''
>>> content += lf.read(1024*512)
>>> content += lf.read(1024*512)
>>> content += lf.read(1024*512)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/yowie/fileutils.py", line 55, in read
    raise LimitedFileSizeOverflow('File content is too long')
yowie.fileutils.LimitedFileSizeOverflow: File content is too long

>>> lf.total
1048577

About

This Python/Django library that provides utilities for safe reading large file-like objects.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages