Skip to content

Commit

Permalink
add readme (#20)
Browse files Browse the repository at this point in the history
  • Loading branch information
skshetry authored Apr 7, 2023
1 parent fe31102 commit c9c83a0
Show file tree
Hide file tree
Showing 2 changed files with 74 additions and 9 deletions.
81 changes: 73 additions & 8 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,11 @@ morefs
Features
--------

* TODO


Requirements
------------

* TODO

*morefs* provides standalone fsspec-based filesystems like:
# ``AsyncLocalFileSystem`` that provides async implementation of ``LocalFileSystem``.
# In-memory filesystems ``DictFileSystem`` built on nested dictionaries and ``MemFS`` built on tries,
and are much faster than fsspec's ``MemoryFileSystem``.
# ``OverlayFileSystem`` that allows to overlay multiple fsspec-based filesystems.

Installation
------------
Expand All @@ -52,10 +49,78 @@ You can install *morefs* via pip_ from PyPI_:
$ pip install morefs
You might need to install with extras for some filesystems:

.. code:: console
$ pip install morefs[asynclocal] # for installing aiofile dependency for AsyncLocalFileSystem
$ pip install morefs[memfs] # for installing pygtrie dependency for MemFS
Usage
-----

AsyncLocalFileSystem
~~~~~~~~~~~~~~~~~~~~

Extended version of ``LocalFileSystem`` that also provides async methods.

.. code:: python
import asyncio
from morefs.asyn_local import AsyncLocalFileSystem
async def main():
fs = AsyncLocalFileSystem(auto_mkdir=False)
async with fs.open_async("foo", mode="w") as f:
await f.write("foobar")
content = await fs._cat("foo")
print(content)
print(fs.cat("foo")) # you can still use sync methods
asyncio.run(main())
DictFS
~~~~~~

DictFS is a nested dictionary-based, in-memory filesystem
and acts more like a real LocalFileSystem.

.. code:: python
from morefs.dictfs import DictFileSystem
fs = DictFileSystem()
MemFS
~~~~~

MemFS is a trie-based in-memory filesystem, and acts like a bucket storage.

.. code:: python
from morefs.memfs import MemFS
fs = MemFS()
OverlayFileSystem
~~~~~~~~~~~~~~~~~

.. code:: python
from morefs.overlay import OverlayFileSystem
# use localfilesystem for write, overlay all filesystems for read
fs = OverlayFileSystem(file={"auto_mkdir": True}, s3={"anon": True})
# or you can pass filesystem instances directly
# as variable positional arguments or with keyword argument `filesystems=[]`
fs = OverlayFileSystem(LocalFileSystem(), s3={"anon": True})
Contributing
------------
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ tests =
pytest-sugar==0.9.5
pytest-cov==3.0.0
pytest-mock==3.8.2
pytest-asyncio==0.19.0
pytest-asyncio==0.21.0
pylint==2.15.0
mypy==0.971
%(all)s
Expand Down

0 comments on commit c9c83a0

Please sign in to comment.