Merge branch 's/swinx' into swinx-nagydani

Conflicts:
	swarm/docs/sw^3/storage.rst

swarm/docs/refs.bib

@@ -14,26 +14,26 @@ @misc{nakamoto2008bitcoin
 }
 
 @TECHREPORT{ethersphere2016sw3,
-  author = {Viktor Trón and Aron Fischer and Daniel Nagy A and Zsolt Felföldi},
+  author = {Viktor Tron and Aron Fischer and Daniel Nagy A and Zsolt Felföldi},
   title = {swap, swear and swindle: incentive system for swarm},
   institution = {Ethersphere},
   note = {Ethersphere Orange Papers 1},
   year = {2016}
 }
 
 @TECHREPORT{ethersphere2016smash,
-  author = {Viktor Trón and Aron Fischer and Daniel Varga},
+  author = {Viktor Tron and Aron Fischer and Daniel Varga},
   title = {smash-proof: auditable storage in swarm secured by masked audit secret hash},
   institution = {Ethersphere},
   note = {Ethersphere Orange Papers 2},
   year = {2016}
 }
 @TECHREPORT{ethersphere2016swap,
-  author = {Viktor Trón and Aron Fischer},
+  author = {Viktor Tron and Aron Fischer},
   title = {State channel networks: accounting and litigation on the blockchain (tentative title)},
   institution = {Ethersphere},
-  note = {Ethersphere Orange Papers},
-  year = {2016 (to be published)}
+  note = {Ethersphere Orange Papers  (to be published)},
+  year = {2016}
 }
 
 @misc{filecoin2014,

swarm/docs/smash/conf.py

@@ -30,14 +30,17 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    # 'sphinx.ext.intersphinx',
-    # 'sphinx.ext.todo',
     'sphinx.ext.mathjax',
     'sphinx.ext.ifconfig',
-    # 'sphinxcontrib.numref',
     'sphinxcontrib.bibtex',
 ]
 
+
+natbib = {
+  "file": "refs.bib"
+}
+
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
@@ -56,7 +59,7 @@
 project = u'swarm'
 # copyright = u'2016, ΞTHΞЯSPHΞЯΞ'
 copyright = u'2016, ΞTHΞRSPHΞRΞ'
-author = u'viktor trón, aron fischer, daniel varga'
+author = u'viktor trón, aron fischer'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -233,15 +236,15 @@
 # Additional stuff for the LaTeX preamble.
 'preamble': '\\setcounter{secnumdepth}{5}\
 \\pagestyle{plain}\
+\\usepackage[utf8]{inputenc}\
 \\pagenumbering{arabic}\
-   \\usepackage{color}\
+  \\usepackage{color}\
   \\usepackage{amsmath}\
   \\usepackage{stmaryrd}\
   \\newcommand\\concat{\\mathbin{\\|}}\
   \\newcommand\\defeq{\\stackrel{\\text{\\tiny def}}{=}}\
   \\newcommand\\defequiv{\\stackrel{\\text{\\tiny def}}{\\Leftrightarrow}}\
     \\newcommand\\chunk{c}\
-    \\newcommand\\nexists{\\not\\exists}\
     \\newcommand\\segment{\\sigma}\
     \\newcommand\\level{\\lambda}\
     \\newcommand\\node{\\nu}\
@@ -297,7 +300,7 @@
 'maketitle': '\
 \\definecolor{orange}{rgb}{1.0, 0.49, 0.0}''\\definecolor{orange}{rgb}{1.0, 0.55, 0.0}\
 \\renewcommand{\\releasename}{\\huge\\scshape auditable storage for swarm\\\\secured by masked audit secret hash}\
-\\date{\\large\\rm draft version Aprw 2016\\\\{\color{orange} {\\bfseries\\scshape ethersphere orange papers 2}}\\\\ licensed under the Creative Commons Attribution License http://creativecommons.org/licenses/by/2.0/}\
+\\date{\\large\\rm draft version Apr 2016\\\\{\color{orange} {\\bfseries\\scshape ethersphere orange papers 2}}\\\\ licensed under the Creative Commons Attribution License http://creativecommons.org/licenses/by/2.0/}\
 \\release{}\
 \\title{smash-proof}\
 \\enlargethispage{.5cm}\
@@ -316,7 +319,7 @@
 #  author, documentclass [howto, manual, or own class]).
 latex_documents = [
   (master_doc, 'smash.tex', u'',
-   u'viktor trón, aron fischer, daniel varga',  'howto'),
+   u'viktor trón, aron fischer',  'howto'),
    # u'viktor trón', 'manual'),
 ]
 

swarm/docs/smash/index.rst

@@ -18,10 +18,3 @@ Welcome to the swarm documentation!
 .. raw:: latex
   \newpage
 
-.. Indices and tables
-.. ==================
-
-.. * :ref:`genindex`
-.. * :ref:`modindex`
-.. * :ref:`search`
-

swarm/docs/smash/smash.rst

@@ -7,14 +7,13 @@
     \newpage
 
 
-
 Introduction
 =========================================
 
 Proof of custody [#]_ is a construct that proves to an auditor that a storer has custody of data without the auditor having it. Such constructs can be used to audit the integrity of remotely stored documents trustlessly. In the context of a distributed file storage system, proof of custody schemes need to consider both storage and network communication overhead and offer flexible trade-offs between security and cost.
 
-..  rubric:: Footnotes
-..  [#] proof of existence, proof of storage, proof of resource, proof of retrievability
+.. rubric:: Footnotes
+.. [#] profof of existence, proof of storage, proof of resource, proof of retrievability
 
 In the specific context of swarm, we need to make sure we have a scheme that best adapts to
 
@@ -27,7 +26,7 @@ In the specific context of swarm, we need to make sure we have a scheme that bes
 * existing escalation to retributive measures: litigation on the ethereum blockchain
 
 
-The proof of custody scheme proposed here draws on earlier work based on well understood basic cryptography (:cite:`shacham2013compact`, :cite:`bowers2009proofs`), but in light of the above considerations a novel approach was warranted.
+The proof of custody scheme proposed here draws on earlier work based on well understood basic cryptography (:cite:`shacham2013compact`, :cite:`bowers2009proofs`), but in light of the above considerations a novel approach was warranted. In what follows we spell out a proof of custody scheme and present audit schemes that use variants of the scheme depending on
 
 Auditing chunks with pregenerated secrets
 ============================================
@@ -578,7 +577,7 @@ It is easy to see that this process follows the order defined in the previous se
 
 **Failure:**
 
-14. If at any time during the audit process there is no response to an audit request about a chunk, the guardian of that chunk is looked up by the responsible auditor and is sent an Ash proof request. Upon receiving a repsonse to the ASH proof request, the auditor verifies the proof and calculates the ASH secret and proceeds according to A-1--A-8. If there is no response, the audit is escalated and litigation on the blockchain starts: the auditor sends the ASH proof challenge to the blockchain accusing the guardian of having lost the chunk in question. From here on the standard deadline for refutation starts. The exact procedure is discussed in :cite:`ethersphere2016sw3`.
+14. If at any time during the audit process there is no response to an audit request about a chunk, the guardian of that chunk is looked up by the responsible auditor and is sent an Ash proof request. Upon receiving a repsonse to the ASH proof request, the auditor verifies the proof and calculates the ASH secret and proceeds according to steps 1--8. If there is no response, the audit is escalated and litigation on the blockchain starts: the auditor sends the ASH proof challenge to the blockchain accusing the guardian of having lost the chunk in question. From here on the standard deadline for refutation starts. The exact procedure is discussed in :cite:`ethersphere2016sw3`.
 
 15. Errors are detected in two ways: either an intermediate auditor finds that one of their children returned an audit secret that does not match the verification bits, or the main auditor finds that the final secret does not match the respective MASH. When this happens we need to find the culprit, i.e., the node that lost the chunk. This is done by sending out successive ASH proof challenges. Luckily, due to the iterative error coding scheme used (in which one segment's ASH is the input to the seed of the next challenge), once an error occurs the probability of it staying undetected falls exponentially. Therefore the culprit is most likely to be among the most recently audited chunks.
 
@@ -593,14 +592,14 @@ Ensuring correct syncing and distribution
 ------------------------------------------------------------
 
 As it turns out, collective auditing has great advantages in policing correct syncronisation.
-As a result of recursive audits, when audit responses are retrieved, the audit requests come from nodes independent of the owner. This helps nodes to identify neighbours that refused to sync. If an audit request reaches a node that is most proximate to the target chunk, the node recognizes that it is a chunk that it was  supposed to receive while syncing with one of its peers. If it did not, then it sends an  audit request to the chunk's guardian and feedback to its parent auditor.
+As a result of recursive audits, when audit responses are retrieved, the audit requests come from nodes independent of the owner. This helps nodes identify neighbours that refused to sync. If an audit request reaches a node that is most proximate to the target chunk, the node recognizes that it is a chunk that it was  supposed to receive while syncing with one of its peers. If it did not, then it sends an  audit request to the chunk's guardian and feedback to its parent auditor (see :numref:`Figure %s <fig:policing>`).
 
-This can be thought of as a  warning to the guardian (or in fact the node that acts as custodian). If they still keep the chunk to themselves, they will lose money their deposit as a result of litigation.
-Even if they are innocent, they are motivated to forward since that is cheaper still than litigation. Therefore they will forward the audit request to  their appropriate online peer towards the node they had originally forwarded it to. If all nodes delegate and forward, the proximate node will eventually receive the chunk (see :numref:`Figure %s <fig:policing>`.
+This can be thought of as a  warning to the guardian (or in fact the node that acts as custodian). If they still keep the chunk to themselves, they will lose money as a result of litigation.
+Even if they are innocent, they are motivated to forward since that is cheaper still than litigation. Therefore they will forward the audit request to  their appropriate online peer towards the node that they had forwarded the original store request to. If all nodes delegate and forward, the proximate node will eventually receive the chunk and can act as  custodian.
 Interestingly, this situation could also happen as a  result of network growth and  latency. We conclude that SWINDLE recursive auditing can repair retrievability [#]_ .
 
 .. rubric:: Footnotes
-.. [#] Note that adaptation to network  growth and shrinking is taken care of by the syncing protocol. However if network connections are saturated and/or nodes have not yet heard of each other it could happen that they are genuine yet appear not synchronized. We restrict these cases to the case when the offending node issues a proof that they believed to be the most proximate one to the target.
+.. [#] Note that adaptation to network  growth and shrinking is taken care of by the syncing protocol. However if network connections are saturated and/or nodes have not yet heard of each other it could happen that they are genuine yet appear not syncronized.
 
 
 ..  _fig:policing:
@@ -610,21 +609,21 @@ Interestingly, this situation could also happen as a  result of network growth a
     :alt: repair retrievability with audit
     :figclass: align-center
 
-    The arrows represent local transactions between connected peers. After the audit reaches the closest node and the chunk is not found, the closest node challenges the guardian who in turn challenges the node it originally bought a receipt from, and so on until the challenge lands on the current custodian who now has the chance to connect to the node closest to the chunk address or at least a registered node closer.
+    The arrows represent local transactions between connected peers. After the audit reaches the closest node and the chunk is not found, the closest node challenges the guardian who in turn challenges the node it originally bought a receipt from, and so on until the challenge lands on the current custodian who now has the chance to connect to the node that is actually closest to the chunk address (or at least closer).
 
-If the proximate node gets the chunk, it calculates the audit secret and the audit can continue. If there is a delay longer than the timeout, the audit concludes and litigation starts. Note that litigation is only possible as long as the initiator includes the address of the real proximate node.
-If such an address is not provided, an offending node further out claiming to be proximate cannot be  prosecuted.
+If the proximate node gets the chunk, it calculates the audit secret and the audit can continue. If there is a delay longer than the timeout, the audit concludes and litigation starts against the impostor custodian. The initiator includes the address of the known closer node without which an offending node further out claiming to be righful custodian cannot be  prosecuted.
 
+The collective audit can also be used to repair chunks in erasure coded collections, this is not primarily about reporting aid to balance out lost reliability for large files
 
 Conclusion
 =============
 
-In this paper we presented a simple proof of custody formula inspired by the Wilkinson--Buterik proof of storage used by Storj (:cite:`wilkinsonetal2014storj`). The formula offers 3 different types of challenge that auditors can use in different stages. We specified an auditing and litigation scheme that has ideal properties to secure the swarm against chunk loss.
+In this paper we presented a simple proof of custody formula inspired by the Wilkinson--Buterik proof of storage used by Storj (:cite:`wilkinsonetal2014storj`). The formula offers 3 different types of challenge that auditors can use in different stages. We specified an auditing and litigation scheme that has ideal properties to secure the swarm against chunk loss (:cite:`wilkinsonetal2014storj`).
 
 SMASH proofs offer integrity checking for chunks as well as for documents and document collections that
 
 * allows storers to initiate and control audits without storing any information other than the swarm hash of the chunk;
-* allows owners to outsource auditing without a trusted third party allow;
+* allows owners to outsource auditing without a trusted third party;
 * it provides a seed generation algorithm for securing large document collections with a single audit secret so it scales for both storage and bandwidth;
 * the successive seeds contain error detection which makes it very efficient to find offending nodes without remembering the (masked) secret for each chunk;
 * allows easy verification by third parties like smart contracts to serve as evidence  when it comes to litigation on the blockchain;
@@ -639,10 +638,6 @@ We outlined an auditing and litigation protocol which
 * helps nodes identify greedy peers that do not forward chunks;
 * offer a way to repair improper syncronisation state.
 
-.. bibliography:: ../refs.bib
-   :cited:
-   :style: alpha
-
-.. apalike not available so boring
-
 
+.. bibliography:: ../refs.bib
+   :style:  plain

swarm/docs/sw^3/abstract.rst

@@ -3,12 +3,12 @@
 
    This paper [#]_ is the result of our research into incentives for decentralised storage and distribution in the context of implementing an incentive layer for swarm.
 
-   Swarm is a distributed storage platform and content distribution service, a native base layer service of the ethereum web 3 stack. The primary objective of Swarm is to provide a sufficiently decentralized and redundant store of Ethereum's public record, in particular to store and distribute Đapp code and data as well as block chain data. From an economic point of view, it allows participants to efficiently pool their storage and bandwidth resources in order to provide the aforementioned services to all participants.
+   Swarm is a distributed storage platform and content distribution service, a native base layer service of the Ethereum web 3 stack. The primary objective of Swarm is to provide a  decentralized and redundant store of Ethereum's public record, in particular to store and distribute dapp code and data as well as block chain data. From an economic point of view, it allows participants to efficiently pool their storage and bandwidth resources in order to provide these services to all participants.
 
-   The goal is a peer-to-peer storage and serving solution that is DDOS-resistant, has zero-downtime, is fault-tolerant and censorship-resistant as well as being self-sustaining due to a built-in incentive system.
+   The goal is a peer-to-peer serverless hosting, storage and serving solution that is DDOS-resistant, has zero-downtime, is fault-tolerant and censorship-resistant as well as  self-sustaining due to a built-in incentive system.
 
-   As storage layer for interactive Web 3.0 applications and blockchains, the swarm must satisfy some key requirements. The swarm has to show dynamic scalability, i.e. it must adapt to sudden surges in popular demand, and at the same time the swarm must also preserve niche content and ensure its availablility. In this document we aim to present an incentive structure for nodes in the swarm, that is in alignment with and conducive to these requirements. Putting aside the -- otherwise very important -- question of search and content promotion, this paper focuses on the issues of bandwidth and storage.
+   As the storage layer for interactive web 3 applications and blockchains, the swarm must satisfy some key requirements. The swarm has to show dynamic scalability, i.e. it must adapt to sudden surges in popular demand, and at the same time the swarm must also preserve niche content and ensure its availablility. In this document we aim to present an incentive structure for nodes in the swarm, that is in alignment with and conducive to these requirements. Putting aside the -- otherwise very important -- question of search and content promotion, this paper focuses on the issues of bandwidth and storage.
 
 
 .. rubric:: Footnotes
-.. [#] The authors are indebted to Gavin Wood, Vitalik Buterin and Jeffrey Wilcke for their daring  vision of the next generation internet and for their original concepts of web 3 and swarm. We thank Vitalik Buterin, Nick Johnson and Andras Kornai for their meticulous review of this paper.
+.. [#] The authors are indebted to Gavin Wood, Vitalik Buterin and Jeffrey Wilcke for their daring  vision of the next generation internet and for their original concepts of web 3 and swarm. We thank Vitalik Buterin, Nick Johnson, Piper Merriam and Andras Kornai for their meticulous review of this paper. Their names here by no  means imply endorsement. All errors are ours.