Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for circular payments #1257

Closed
tuloski opened this issue Aug 30, 2015 · 41 comments
Closed

Support for circular payments #1257

tuloski opened this issue Aug 30, 2015 · 41 comments
Assignees
Labels
Feature Request Used to indicate requests to add new features

Comments

@tuloski
Copy link

tuloski commented Aug 30, 2015

I think that in Ripple network the circular payments are a mandatory tool (arbitrage and other usages).
Circular payments are when the source and destination currencies are the same IOU but with a path that is not empty. Example: USD.Bitastamp-->XRP-->BTC.bitstamp-->USD.Bitstamp

@vinniefalco
Copy link
Contributor

I'm strongly against such a feature

@tuloski
Copy link
Author

tuloski commented Feb 3, 2017

Disadvantages: removes liquidity.
Advantages: useful to re-equilibrate the markets.

But sooner or later people will find the unbalanced markets and will remove the liquidity, with or without this feature. It is already possible to do such transaction with the "trick" of sending two transactions with the path split (with some more risks).

@MarkusTeufelberger
Copy link
Collaborator

@vinniefalco - Why? It only means that there will be 2 transactions sent instead of a single one as a workaround in case such a path is desired.

@vinniefalco
Copy link
Contributor

It seems to be an uncommon use-case.

@MarkusTeufelberger
Copy link
Collaborator

If I want to be snarky, rippled itself seems to be an uncommon use-case. :-P

Right now it is an impossible use case (circular transactions are not accepted by rippled), actually uncommon features are for example the freeze flag, which rarely ever has been used to date.
https://forum.ripple.com/viewtopic.php?p=63362 states that this is a planned feature.

@vinniefalco
Copy link
Contributor

David looks like he's for it (although, I don't know if he's changed his mind). We don't always agree.

@donovanhide
Copy link
Contributor

It's not an uncommon use case, rogue5HnPRSszD9CWGSUz8UGHMVwSSKF6 has submitted 768,770 such circular payments, with the workaround of partitioning the circuits into multiple payments and offers.

@tuloski
Copy link
Author

tuloski commented Feb 3, 2017

ahaha but I think me and you were the only crazy one doing this...I gave up because of bad fees in the payment engine, but now that should be fixed. Also I gave up because of your bot :)

@donovanhide
Copy link
Contributor

@tuloski rEiUs9rEiGHmpaprkYDNyXnJYg4ANxWLy9 is also doing circular payments by mirroring all the circuits that rogue5HnPRSszD9CWGSUz8UGHMVwSSKF6 generates, so maybe that makes three of us :-) Other people have requested it previously as well.

This feature would be useful as there would no longer be a need to trust the IOU's of the intermediate legs, i.e. the last IOU of each path. Tangentially, it seems arbitrary that payments to the same account require that TrustSet limit is set anyway; OfferCreates don't require it. Also, it could reduce payment transaction volume.

For this feature to be useful, the pathset limits would need to be altered. Currently you can have 4 sets of 8. It would be preferable if the sum of all pathsets was the limit i.e. you can have a single path of length 32. The longest profitable path I've ever found was 25 steps in length.

@sublimator
Copy link
Contributor

sublimator commented Feb 4, 2017 via email

@vinniefalco
Copy link
Contributor

The longest profitable path I've ever found was 25 steps in length.

Wow!!! I had no idea

@sublimator
Copy link
Contributor

sublimator commented Feb 4, 2017 via email

@JoelKatz
Copy link
Collaborator

JoelKatz commented Feb 4, 2017

Deferred credits makes most circular paths work. However, if a path tries to use the same limited liquidity twice, it will stall and exclude itself from the rest of the payment. That can result in very sub-optimal payment execution.

The payment engine must run each payment path backwards and then forwards. This is unavoidable. If two hops in the path attempts to use the same liquidity (for example, an XRP->BTC offer and an XRP->USD offer from the same owner) then which one succeeds and which one fails can depend on the order in which they are examined, causing the path to produce a different result in the forward direction as in the reverse.

We could allow circular paths with no limits; however, such paths could execute in horribly suboptimal ways as a stalled path removes itself from consideration. Also, this can result in payments that examine large numbers of orders and only consume a very small number of them.

On balance, I don't think they should be allowed. However, we could do the following:

  1. Disallow circular paths (tem error) unless a "circular paths allowed" flag is set. This protects people from accidentally getting suboptimal payment executions.
  2. Require a higher fee if the flag is set. This compensates the network from the higher work required.
  3. Explain to people that if they set this flag, their payments are more likely to tec or to get bad rates and it's only for people who know what they're doing.

I'm not sure it's possible to justify this effort though.

@tuloski
Copy link
Author

tuloski commented Feb 4, 2017

I still don't get (after hearing it 5-6 times) the forward-backward thing.
Isn't enough to run the payment either forward or backward? Then if the same liquidity is used in a previous hop, and there is not enough in another hop, the second hop fails, so as the transaction. I see that for partial payments that could become complicated but still there is a solution to the problem...or not?

I don't get why if I can compute if the liquidity is enough to succeed, the payment engine cannot...?!

BTW I see the unjustified effort.

@JoelKatz
Copy link
Collaborator

JoelKatz commented Feb 4, 2017

Let me try to explain it again as clearly as I can:

Say you submit a payment with three paths that use order books. Each path will start at a certain exchange rate and that rate will change as the path goes deeper into the order book. The payment engine will execute each path until its exchange rate changes, then really use the one that provided the best exchange rate. Then it will repeat the process getting worse and worse exchange rates until the payment completes.

Now, think about when it executes the first path. Say the payment is trying to deliver $100 and can spend up to 1,000 XRP. The engine has no idea how much money to put in nor how much will come out. There are two ways to do this and they both have the same issue. I'll explain the method the engine actually uses and you can reason out why the other way has the same problem.

The engine assumes that the full $100 will come out of the path. And it begins executing the path in reverse. Say the last hop takes the full $100 and needs .1 BTC to do it. So fine, the engine tries to make 0.1 BTC come out of the previous step. But say that step is an order book with only one offer at the best rate and that offer only outputs 0.05 BTC. That's fine, the engine propagates the 0.05 BTC to the previous step, but we now know that $100 is not going to come out.

When this execution finishes, the engine now knows that to run this path for one increment it must put, say $12 in. But it has no idea how much will come out, just that it's $100 or less. So it must run the path forwards putting the $12 in.

Now, say it runs the path forwards to try to put the $12 in. But say when it gets to that order book running forwards, that order is gone because a step that's earlier going forwards consumed it. What can the engine do other than stop using this path?

In summary:

  1. When it starts executing a path, the engine has no idea how much to put in nor how much will come out.
  2. To determine how much to put in, it runs the path in reverse. Unless this is the last increment of the payment, it cannot use these results.
  3. The engine now knows what to put in, so it can run the path forwards and produce the final results. However, if the path does not execute the same as it did backwards, the payment engine is jammed.

You can't just ignore the jam or work through it. For example, the last increment of a non-partial payment must deliver the precise remaining amount.

@tuloski
Copy link
Author

tuloski commented Feb 4, 2017

Ok, it is gonna take time to digest it...but I like this kind of riddles.
Tomorrow I'll answer and try to make a counterexample and my ideas.

BTW I don't understand the 12$...wasn't the input XRP? I got lost at that point...and what do you mean for one increment?

@donovanhide
Copy link
Contributor

I'd imagine most users of circular payments would only include a single pathset because they know that it will be profitable, so the situation of a pathset conflicting with liquidity consumed by another pathset should not arise. The simple way to enforce this is that a payment where the amount and sendmax are the same IOU, or are both XRP, must have only one pathset.

@JoelKatz
Copy link
Collaborator

JoelKatz commented Feb 4, 2017

@tuloski I might not have kept the currencies straight. I should have done it on paper to be 100% sure to be consistent. But the point is, the engine runs the path backwards (possibly limiting at each step) to find out how much currency to put in. Then it runs the path forwards to see what actually comes out. If any step in the forward execution is unable to take the amount of currency that comes into it from the previous step, the path jams. If the amount that comes out is not the expected amount out, again the path jams.

If two hops in the path compete for the same limited liquidity, then the path will not produce the same results forwards and backwards. This makes it impossible for the engine to successfully execute the path. The path will jam, and the engine will stop using that path.

@JoelKatz
Copy link
Collaborator

JoelKatz commented Feb 4, 2017

@donovanhide The issue is not conflicts with other paths. That's not an issue because the engine only runs one path at at time. The issue is a path that conflicts with itself. For example, consider a path that visits the same order book twice. Or consider a path that visits two orders books that both pay out XRP and the tip offer in both books comes from the same person who can only afford to pay out on one of them. If a path does not execute the same forwards as backwards, the engine cannot execute that path and it will jam.

Imagine a path that does BTC->XRP->USD->XRP
Now imagine the tip offer in the BTC->XRP order book is owned by the same account as the tip offer in the USD->XRP order book. When running the path backwards, the USD->XRP offer is funded and the BTC->XRP offer is not. Running the path forwards, the BTC->XRP offer is funded and the USD->XRP offer is not.

The simplest way to prevent people from accidentally causing themselves pain is to prohibit such paths. Otherwise, non-jammed paths may allow the payment to execute but cause the resulting rate to be very poor.

@donovanhide
Copy link
Contributor

@JoelKatz I've never actually created a path that visits an orderbook twice, not sure why you would ever want to do that as calculating the input and output liquidity and therefore profitability would be equally as much of a nightmare for the arbitrage path-finder as it would be for rippled. Simple solution is just to limit the number of visits to an individual orderbook in a path (not pathset, sorry for previous confusion) to one and the number of visits to XRP to be one. Are there any actual examples in the ledger history of payments with paths which exceed this limit? Is this an issue due to auto-bridging always trying to route via XRP?
Currently you can't use rippling liquidity of a single account more than once in a path anyway, so there is existing validation already present with similar characteristics.

@donovanhide
Copy link
Contributor

Probably should be clearer... I've never created a path that uses the same IOU twice, so I never encounter an unfunded offer which wasn't funded at the start of the state for a ledger. This whole feature request probably appears to be more an argument for not having unfunded offers, but that's a whole different conversation :-)

@JoelKatz
Copy link
Collaborator

JoelKatz commented Feb 5, 2017

@donovanhide The problem can occur even with a path that doesn't visit the same orderbook twice. It can occur with any path that visits the same asset twice. The simplest solution to avoiding unexpected results is to prohibit such paths.

@tuloski
Copy link
Author

tuloski commented Feb 5, 2017

So, I try to make a complete example of the WORST case: partial payment, and same IOU in two hops (BTC in this case), and we suppose that BTC are only provided by 1 liquidity provider, such that if consumed in one hop, there are less or there aren't for the other hop.
Let's suppose I want to deliver 1000 XRP for 900 XRP using this path:

XRP--BTC--USD--CNY--BTC--JPY--XRP (BTC are twice in the path...this could happen only for a path of minimum 6 hops).

Doing it in reverse:

  • To deliver 1000 XRP I need 100 JPY
  • To have 100 JPY I need 0.1 BTC, the liquidity provider has only 0.15, so after this hop there are left 0.05BTC
  • To have 0.1 BTC I need 500 CNY
  • To have 500 CNY I need 10 USD
  • To have 10 USD I need 0.1 BTC --> here is the problem, since the liquidity provider has only 0.05, but I don't see the problem.
    At this point there are various options: I can consider that only 0.05 BTC are used, so that the payment cannot deliver anymore all the 1000 XRP, but only 500XRP (or the equivalent calculating the correct quality forward).
    This case is suboptimal, but it solves the problem.
  • Finally to have 0.05 BTC USD I need to put in 450XRP
  • I have the complete payment: 450XRP-->0.05BTC-->5USD-->250CNY-->0.05BTC-->50JPY-->500XRP
  • If the payment was not partial, then it has to fail.

I see it is not optimal, since only 0.1 BTC were used instead of the full 0.15BTC, but with other ways it could be optimized (split the liquidity for the two hops, recursively,...).
Probably splitting the liquidity in the hops is the optimal solution, but I need to prove it.
Also I think the optimization problem is convex, so that any optimization tool should find the unique maximum/minimum.

BTW in a more general case: why just forbid transactions with different forward-backward results, but allow circular payments? Not all circular payments will cause that behavior. Isn't it? It would require zero work.

@donovanhide
Copy link
Contributor

donovanhide commented Feb 5, 2017

@tuloski The situation where you would revisit the same IOU twice in a path would mean you have found a high speed solution to the generalized network max flow problem. That is flow-generating loops within the main circuit. If you have done such a thing, you should probably be trading on a real world forex platform, as you would be making a fortune :-)

https://www.cs.princeton.edu/~wayne/papers/gain-scaling_talk.pdf

I honestly can't imagine anyone is doing this on the Ripple network (I would have spotted it :-)). The simple case of non-revisiting IOU paths is more than suitable for your and my use case. Recursive application of Bellman Ford, using the negated logarithm of the exchange rates allows you to find a set of profitable paths, where you remove part or wholly consumed offers or rippling liquidity from the graph on each pass. Theoretically they might not be the most optimal combination, but they are profitable and can be found in a matter of tens of milliseconds...

@tuloski
Copy link
Author

tuloski commented Feb 5, 2017

@donovanhide Personally I wasn't visiting the same IOU twice or more, but we have to think at what rippled should do in that case...not only thinking about realistic paths for arbitrage. Also the example of Joel BTC->XRP->USD->XRP-->BTC is not good for arbitrage, because you'd always have quality < 1, but still a path like that could be sent in a payment in RCL.

I'll think about the generalized max flow problem ;)...I might have a new business

@donovanhide
Copy link
Contributor

donovanhide commented Feb 5, 2017

Simplest feature request would be: if Amount and SendMax are the same currency, each path in the pathset must not contain the same currency and issuer pair more than once. No need for a flag as the circularity is implied. Extend the limits for path elements to be the sum of all path lengths, and this number must be less than or equal to 32.

@robcat
Copy link
Contributor

robcat commented Feb 6, 2017

I am enjoying this discussion, but can someone explain why anyone would ever send a payment of this form (B¹ and B² are the same asset)?
A → … → B¹ → … → B² → … → A

Looking at the internal loop B¹ → … → B², there are only two basic cases:

  • It is not profitable, so it can be collapsed into B;
  • It is profitable, and the flows of B¹ → … → B² and B² → … → A → … → B¹ are different; in this case it's more profitable to split the two loops.

To me the sensible default should be: internal loops are disallowed (or automatically collapsed).

@donovanhide
Copy link
Contributor

donovanhide commented Feb 6, 2017

@robcat because the steps between B¹ and B² result in an increase in the holding of that asset! Given that the arbitrageur probably has a base asset that he/she wants to increase their holding of, which is not B (probably XRP given the current state of the orderbook pairs) any "flow-generating" loop which allows an increase in the in/out ratio of a long payment path is beneficial. However, max flow in generalized networks is a hard problem, so in reality, you'd tend to dismiss those paths and go for the paths without those additional loops.

Great Superbowl by the way!

@robcat
Copy link
Contributor

robcat commented Feb 6, 2017

@donovanhide Ok, understood!

Just to make clear how far should circular payments could go, let me propose an example:

There is loop of trustlines between three branches of a bank, such that is possible for each of them to make a circular payment in the right direction and profit (due to the quality in/out parameters). These branches are completely disconnected from the graph otherwise, because they have decided to use RCL just to use it to settle their debts with each other.

Without circular payments: it's impossible to make the arbitrage from outside the loop.

With circular payments: An external arbitrageur can extract the profit even without orderbooks (since payments are executed from the back to the front, as @JoelKatz explained).
After the circular payment there is more liquidity, since the balances between the branches have been reduced. Is this desirable?

@tuloski
Copy link
Author

tuloski commented Feb 6, 2017

@robcat Hi Roberto, I don't get how an external arbitrageur could use those internal trustlines? If they are completely disconnected it means there are no books with their IOUs (trustlines) and the trust is only between them...no way an external can use a payment involving them.

@MarkusTeufelberger
Copy link
Collaborator

I don't think circular payments require trustlines to be set up, they depend on (funded parts of) offers and/or direct rippling, if enabled (which could be viewed as implicit offer). They will modify various trust lines as a result, but they operate on order books mostly.

The idea is to be able to submit a path that instead of going via various hops from A over B to C, will end up at A again. This means you already need to own some of A to begin with and at the end of the payment you'd ideally have more of A than you started out with.

As far as I understand JoelKatz's concerns he thinks it would be difficult to allow/evaluate "sub-circles" (so starting at A and ending at A, but also in between reaching B twice). While this might also be a desirable thing (you identify e.g. 4 profitable circles that start and end at XRP in the current ledger and submit a single transaction that consumes all 4 at the same time, taking the increased output of the first one as the input of the second one and so on) this would likely make it harder to evaluate how much money actually is sent/received if you just enter generic amounts (e.g. "spend/receive as much as possible").

@robcat
Copy link
Contributor

robcat commented Feb 6, 2017

@robcat Hi Roberto, I don't get how an external arbitrageur could use those internal trustlines? If they are completely disconnected it means there are no books with their IOUs (trustlines) and the trust is only between them...no way an external can use a payment involving them.

Well, of course it's not possible at the moment (since circular payments are not allowed). In my strange transaction I was using the following observations:

  • A trustline is already created automatically when making self-payments using an orderbook (the arbitrageur could create one beforehand, anyway)
  • The transaction is desirable because it frees up trustlines
  • The invariants on the bank side are respected
  • The payment executes correctly from the back to the front: the balance is first created and then (almost) destroyed, and with no need of trust on the bank side
  • The path forward could execute in the forward direction using a "virtual" trustline, that disappears at the end (very much like a virtual particle)

But in the end it's too complex.

@JoelKatz What about the safety of a very simple loop BTC → EUR → USD → BTC (3 order books, different assets)? I think that the autobridging feature could potentially expand it to BTC → XRP → EUR → XRP → USD → XRP → BTC and create problems.

@donovanhide
Copy link
Contributor

donovanhide commented Feb 6, 2017

@robcat I think QualityIn/Out settings on trustlines are probably the most obscure and unusued feature of ripple. I ignore trustlines with those values set as I could never get the figures to reconcile after a Payment used their rippling liquidity.

A trustline is already created automatically when making self-payments using an orderbook.

This is not true for a Payment, but is true for an OfferCreate.

@robcat
Copy link
Contributor

robcat commented Feb 6, 2017

@donovanhide

This is not true for a Payment, but is true for an OfferCreate.

You are right, but any plausible implementation of circular payments I could think of would allow to construct a perfect equivalent of an OfferCreate in the following way:

OfferCreate = {
    TransactionType: OfferCreate,
    Account: sender,
    TakerGets: 5,
    TakerPays: {
      currency: BTC,
      issuer: gateway,
      value: 1
    }
}
SelfPayment = {
  TransactionType : Payment,
  Account : sender,
  Destination : sender,
  SendMax: 5,
  Amount : {
      currency: BTC,
      issuer: gateway,
      value: 1
  },
  Path (can be auto-filled): [
    {account: sender},
    {account: gateway, currency: BTC},
    {account: sender}
  ]
}

I would expect these two equivalent transactions to produce the same effects†, and thus behave in a similar way. In both cases who signs the transaction implicitly agrees to hold the asset he/she tried to buy.

† with the only difference that the effects that OfferCreate are "delayed"

@donovanhide
Copy link
Contributor

donovanhide commented Feb 6, 2017

@robcat Your circular payment example isn't circular :-)

The SendMax and Amount would need to have the same currency. For XRP, you'd probably say something like an 999999 Amount and 5 for SendMax, and select the partial payment flag. You don't care how much you receive as long as it is more than the 5 you put in. It's not well documented (at least it wasn't a year or two ago...) but the first and last step in the path are implied and redundant. For the sake of a real example:

[
	{
		"TransactionType": "Payment",
		"Flags": 458752,
		"Account": "rrrrrrrrrrrrrrrrrrrrrhoLvTp",
		"Sequence": 0,
		"Fee": "0",
		"hash": "0000000000000000000000000000000000000000000000000000000000000000",
		"Destination": "rrrrrrrrrrrrrrrrrrrrrhoLvTp",
		"Amount": {
			"value": "26.10927",
			"currency": "GCB",
			"issuer": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n"
		},
		"SendMax": "360484",
		"Paths": [
			[
				{
					"currency": "GCB",
					"issuer": "rNdwi8ain5ibXNB9A7H3zzKtSxgVzAqqAe",
					"type": 48,
					"type_hex": "0000000000000030"
				},
				{
					"account": "rNdwi8ain5ibXNB9A7H3zzKtSxgVzAqqAe",
					"type": 1,
					"type_hex": "0000000000000001"
				},
				{
					"currency": "BTC",
					"issuer": "rHSFu2BEV15GY3QmDeFznRXqgXkMhihccC",
					"type": 48,
					"type_hex": "0000000000000030"
				},
				{
					"account": "rHSFu2BEV15GY3QmDeFznRXqgXkMhihccC",
					"type": 1,
					"type_hex": "0000000000000001"
				},
				{
					"currency": "GCB",
					"issuer": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n",
					"type": 48,
					"type_hex": "0000000000000030"
				},
				{
					"account": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n",
					"type": 1,
					"type_hex": "0000000000000001"
				}
			]
		]
	},
	{
		"TransactionType": "OfferCreate",
		"Flags": 655360,
		"Account": "rrrrrrrrrrrrrrrrrrrrrhoLvTp",
		"Sequence": 0,
		"Fee": "0",
		"hash": "0000000000000000000000000000000000000000000000000000000000000000",
		"TakerPays": "402027",
		"TakerGets": {
			"value": "26.373",
			"currency": "GCB",
			"issuer": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n"
		}
	}
]

The goal is to remove the need for the OfferCreate step to complete the XRP=>XRP circuit.

The point about trustlines I was making earlier is that it is inconsistent that OfferCreates do not require a pre-existing TrustLine limit set, whereas Payments to the same account do. It makes sense for incoming payments from another account to require this be set, and it also makes sense if you want to provide rippling liquidity. But for the conversion scenario, it is just annoying :-)

@tuloski
Copy link
Author

tuloski commented Feb 6, 2017

@donovanhide I agree on the Trust line for payments, but it is not a big deal, you make a script to set all the lines and you're done forever. The annoying part is that you need two transactions. And that has two drawbacks:
-You need the liquidity of another currency (GCB in your example) or you need to send the second transaction in the next ledger, after you receive the first payment.
-With two transactions it is most likely that one succeed and the other not, that is super annoying because you are left with a currency you didn't want.

I see a very cheap solution to this: remove the error, check the forward-backward, if it fails just fail the transaction, otherwise succeed...so it is up to the designer of the path to find a "good" path without internal loops, if that was the only problem to circular payments.

@tuloski
Copy link
Author

tuloski commented Feb 7, 2017

@donovanhide how long did it take to generate the rogue account? :) how many wallet were created before?

JoelKatz added a commit to JoelKatz/rippled that referenced this issue Apr 17, 2017
e52d6ad bump version to 5.2.1
25ea289 Reset DBIter::saved_key_ with proper user key anywhere before pass to DBIter::FindNextUserEntry
4dd267d Make compaction_pri settable through option string
32be357 Update HISTORY.md for the Direct I/O fixes.
3eb6956 add use_direct_io() to ReadaheadRandomAccessFile
18ba3e2 truncate patch
1c91104 posix writablefile truncate
e47027d alignment is on in ReadaheadRandomAccessFile::Read()
e5fd8bd Update version number and HISTORY.md
ac2a77a Announce the experimetnal two-level index feature in HISTORY.md
c4a37dc Print the missed last layer in cfstats
a12818a Blog post for 5.1.2 release
3b4ac80 Clarify ldb column family argument
d70ce7e Move db_bench flags out of unnamed namespace
186c7ee Remove incorrect statistics warning
53bb015 [rocksdb][PR] compaction_style and compaction_pri should output their value as a st…
69d5262 Two-level Indexes
0a4cdde Windows thread
1aaa898 Adding GetApproximateMemTableStats method
9fc23c5 Use gcc-4.9-glibc-2.20-fb python in precommit_checker
b797e42 Dump compression dictionary meta-block
036d668 Fix wrong result in data race case related to Get()
574b543 Rename merger.h -> merging_iterator.h
add8b50 Move ThreadLocal implementation into .cc
71d2496 Fix arc setting for Facebook internal tools
f289d9f Fix OSX build break after the fallocate change
4a3e7d3 Change the default of delayed slowdown value to 16MB/s
0513e21 RangeSync() should work with ROCKSDB_FALLOCATE_PRESENT not set
8b369ae Cleaner default options using C++11 in-class init
ec79a7b Dedup code in option.cc and db_options.cc
b96372d improving the C wrapper
04c4ec4 Change corruption_test to use 4 bits.
2d75cd4 NewLRUCache() to pick number of shard bits based on capacity if not given
f25f1ec Add test DBTest2.GetRaceFlush which can expose a data race bug
37d4a79 Deserialize custom Statistics object in db_bench
3b35134 Avoid cache lookups for range deletion meta-block
94a0c32 Fix LRU Ref() for handles with external references only
17c1180 Generalize Env registration framework
07dddd5 EnvPosixTestWithParam should wait for all threads to finish
5dad9d6 Avoid logs_ operation out of DB mutex
a7b1391 Fix CompactFiles() bug when used with CompactionFilter using SuperVersion
616a146 Fix DeleteRange including sentinels in output files
c918c4b Update USERS.md add user Pika
03ca2ac Remove function from DBImpl that are not used anywhere
b0029bc Test merge op covered by range deletion in memtable
d438e1e Test range deletion block outlives table reader
fba726e Version librocksdb.so
9da4d54 Range deletions unsupported in tailing iterator
f2b4939 fixed typo
973f1b7 memtable: delete merge value for range deleteion
aebfd17 fix non-portable behavior in encoder
753ff84 Fix get approx size
d7ea44f Fixup a couple of builds errors on Linux.
537da37 c: allow set savepoint to writebatch
af6ec4d fix batchresult handle leak
e29bb93 Zlib 1.2.8 is no longer available, switched to 1.2.10
5ac9731 Fix std::out_of_range when DBOptions::keep_log_file_num is zero
4e35ffd cmake: check -momit-leaf-frame-pointer before using it
3c0852d Make ingest external file backward compatible
0e8dfd6 Fix OptimizeForPointLookup()
e840213 Change DB::GetApproximateSizes for more flexibility needed for MyRocks
9239103 Flush job should release reference current version if sync log failed
da54d36 Disable IngestExternalFile in ReadOnly mode
5cf176c Fix for 2PC causing WAL to grow too large
4a73bb0 Split travis jobs
c70d3c7 Enable DBTest.GroupCommit as it runs in a reasonlable time now.
602c13a Remove fadvise with direct IO read
f9d18e2 Fix DeleteRange file boundary correctness issue with max_compaction_bytes
3ce091f Add KEEP_DB env var option
77b4806 Fix 2PC with concurrent memtable insert
e8a0960 util/thread_local.h: silence a clang-build warning
324a0f9 Follow up for DirectIO refactor
bc5d7b7 travis: For linux, do all tests under gcc
3e6899d change UseDirectIO() to use_direct_io()
d4e07a8 fix warning of unused direct io helper functions
dc2584e direct reads refactor
d18dd2c Abort compactions more reliably when closing DB
62384eb Guarding extra fallocate call with TRAVIS because its not working pro…
9f24629 Performance: Iterate vector by reference
fe395fb Allow incrementing refcount on cache handles
2172b66 Fix build on FreeBSD
3c233ca Fix Windows environment issues
7631734 Fix the error in ColumnFamiliesTest
7a02ad0 Update travis to ubuntu trusty
60c509f Fix valgrind failure in test CurrentFileModifiedWhileCheckpointing2PC
d0ba8ec Revert "PinnableSlice"
54d94e9 PinnableSlice
e04480f Fix MS warnings. Use ROCKSDB_Prsz for size_t.
c081f72 5.0.1 release blog post
ac73d75 Add GetSupportedCompressions() convenience function
b104b87 Maintain position in range deletions map
640d724 Update db_bench and sst_dump to test with block cache mid-point inser?
653ac1f C API: support total_order_mode
85ac1a3 Fix rocksdb::Status::getState
76711b6 Make ExternalSSTFileTest::CompactionDeadlock more deterministic
c963460 Fix tests under GCC_481
33c86d6 Fix backupable db test
e425ec1 utilities/backupable: backup should limit the copy size of wal.
0712d54 Delegate Cleanables
d58ef52 Allow SstFileWriter to Fadvise the file away from page cache
17a4b75 Always fsync the file after file copying
2fb70dc examples: Add options_file_example to target all
a738af8 db/pinned_iterators_manager.h: bugfix
906523d Add description to the 2PC checkpooint bug in HISTORY.md
438f22b Fix bug of Checkpoint loses recent transactions with 2PC
335981d Fix the directory path for RocksDB repo
548b628 Enable conditionally using adaptive mutexes
4e07b08 include/rocksdb/utilities/env_librados: fix typo
ab48c16 Print cache options to info log
972f96b direct io write support
989e644 Remove sst_file_manager option from LITE
1beef65 Fix c_test
3d69282 persistent_cache: fix two timer
046099c The array is malloced by backtrace_symbols(), and must be freed
6ff2c8d Remove gflags as travis build dependency
3cd9ed1 Show sandcastle URL in phabricator
50e305d Collapse range deletions
5d1457d Dump persistent cache options
7bd725e db_bench: introduce --benchmark_read_rate_limit
2966918 Update Netflix section of USERS.md
342370f Simplify MemTable::Update
1a136c1 Expose file size
fbff462 Reduce compaction iterator status checks
bd6cf7b WritableFileWriter: default buffer size equal min(64k,options.writabl?
fc0c6fd "make format" format diff since last commit from master
816c1e3 gcc-7 requires include <functional> for std::function
c270735 Iterator should be in corrupted status if merge operator return false
a8bf4d6 Make format shows wrong curl command for retrieving clang-format-diff.py
8f5d24a C API: support get usage and pinned_usage for cache
0ab6fc1 Gcc-7 buffer size insufficient
b7239bf Gcc 7 fallthrough
477b6ea std::remove_if requires <algorithm>
83f9a6f Fail BackupEngine::Open upon meta-file read error
a79eae4 Add pcache documentation images
d71e728 Print user collected properties in sst_dump
7004a6f Add missing copyright header
3cdfaec Fixes for MSVC compilation
e097222 util/logging.cc: buffer of insufficient size (gcc-7 -Werror=format-length)
cfc34d7 Missing break in case in DBTestBase::CurrentOptions
bfbcec2 Gcc 7 error expansion to defined
6653e32 build: make it easier to pass PORTABLE
67adc93 intentional fallthough (prevents gcc-7/clang-4 error)
1a146f8 break Flush wait for dropped CF
c3e5ee7 util/histogram.cc: HistogramStat::toString buffer insufficient
5334d8b table/block_based_table_builder.cc: intentional fallthrough - comment to match gcc pattern
36d42e6 Disable test to unblock travis build
b57dd92 C API: support writebatch delete range
2ba59b5 Disallow ingesting files into dropped CFs
1f6f7e3 cast to signed char in ldb_cmd_test for ppc64le
243975d More accurate error status for BackupEngine::Open
f0c509e Return finer-granularity status from Env::GetChildren*
dc64f46 Add db_bench option for stderr logging
2cabdb8 Increase buffer size
4a17b47 Remove unnecessary header include
8c2b921 Fixed a crash in debug build in flush_job.cc
20ce081 Fix issue where IngestExternalFile insert blocks in block cache with g_seqno=0
5241e0d fix db_bench argument type
c04f6a0 Specify shell in makefile
45c7ce1 CompactRangeOptions C API
2c2ba68 db_stress support for range deletions
b821984 DeleteRange read path end-to-end tests
2f4fc53 Compaction::IsTrivialMove relaxing
1dce75b Update USERS.md
304b3c7 Update USERS.md
fa50fff Option to expand range tombstones in db_bench
c26a4d8 Fix compile error in trasaction_lock_mgr.cc
ed8fbdb Add EventListener::OnExternalFileIngested() event
2005c88 Implement non-exclusive locks
0b0f235 Mention IngestExternalFile changes in HISTORY.md
23db48e Update HISTORY.md for 5.0 branch
beb36d9 Fixed CompactionFilter::Decision::kRemoveAndSkipUntil
67f37cf Allow user to specify a CF for SST files generated by SstFileWriter
9053fe2 Made delete_obsolete_files_period_micros option dynamic
edde954 fix clang build
56281f3 Add memtable_insert_with_hint_prefix_size option to db_bench
4a21b14 Cache heap::downheap() root comparison (optimize heap cmp call)
e39d080 Fix travis (compile for clang < 3.9)
3f407b0 Kill flashcache code in RocksDB
b77007d Bug: paralle_group status updated in WriteThread::CompleteParallelWorker
247d097 Support for range skips in compaction filter
96fcefb c api: expose option for dynamic level size target
00197cf Add C API to set base_backgroud_compactions
5b219ec deleterange end-to-end test improvements for lite/robustness
aad1191 pass rocksdb oncall to mysql_mtr_filter otherwise tasks get created w…
e333528 DeleteRange write path end-to-end tests
7784980 Fix mis-reporting of compaction read bytes to the base level
3c6b49e Fix implicit conversion between int64_t to int
b3b8756 Remove unused assignment in db/db_iter.cc
4f6e89b Fix range deletion covering key in same SST file
a2bf265 Avoid intentional overflow in GetL0ThresholdSpeedupCompaction
52fd1ff disable UBSAN for functions with intentional -ve shift / overflow
1886c43 Fix CompactionJob::Install division by zero
63c30de fix options_test ubsan
13e66a8 Fix compaction_job.cc division by zero
01eabf7 Fix double-counted deletion stat
7ffb10f DeleteRange compaction statistics
236d4c6 Less linear search in DBIter::Seek() when keys are overwritten a lot
cd7c414 Improve Write Stalling System
dfb6fe6 Unified InlineSkipList::Insert algorithm with hinting
3068870 Making persistent cache more resilient to filesystem failures
734e4ac Eliminate redundant cache lookup with range deletion
182b940 Add WriteOptions.no_slowdown
4118e13 Persistent Cache: Expose stats to user via public API
f2a8f92 rocks_lua_compaction_filter: add unused attribute to a variable
4444256 Remove use of deprecated LZ4 function
548d7fb Fix fd leak when using direct IOs
fd43ee0 Range deletion microoptimizations
23a18ca Reword support a little bit to more clear and concise
481856a Update support to separate code issues with general questions
a0deec9 Fix deadlock when calling getMergedHistogram
fe349db Remove Arena in RangeDelAggregator
e63350e Use more efficient hash map for deadlock detection
a13bde3 Skip ldb test in Travis
73843aa Direct I/O Reads Handle the last sector correctly.
9d60151 Implement PositionedAppend for PosixWritableFile
3f62215 Lazily initialize RangeDelAggregator's map and pinning manager
41e77b8 cmake: s/STEQUAL/STREQUAL/
c1038d2 Release RocksDB 5.0
635a7bd refactor TableCache Get/NewIterator for single exit points
f39452e Fix heap use after free ASAN/Valgrind
a4eb738 Allow plain table to store index on file with bloom filter disabled
36e4762 Remove Ticker::SEQUENCE_NUMBER
86eb2b9 Fix src.mk
0765bab Remove LATEST_BACKUP file
647eafd Introduce Lua Extension: RocksLuaCompactionFilter
760ef68 fix deleterange asan issue
327085b fix valgrind
715591b Ask travis to use JDK 7
972e3ff Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
420bdb4 option_change_migration_test: force full compaction when needed
1543d5d Report memory usage by memtable insert hints map.
018bb2e DeleteRange support for db_bench
dc51bd7 CMakeLists.txt: FreeBSD has jemalloc as default malloc
48e8bae Decouple data iterator and range deletion iterator in TableCache
4b0aa3c Fix failed compaction_filter_example and add it into make all
53b693f ldb support for range delete
661e4c9 DeleteRange unsupported in non-block-based tables
489d142 DeleteRange interface
eba99c2 Fix min_write_buffer_number_to_merge = 0 bug
2ef92fe Remove all instances of relative_url until GitHub pages problem is fixed.
91300d0 Dynamic max_total_wal_size option
ec2f647 Consider subcompaction boundaries when updating file boundaries for range deletion
800e515 Fix CSS issues again
b952c89 Parallize persistent_cache_test and transaction_test
3b192f6 Handle full final subcompaction output file with range deletions
6c57952 Make range deletion inclusive-exclusive
425210c CSS issues are arising on the Github Pages side. Temp fix.
1ea79a7 Optimize sequential insert into memtable - Part 1: Interface
df5eeb8 Optimize sequential insert into memtable - Part 2: Implementation
5ed6508 Fix SstFileWriter destructor
adb665e Allowed delayed_write_rate option to be dynamically set.
307a4e8 sst_dump support for range deletion
361010d Exporting compaction stats in the form of a map
672300f Use relative Urls for stylesheets
b39b2ee do not call get() in recovery mode
1ca5f6d Fix 2PC Recovery SeqId Miscount
e095d0c Rocksdb contruns to new Sandcastle API
14c0380 Convenience option to parse an internal key on command line
c90fef8 fix open failure with empty wal
4e20c5d Store internal keys in TombstoneMap
a9fb346 Fix RocksDB Lite build failure in c_test.cc
d133b08 Use correct sequence number when creating memtable
144cdb8 16384 as e.g .value for compression_max_dict_bytes
9bd191d Fix deadlock between (WriterThread/Compaction/IngestExternalFile)
a9fae0a CSS problems again :(
193221e Fix Forward Iterator Seek()/SeekToFirst()
e48f3f8 remove tabs and duplicate #include in c api
85bd8f5 Minor fix to GFLAGS usage in persistent cache
a787527 c: support seek_for_prev
0f17f92 Make the header links a bit more flexible
cf19f55 single quotes in feed
2dc019e Fix header links
f1aedda More Jekyll 3.3 fixes
c54cdc3 More Jekyll 3.3 updates
2bcaf82 Update product and feature template for Jekyll 3.3
24bceb0 Java API - Implement GetFromBatch and GetFromBatchAndDB in WBWI
815f54a Insert range deletion meta-block into block cache
9e7cf34 DeleteRange user iterator support
5c5d01a Fix wrong comment (Maximum supported block size)
f998c97 DeleteRange Get support
879f366 Add C api for RateLimiter
557034f Remove all instances of baseurl
437942e Add avoid_flush_during_shutdown DB option
2b16d66 Change max_bytes_for_level_multiplier to double
16fb044 expose IngestExternalFile to c abi
ce22ea9 Fix casts for MSVC
196af03 Introduce FAIL_ON_WARNINGS CMake variable (default ON)
40a2e40 DeleteRange flush support
d5555d9 Fix MSVC compile error in 32 bit compilation
da61f34 Print compression and Fast CRC support info as Header level
f9eb567 db_bench: --dump_malloc_stats takes no effect
6a4faee fix freebsd build include path err and so & jar file name
c90c48d Show More DB Stats in info logs
1b295ac DBTest.GetThreadStatus: Wait for test results for longer
25f5742 Update documentation to point at gcc 4.8
b50a81a Add a test for tailing_iterator
04751d5 L0 compression should follow options.compression_per_level if not empty
2946cad Improve RangeDelAggregator documentation
0a9fd05 Update Vagrant file (test internal phabricator workflow)
fcd1e0b Make rocksdb work with internal repo
0aab5e5 FreeBSD: malloc_usable_size is in <malloc_np.h> (#1428)
9c0bb7f cmake: drop "-march=native" from CXX_FLAGS (#1429)
eeb27e1 Add handy option to turn on direct I/O in db_bench (#1424)
c6168d1 removed some declarations from c.h which resulted in undefined symbols (#1407)
bc429de revert fractional cascading in farward iterator
b9bc7a2 Use skiplist rep for range tombstone memtable
60a2bbb Makefile: generate util/build_version.cc from .in file (#1384)
9ee8406 Disable DBTest.RepeatedWritesToSameKey (#1420)
f41df30 OptionChangeMigration() to support FIFO compaction
2e8004e Changing the legocastle run to use valgrind_test instead of _check
9de2f75 revert Prev() in MergingIterator to use previous code in non-prefix-seek mode
2449518 DBSSTTest.RateLimitedDelete: not to use real clock
1168cb8 Fix a bug that may cause a deleted row to appear again
99c052a Fix integer overflow in GetL0ThresholdSpeedupCompaction (#1378)
f83cd64 Fix a bug that mistakenly disable regression_test.sh to update commit (#1415)
0e926b8 Passing DISABLE_JEMALLOC=1 to valgrind_check if run locally
4dfaa66 Make IsDeadlockDetect() virtual member of Transaction
59a7c03 Change ioptions to store user_comparator, fix bug
869ae5d Support IngestExternalFile (remove AddFile restrictions)
1d9dbef Restrict running condition of UniversalCompactionTrivialMoveTest2
4edd39f Implement deadlock detection
48fd619 Minor fixes to RocksJava Native Library initialization (#1287)
48e4e84 Disable auto compactions in memory_test and re-enable the test (#1408)
fb2e412 column_family_test: disable some tests in LITE
5af651d fix data race in compact_files_test
a0ba0aa Fix uninitialized variable gcc error for MyRocks
b88f8e8 Support SST files with Global sequence numbers [reland]
08616b4 [db_bench] add filldeterministic (Universal+level compaction)
52c9808 not split file in compaciton on level 0
5e0d6b4 fix db_stress assertion failure
ab53998 Bump RocksDB version to 4.13 (#1405)
b4d0712 SamePrefixTest.InDomainTest to clear the test directory before testing
aa09d03 Avoid calling GetDBOptions() inside GetFromBatchAndDB()
6fbe96b Compaction Support for Range Deletion
257de78 Remove "-Xcheck:jni" from Java tests (#1402)
d88dff4 add seeforprev in history
5027dd1 Fix a minor bug in the ldb tool that was not selecting the specified (#1399)
fea6fdd Fix @see in two Java functions (#1396)
b1031d6 Remove function local statics that interfere with memory pooling (#1392)
f470540 Handle WAL deletion when using avoid_flush_during_recovery
e29d3b6 Make max_background_compactions and base_background_compactions dynamic changeable
21e8dac fix assertion failure in Prev()
b9311aa Implement WinRandomRW file and improve code reuse (#1388)
a249a0b check_format_compatible.sh to use some branch which allows to run with GCC 4.8 (#1393)
040328a Remove an assertion for single-delete in MergeHelper::MergeUntil
8cbe3e1 Relax the acceptable bias RateLimiterTest::Rate test be 25%
f26a139 Log successful AddFile
5691a1d Fix compaction conflict with running compaction
017de66 fixup commit
1b7af5f Redo handling of recycled logs in full purge
27bfe32 Editorial change to README.md
89cc404 A bit of doc restructuring
9e7fda8 Fix arcanist
2e4b5ca Add missing RateLimiter class to the Windows build (#1382)
ce4963f [doc] Document that Visual Studio 2015+ is now required for Windows builds (#1389)
e489270 Fix scoped arena iterator (#1387)
f8d8cf5 Fix log_write_bench -bytes_per_sync option. (#1375)
02b3e39 Make txn->GetState() const
447f171 new Prev() prefix support using SeekForPrev()
991b585 More block cache tickers
d6ae6de Add Statistics::getAndResetTickerCount().
aea3ce4 Avoid string CONCAT which is not supported in cmake 2.6 (#1383)
2ad68b9 Support running consistency checks in release mode
67501cf Fix -ve std::string::resize
04b02dd Testing asset links after config change
8c55bb8 Make Lock Info test multiple column families
d062328 Revert "Support SST files with Global sequence numbers"
5cd2883 [RocksJava] Adjusted RateLimiter to 3.10.0 (#1368)
37737c3 Expose Transaction State Publicly
2c1f952 Add facility to write only a portion of WriteBatch to WAL
043cb62 Fix record_size in log_write_bench, swap args to std::string::assign. (#1373)
4985f60 env_mirror: fix a few leaks (#1363)
5aded67 update of c.h (#1371)
912aec1 "Recent Posts" -> "All Posts"
7cbb298 Make sure that when contribtuing we call out creating appropriate directories
a06ad47 Add top level doc information to CONTRIBUTING.md
3fdd5b9 A little more generic CONTRIBUTING.md
ed4fc31 Add link to CONTRIBUTING.md to main docs README.md
e4922e1 Forgot to truncate one blog post
6d8cd7e Add CONTRIBUTING.md for rocksdb.org contribution guidance
bd55e5a Fix some formatting of compaction blog post
0f60358 CRLF -> LF mod (including removing trailing whitespace for those files)
b90e29c Truncate posts on the main /blog/ page
0d7acad Add author fields to blog posts
01be441 Add GitHub link to the landing page header
9d6c961 Fix Mac build
ab01da5 Support SST files with Global sequence numbers
d346ba2 Minor fixes around Windows 64 Java Artifacts (#1366)
e91b4d0 Add factory method for creating persistent cache that is accessible from public
be1f109 Expose transaction id, lock state information and transaction wait information
6009c47 Store range tombstones in memtable
3c21c64 Use size hint for HashMap in multiGet. Similar to https://github.com/facebook/rocksdb/pull/1344 (#1367)
13f7a01 Fixing JNI release build for gcc (#975)
7260662 Add Java API for SstFileWriter
2638824 delete unused variable for PrevInterval()
87dfc1d Fix conflict between AddFile() and CompactRange()
eb44ed6 Update 2016-09-28-rocksdb-4-11-2-released.markdown
e443761 Update 2016-09-28-rocksdb-4-11-2-released.markdown
501f051 Update 2016-09-28-rocksdb-4-11-2-released.markdown
dec9009 Update 2016-09-28-rocksdb-4-11-2-released.markdown
4ed69dd Create 2016-09-28-rocksdb-4-11-2-released.markdown
21f4bb5 cmake support for linux and osx (#1358)
4defe30 fix typo in comments (#1360)
f517d9d Add SeekForPrev() to Iterator
eb3894c Recompute compaction score on SetOptions (#1346)
5c64fb6 Fix AddFile() conflict with compaction output [WaitForAddFile()]
9e9f5a0 Fix CompactFilesTest.ObsoleteFiles timeout (#1353)
c2a62a4 not cut compaction output when compact to level 0
9ed928e Split DBOptions into ImmutableDBOptions and MutableDBOptions
4bc8c88 Recover same sequence id from WAL (#1350)
0a1bd9c add cfh deletion started listener
da5a9a6 Fix mac build
d45eb6c Fix typo (#1349)
abc0ae4 Add AddFile() InternalStats for Total files/L0 files/total keys ingested
7152563 forbid merge during recovery
5735b3d Fix compiling under -Werror=missing-field-initializers
654ed9a loose the assertion condition of rate_limiter_test
e4d3f5d Fix DBImpl::GetWalPreallocateBlockSize Mac build error
7afbb74 solve the problem of table_factory_to_write_=nullptr (#1342)
d78a440 DBImpl::GetWalPreallocateBlockSize() should return size_t
42ac9c5 Retry getting arcanist token on failure
b666f85 Consider more factors when determining preallocation size of WAL files
4c3f449 Add TableBuilderOptions::level and relevant changes (#1335)
3edb946 Avoid hard-coded sleep in EnvPosixTestWithParam.TwoPools
0a88f38 Remove ColumnFamilyData::options()
41a9070 Fix java makefile dependencies
8d9bf5c Fix DBOptionsTest.GetLatestOptions
40cfa3e Fix DBWALTest.RecoveryWithLogDataForSomeCFs with mac
06b4785 Fix recovery for WALs without data for all CFs
d7242ff Fix GetSortedWalFiles when log recycling enabled
17f76fc DB::GetOptions() reflect dynamic changed options
215d128 Fix typo (#903)
a958c26 Rename jvalue to jval in rocksjni
0a165bd Have Facebook link point to RocksDB on FB
3639f32 Fix bug in UnScSigned-off-by: xh931076284 <931076284@qq.com> (#1336)
8e061f9 Refactor GetMutableOptionsFromStrings
81747f1 Refactor MutableCFOptions
ba65c81 Support POSIX RandomRWFile
1d980a8 Create CNAME
2adab1d Add API links to the header bar
a182b29 Preserve blog comments in markdown
f54de92 Adding Dgraph to list of Users (#1291)
9e4aa79 Summary: (#1313)
a10e8a0 Fix C api memtable rep bugs. (#1328)
eb1d4d5 Release RocksDB 4.12
22d88e2 Allow an offset as well as a length to be specified for byte[] operations in RocksJava JNI (#1264)
b06b191 add C api for set wal_recovery_mode (#1327)
1cca091 Temporarily revert Prev() prefix support
de28a25 Update HISTORY.md for thread-local stats
0fcb6db Remove extraneous function prototypes from c.h (#1326)
52ee07b Move AddFile() tests to external_sst_file_test.cc
66a91e2 Add NoSpace subcode to IOError (#1320)
67036c0 Fix Flaky ColumnFamilyTest.FlushCloseWALFiles
0e2da49 fix typo in option.h's comment (#1321)
6d61358 Add real Google Analytics ID
2d9d36e Have "Edit on GitHub" point to master instead of gh-pages
9377518 Update landing page content
1ec75ee Add redirects from old blog posts link to new format
607628d Support ZSTD with finalized format
ce1be2c Fix build error on Windows (AppVeyor) (#1315)
f7669b4 Fix Windows Build
22696b0 Fix uninitlized CompactionJob::SubcompactionState::current_output_file_size
c1865e0 Trigger more tests per diff
a88677d Remove ImmutableCFOptions from public API
80c7559 Fix data race in AddFile() with multiple files + custom comparator bug
5051755 Fix db_bench memory use after free (detected by clang_analyze)
4fd08f4 Ensure Correct Behavior of StatsLevel kExceptDetailedTimers and kExceptTimeForMutex  (#1308)
e14fbaa Add FAQ based on the front page of the current rocksdb.org
3c22624 Migrate the RocksDB Worpdress blog over to Jekyll
ee0e220 Transfer the current Getting Started contents to GitHub Pages
5a0e9a4 Initial Landing Page
9447a85 Remove the `doc` directory
3214905 Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes
4590b53 add stats to Cache::LookUp()
85bb308 Expose Utility function StringToMap() (#1306)
8ce1b84 Fix Travis on Mac
380e651 Fix Mac build failure (#1309)
1613fa9 Thread-specific histogram statistics
6a14d55 add prefix_seek_mode to db_iter_test
de47e2b Fix ClockCache memory leak
f099af4 Fix travis
db74b1a fix bug in merge_iterator when data race happens
b18f9c9 add nullptr check to internal_prefix_transform
4e395e8 Update docs README.md
2482d5f support Prev() in prefix seek mode
7541c7a Fix cache_test valgrind_check failure
c8513cd Update the download location of Snappy (#1304)
b49b92c Introduce Read amplification bitmap (read amp statistics)
c700484 store prefix_extractor_name in table
4ad928e add comment to SimCache to estimate actual capacity
e9b2af8 Expose ThreadPool under include/rocksdb/threadpool.h
23a0570 Document memtable flush behavior in CancelAllBackgroundWork()
dade61a Mitigate regression bug of options.max_successive_merges hit during DB Recovery
cce702a [db_bench] Support single benchmark arguments (Repeat for X times, Warm up for X times), Support CombinedStats (AVG / MEDIAN)
3586901 cat tests logs sorted by exit code
b2ce595 Persist data during user initiated shutdown
4b3438d Fix parallel valgrind (valgrind_check)
a081f79 Relax consistency for thread-local ticker stats
b10d65c Update and slightly clarify instructions in build_detect_platform (#1301)
f85f99b Fix the Windows build of RocksDB Java. Similar to https://github.com/facebook/rocksdb/issues/1220 (#1284)
7b81095 Fix a crash when compaction fails to open a file
7c95868 Thread-specific ticker statistics
ea9e075 Add initial GitHub pages infra for RocksDB documentation move and update. (#1294)
2a9c971 [Flaky Test] Disable DBPropertiesTest.GetProperty
d76ddf3 Disable ClockCache db_crashtest
cec2c64 fix data race in NewIndexIterator() in block_based_table_reader.cc
badbff6 Not insert into block cache if cache is full and not holding handle
4a16c32 Option to cache index/filter blocks with priority
99c4af7 Make ClockCache available with TSAN build
f57bc1d Fix lambda expression for clang/windows
5440675 Fix lambda capture expression for windows
6584cec Fold function for thread-local data
817eeb2 Add singleDelete to RocksJava (#1275)
ffdf6ee Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
ecf9003 Fix bug in printing values for block-based table
72f8cc7 LRU cache mid-point insertion
6a17b07 Add TablePropertiesCollector support in SstFileWriter
78837f5 TableBuilder / TableReader support for range deletion
4cc37f5 Introduce ClockCache
ff17a2a Adding TBB as dependency.
49d88be c abi: allow compaction filter ignore snapshot (#1268)
0b63f51 fixes 1215: execute_process(COMMAND mkdir ${DIR}) fails to create a directory with cmake on Windows (#1219)
3981345 Small nits (#1280)
2a2ebb6 Move LRUCache structs to lru_cache.h header
2fc2fd9 Single Delete Mismatch and Fallthrough statistics
3771e37 WriteBatch support for range deletion
236756f Make SyncPoint return immediately when disabled
64a0082 Fix DBSSTest::AddExternalSstFileSkipSnapshot valgrind fail
dd7a748 Fix java build
4fe12ba Make db_bench less space for --stats_per_interval
6525ce4 Compaction stats printing: "batch" => "commit group"
a117891 Fixed typo (#1279)
b248e98 Fix a destruction order issue in ThreadStatusUpdater
deda159 Added min/max/avg data block size output to sst_dump
e408e98 add Name() to Cache
a297643 Fix valgrind memory leak
d11c09d Eliminate memcpy from ForwardIterator
d367555 Added further Java API options for controlling concurrent writes
ebdfe34 Exposed further Java API options for controlling compaction
d1be594 Improve documentation of SliceTransform.
6056d63 Improve comment and bug fix for GetOptionsFromMap functions in convenience.h
76a67cf support stackableDB as the baseDB of transactionDB
67c1ae8 Travis build break fix
b693ba6 Minor PinnedIteratorsManager Refactoring
db3dfb1 Fixes for arcanist config (#1271)
87c91bd Persistent Read Cache (8) Benchmark tooling
2914de6 add sim_cache stats to Statistics
8b79422 [Proof-Of-Concept] RocksDB Blob Storage with a blob log file.
4beffe0 Fix test data race in two FaultInjectionTest tests
821bcb0 util/arena.cc: FreeBSD: More portable use of mmap(MAP_ANON) (#1254)
5370f44 Increase RocksDB version
56dd034 read_options.background_purge_on_iterator_cleanup to cover forward iterator and log file closing too.
ccecf3f UniversalCompaction should ignore sorted runs being compacted (when compacting for file num)
1b0069c Remove non-gtest from parallelized tests
638c49f Change HISTORY.md for release 4.11
6b8e9c6 fix vs generator (#1269)
c38b075 Update HISTORY.md
8f399e3 Update HISTORY.md
98d0b78 Added check_snapshot option in the DB's AddFile function (#1261)
9fd68b7 set travis open file descriptor limit
59ddb50 Fix travis build break
f4d9863 Added SetOptions support to RocksJava (#1243)
7882cb9 Make DBOptionsTest::EnableAutoCompactionAndTriggerStall less falky
44f5cc5 Add time series database (resubmitted)
7c4615c A utility function to help users migrate DB after options change
5bb0a7f Update appveyor.yml
86396cc Update appveyor.yml
c1db098 Update appveyor.yml
7da2eaf Update appveyor.yml
34723b4 Cleanup unused variable pending_fsync_.
7cc0dbd cat all logs in sandcastle output
9253767 Correct geHistogramData() -> getHistogramData() (#1257)
f35b16f db_bench add an option of --base_background_compactions
c3a4bea Fix flaky test `ObsoleteFiles`
8234faa Fix failed test
4990c0d Remove deprecated LEVELDB_PLATFORM_POSIX
7323e4c Fix clang on macOS
ee027fc Ignore write stall triggers when auto-compaction is disabled
e4609a7 Fix Windows build issues (#1253)
2306167 Fix clang build failure and refactor unit test
343304e Use StopWatch to do statistic job in db_impl_add_file.cc
cdc4eb6 Add a GetComparator() function to the ColumnFamilyHandle base class so that the user's comparator can be retrieved.
712dd27 Build break fixes
0155c73 Fix parallel tests `make check -j`
c49ea68 Fix to enable running CI jobs locally
726c2f7 Build break fix
d51dc96 Experiments on column-aware encodings
c116b47 Persistent Read Cache (part 6) Block Cache Tier Implementation
64046e5 Write a benchmark to emulate time series data
9ae92f5 More granular steps in the Makefile, can help with running all or single Java tests (and with ASAN build - https://github.com/facebook/rocksdb/wiki/JNI-Debugging) (#1237)
7c01d65 [Fix Java] Remove duplicate cases in LoggerJniCallback::Logv
8796934 Added missing Java ReadOptions settings (#1109)
5e2c796 Make DBTest.CompressionStatsTest more deterministic
557748f Fix db_stress failure (pass merge_operator even if not used)
811ee21 Bugfix to ensure that logging can be achieved from threads that are not known to the JVM (#1106)
afad5bd Simplify thread-local static initialization
6920cde Remove an extra apostrophe
e72ea48 add InDomain regression test
9c8ac14 Avoid duplicate task creation for RocksDB contruns
d4c4542 db_stress shouldn't assert file size 0 if file creation fails
d3bfd33 Testing out parallel sandcastle changes
7efd9c2 Increse timeout in some tests
50b8d29 fixes 1230: Error:string sub-command REGEX, mode REPLACE needs at least 6 arguments total to command (#1231)
5c858dd fix errata in libnuma test (#1244)
e5b5f12 Change options memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size and cover huge page to memtable too
0ce258f Compaction picker to expand output level files for keys cross files' boundary too.
ac0d93b fixes 1217: rocksdbjni javac and javah execute_processes fail on windows (#1218)
1ae4609 Appveyor badge to show master branch
8745f01 [Fix java build] Stop using non standard std::make_unique
e12270d fix previous typo
bbd6a5a ldb restore subcommand
9498069 Run error-filtering script on diff-triggered tests
f8061a2 Fix Statistics TickersNameMap miss match with Tickers enum
16e225f Fix MergeContext::copied_operands_ strings moving
a4955b3 Run sandcastle tests in /dev/shm
ae0ad71 Fix flaky DBSSTTEST::DeleteObsoleteFilesPendingOutputs
b2a8016 Update db_bench_tool.cc (#1239)
c665458 Disable two dynamic options tests under lite build
2a6d0cd Ignore stale logs while restarting DBs
ee8bf2e fixes 1228: rockdbjni loadLibraryFromJarToTemp fails when file is already present (#1232)
f85df12 Re-enable tsan crash white-box test with reduced killing odds
89e4c48 Update README.md to include appveyor badge
b06ca5f ldb load, prefer ifsteam(/dev/stdin) to std::cin (#1207)
4ea0ab3 Revert "Remove bashism from `make check` (#1225)"
12767b3 fixes 1220: rocksjni build fails on Windows due to variable-size array declaration (#1223)
a9d512a Update .gitignore for internal release
d5a51d4 Need to make sure log file synced before flushing memtable of one column family
89f319c Fix unit test which breaks lite build
b506329 Add unit test not on /dev/shm as part of the pre-commit tests
b9a9718 Bump next release version
663afef Add EnvLibrados - RocksDB Env of RADOS (#1222)
32604e6 Fix flush not being commit while writing manifest
9ab38c4 Remove %z Format Specifier and Fix Windows Build of sim_cache.cc (#1224)
08ab1d8 Remove bashism from `make check` (#1225)
f9b14be Re-enable TSAN crash test but only with black box crash test
68f3eb7 Run release build for CLANG and GCC 4.8.1 in pre-commit tests too
e70020e Only cache level 0 indexes and filter when opening table reader
7bedd94 Build break fix
68a8e6b Introduce FullMergeV2 (eliminate memcpy from merge operators)
e70ba4e MemTable::PostProcess() can skip updating num_deletes if the delta is 0
2a282e5 DBTablePropertiesTest.GetPropertiesOfTablesInRange: Fix Flaky
d9cfaa2 Persistent Read Cache (6) Persistent cache tier implentation - File layout
9430333 New Statistics to track Compression/Decompression (#1197)
515b11f fixes #1210: rocksdb/java/CMakeLists.txt lacks cmake_minimum_required (#1214)
876cb8b fixes #1212: rocksdbjni maven build does not escape slashes in groovy script (#1213)
21c55bd DBTest.DynamicLevelCompressionPerLevel: Tune Threshold
4b95253 Refactor cache.cc
c6a8665 Update LANGUAGE-BINDINGS.md
880ee36 ldb backup support
6797e6f Avoid updating memtable allocated bytes if write_buffer_size is not set
dda6c72 Add DestroyColumnFamilyHandle(ColumnFamilyHandle**) to db.h
56222f5 Avoid FileMetaData copy
15b7a4a Fixed output size and removed unneeded loop
6ea41f8 Fix deadlock when trying update options when write stalls
efd013d Miscellaneous performance improvements
e6f68fa Update Makefile to fix dependency
816ae09 fix test failure
e295da1 Fix Log() doc for default level
8e6b38d update DB::AddFile to ingest list of sst files
296545a Fix clang analyzer errors
61dbfbb Add release build to RocksDB per-diff/post-commit tests
907f24d Concurrent memtable inserter to update counters and flush state after all inserts
0f691c4 CLI option & Rename() allow overwrite
7c19007 delete unnessary pointer cast in beginInternalTransaction() function
e1b3ee8 Cleanup auto-roll logger flush-while-rolling test
cd4178a Add a new feature to enforce a sync point only active on a thread
b954847 Fix release build for MyRocks by using debug-only code only in debug builds
a00bf1b Add More Logging to track total_log_size
01f77cb Update USER.md to include more services at Facebook.
eb53c05 Add comment for GetBackupInfo about returned BackupInfos order
32df973 Add options.write_buffer_manager: control total memtable size across DB instances
5aaef91 group multiple batch of flush into one manifest file (one call to LogAndApply)
a45ee83 Fix a bug that accesses invalid address in iterator cleanup function
38fae9e fix typos in HISTORY.md (#1192)
1a11c93 Disable some persistent cache tests on linux/clang
9b5adea Add More Logging to track total_log_size
95d96ee remove LockFile
ff45d1b if read only backup engine can't find meta dirs, return NotFound() instead of IOError()
cb2476a fix rate limiter to avoid starvation
6b71676 Run env_basic_test on Env::Default
9eb0b53 Move env_basic_test cleanup to TearDown
1fe3bf8 Re-enable linux on travis
4369279 Fixed Minor Bug on Windows Build and db_bench_tool.cc (#1189)
95c1924 writable file close before reset
197b832 Update USERS.md
bdb1d19 Fix UBSan build break caused by variable not initialized
b726bf5 FreeBSD does not have std::to_string (#1190)
faa7eb3 Improve regression_test.sh
c4cef07 Update DBTestUniversalCompaction.UniversalCompactionSingleSortedRun to use max_size_amplification_percent = 0
892e9d3 make transaction WriteOptions modifiable
4f2b094 fix simple typos (#1183)
3b7ed67 ColumnFamilyOptions API [CF + RepairDB part 3/3]
56ac686 Detect column family from properties [CF + RepairDB part 2/3]
3fc713e delete 2nd level children for default env
343507a Refactor to use VersionSet [CF + RepairDB part 1/3]
aa432be Workarounds for continuous build implementation
8cd9f04 Test change to verify new commit detection
8a4ee7e Trivial change to test cont. build
af6248d Fix max_bytes_for_level_base comment
0d7b261 add tests to env_basic_test.cc
6576fa0 Fix minor typos and PHP source file name used to trigger the builds
c4e19b7 Add a read option to enable background purge when cleaning up iterators
fa813f7 Update DB::AddFile() to ingest the file to the lowest possible level
d6b79e2 Remove filter_deletes from crash_test
a52e4d7 Framework for enabling continuous RocksDB build and tests
f9bd667 Makefile warning for invalid paths in make_config.mk
88a2776 Update SstFileWriter to use bottommost_compression if avaliable
e87d5df TiKV use-case (#1172)
7b79238 Deprectate filter_deletes
4939fc3 Bulk load mode shouldn't stop ingest
3a2bccc Fixed a crash bug that incorrectly parse deprecated options in options_helper
cf8adc9 Allow arcanist_util to work with both new and old arc versions
30a24f2 Add InternalStats and logging for AddFile()
d26a848 Temporarily disable travis on linux
249e796 Fix Flaky DBCompactionTest.SkipStatsUpdateTest
162c917 Make sandcastle access secure
8366e10 Fix clang build
0babce5 Move away from enum char value -1
812dbfb Optimize BlockIter::Prev() by caching decoded entries
550bf89 Minor bug fix with log name
886af59 Fix examples/Makefile jemalloc error
e3b1e3d Expose save points in Java WriteBatch and WBWI (#1092)
f5177c7 Remove wasteful instrumentation in FullMerge (stacked on D59577)
97fd2a6 Remove dead Jenkins code and support `arc diff --preview` in RocksDB
7c919de Reuse TimedFullMerge instead of FullMerge + instrumentation
9a33a72 Remove the comments saying allow_concurrent_memtable_write and enable_write_thread_adaptive_yield are not stable
81f6b33 Fix tsan error
bc8af90 add option to not flush memtable on open()
8100ec2 Fix libgcc broken lib path
7360db3 Add a check mode to verify compressed block can be decompressed back
2a79af1 Fix Java Break Related to memtable bloom bits to size ratio change
6faddd7 Merge db/slice.cc into util/slice.cc
5009b53 BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists
2d05eae Fix name conflict in delete_shceduler_test and db_sst_test
bde7d10 Fix clang_analyze path in fbcode_config.sh
ca3db54 Fetch branches from github for format compatibility test
20699df memtable_prefix_bloom_bits -> memtable_prefix_bloom_bits_ratio and deprecate memtable_prefix_bloom_probes
e9c1fac Minor fix to precommit-check.py
fcc47fa New features to precommit check script
56887f6 Backup Options
a683d4a URI-based Env selection for db_bench
53a4bd8 duplicate line
3e86869 release 4.9 update version and history
b2973ea Remove options builder
5b197bf Enabled Windows build for volatile tier implementation
281fbdd Temporarily remove PersistentCacheOptions from persistent_cache_tier.h to fix unity build
2ae15b2 Format compatible test should cover forward compatibility up to 4.8.
00a0587 netflix use-case
5091dfc use branch names in format compatibility test
edc764e Use valgrind built with gcc-4.9-glibc-2.20
8ff59b2 Disable PersistentCacheTierTest.VolatileCacheInsertWithEviction test under TSAN temporarily
1ba4522 Fix for GCC 5.4 (#1157)
972c895 Previously WARN level logging became FATAL level logging in the Java API (#1089)
a73b26f Adding test for contiguous WAL detection
098da83 Fix CLANG build break caused by the recent Persistent Cache change
54db29b Use gvfs links in dependencies.sh
d755c62 Persistent Read Cache (5) Volatile cache tier implementation
fda0984 Allow regression_test.sh to specify OPTIONS_FILE.  Add header comments.
0fee896 Fix Windows build
10d46b9 Update tp2 clang path to fix clang build
774a6aa Java API - Rename geHistogramData -> getHistogramData (#1107)
3070ed9 Persistent Read Cache (4) Interface definitions
e42dc91 Update paths for fbcode dependencies
5647fa4 stack_trace,cc: The current Stacktrace code does not compile for FreeBSD (#1153)
0d65ace threadpool.cc: abort() lives in stdlib.h on FreeBSD (#1155)
19dd5a6 env_chroot.cc: FreeBSD likes stdlib.h for realpaht() and friends (#1154)
5aca977 env_basic_test library for testing new Envs [pluggable Env part 3]
1147e5b Adding support for sharing throttler between multiple backup and restores
6e6622a Create env_basic_test [pluggable Env part 2]
e532877 Add statistics field to show total size of index and filter blocks in block cache
a791a2c Java API - Bugfix for native linking of Compaction Filter (#1099)
af0c9ac Env registry for URI-based Env selection [pluggable Env part 1]
02ec815 allow updating block cache capacity from C (#1149)
630b732 fix flaky sim_cache_test
62d5480 Add persistent cache to Windows build system
8429586 Fix race condition in SwitchMemtable
88acd93 Allows db_bench to take an options file
3a276b0 Add a callback for when memtable is moved to immutable (#1137)
8cf0f86 Allow regression test to run db_bench at a remost host
27ad170 Fix Windows build break
936973d Small tweaks to logging to track the number of immutable memtables
21c047a add readahead size option (#1146)
71c7eed Assert boundary checks for SetPerfLevel()
5d85fdb add missing lock
c40c4ca LDBCommand::SelectCommand to use a struct as the parameter
590e261 fix delete file bug when do checkpoint (#1138)
8dfa980 Add statically-linked library for tools/benchmarks
f62fbd2 Handle overflow case of rate limiter's paramters
57461fb In-memory environment read beyond EOF
0e20000 LDBCommand::InitFromCmdLineArgs() to move from template to function wrapper
472c06e Add low and upper bound values for rocksdb::PerfLevel enum
157e063 MutexLock -> ThreadPoolMutexLock in util/threadpool.cc
23d4cf4 include/rocksdb/sst_file_writer.h should not depend on util/mutable_cf_options.h
345fd73 Fix flaky DBTestDynamicLevel.DynamicLevelMaxBytesBase2
8fc75de Minor fix to disable DynamicLevelMaxBytesBase2
9dd50d9 Fix db_bench
5d66025 add simulator Cache as class SimCache/SimLRUCache(with test)
d379d11 Update CMakeLists.txt for added test
21f847e Direct IO fix for Mac
99765ed Clean up the ComputeCompactionScore() API
def2f7b Expose report_bg_io_stats option in the C API. (#1131)
f89caa1 Direct IO capability for RocksDB
8f12145 C API: Expose DeleteFileInRange (#1132)
11f329b db/db_impl: restrict WALRecoveryMode when using recycled log files
2b2a898 db/log_reader: combine kBadRecord{Len,Checksum} for readability
34df1c9 db/log_reader: treat bad record length or checksum as EOF
7947aba db/log_reader: move kBadRecord{Len,Checksum} handling into ReadRecord
847e471 db/log_test: add recycle log test
4e7e41b Disable lite build/testing for persistent read cache
1d725ca Deprecate BlockBasedTableOptions.hash_index_allow_collision=false.
0e77246 backupable_db.cc: lambada to explictly caputre "this" when escaping scope
2073cf3 Eliminate use of 'using namespace std'.  Also remove a number of ADL references to std functions.
26adaad Split WinEnv into separate classes. (#1128)
bb98ca3 Implement GetUniqueId for Mac
1f2dca0 Add MaxOperator to utilities/merge_operators/
f6e404c Added "number of merge operands" to statistics in ssts.
7383b64 Fix formatting of HISTORY.md (#1126)
0e665c3 Disable long running GroupCommitTest (#1125)
3c69f77 Move IO failure test to separate file
533cda9 Add GetStringFromCompressionType to include/rocksdb/convenience.h
c70a933 Fix mutex unlock issue between scheduled compaction and ReleaseCompactionFiles()
05c5c39 Fix build
a6254f2 Long outstanding prepare test
2ead115 Fix TransactionTest.TwoPhaseMultiThreadTest under TSAN
1f0142c Persistent Read Cache (Part 2) Data structure for building persistent read cache index
43afd72 [rocksdb] make more options dynamic
bac3be7 Fix build issue. (#1123)
f6aedb6 Fix Transaction memory leak
a08c8c8 Added PersistentCache abstraction
5c06e08 [ldb] Templatize the Selector
aab91b8 Use generic threadpool for Windows environment (#1120)
a400336 TransactionLogIterator sequence gap fix
fa3536d Store SST file compression algorithm as a TableProperty
40123b3 signed vs unsigned comparison fix
49815e3 [ldb] Export LDBCommandRunner
c1af07c Disable backupable_db_test.cc on Windows
e61ba05 Isolate db env and backup Env in unit tests
560358d Fix data race in GetObsoleteFiles()
5c1c904 ldb option for compression dictionary size
c27061d [rocksdb] 2PC double recovery bug fix
a657ee9 [rocksdb] Recovery path sequence miscount fix
8a66c85 [rocksdb] Two Phase Transaction
1b8a2e8 [rocksdb] Memtable Log Referencing and Prepared Batch Recovery
0460e9d Modification of WriteBatch to support two phase commit
f548da3 Follow symlinks in chroot directory
d86f9b9 Fix lite build
4b31723 Add bottommost_compression option
bfb6b1b Estimate pending compaction bytes more accurately
258459e Properly destroy ChrootEnv in env_test
fca5aa6 Initial script for the new regression test
e1951b6 Add --index_block_restart_interval option in db_bench
730f7e2 Fix win build
a9b3c47 Fix includes for clang on OS X
3f16a83 Introduce chroot Env
269f6b2 Revert "Modification of WriteBatch to support two phase commit"
04dec2a [ldb] Export ldb_cmd*.h
72c73cd Java API - Add missing HEADER_LEVEL logging (#1104)
4d02bfa Add support for PauseBackgroundWork and ContinueBackgroundWork to the Java API (#1087)
8f65fea Have sandcastle run lite_test for every diff
0d590d9 Make max_dict_bytes optional in options string
7ccb8d6 BlockBasedTable::Get() not to use prefix bloom if read_options.total_order_seek = true
e3c6ba3 OptimizeForSmallDb(): revert some options whose defaults were just changed
967476e Fix valgrind (DBIteratorTest.ReadAhead)
9790b94 Add optimize_filters_for_hits option to db_bench
a4ea345 Fixing lite build
24a24f0 Enable configurable readahead for iterators
ff4b3fb Fix Iterator::Prev memory pinning bug
cba752d sst_dump won't print size for unsupported compression type
6e801b0 Eliminate memcpy in Iterator::Prev() by pinning blocks for keys spanning multiple blocks
1b16692 Release RocksDB 4.8.0
b8cf913 Fix #1110, 32-bit build failure on Mac OSX (#1112)
21441c0 Fix calling GetCurrentMutableCFOptions in CompactionJob::ProcessKeyValueCompaction()
4ea6e05 Fix multiple issues with WinMmapFile fo sequential writing (#1108)
f3bb024 Fix clang build
6e56a11 Modification of WriteBatch to support two phase commit
1d2e4ef ldb support new WAL records
a92049e Added EventListener::OnTableFileCreationStarted() callback
e8115ce Revert "Use async file handle for better parallelism (#1049)" (#1105)
6a14f7a Change several option defaults
c6c770a Use prefix_same_as_start to avoid iteration in FindNextUserEntryInternal. (#1102)
992a8f8 Not enable jemalloc status printing if USE_CLANG=1
029022b Fix crash_test
a06faa6 Skip PresetCompressionDict test for lite
e7899c6 Fix build issue. (#1103)
0f428c5 Fix compression dictionary clang osx error
6d4832a Merge pull request #1101 from flyd1005/wip-fix-typo
af70f9a Fix typo in build_tools/fbcode_config.sh
54de13a Fix compression dictionary clang errors
0850bc5 Fix build on machines without jemalloc
4032145 Configurable compression in db_bench
843d2e3 Shared dictionary compression using reference block
ad573b9 Temporarily disable CompactFiles in db_stress in its default setting
1c80dfa Print memory allocation counters
eb73980 Fix BackupableDBTest
ac0e54b CompactedDB should not be used if there is outstanding WAL files
d719b09 Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData)
1995e34 Retrieve file size from proper Env
7c14abf Improve BytewiseComparatorImpl::FindShortestSeparator
f3eb0b5 Make EventListenerTest.CompactionReasonLevel more deterministic
7b78d62 Shouldn't report default column family's compaction stats as DB compaction stats
995353e Fix null-pointer-dereference detected by Infer (https://github.com/facebook/infer)
24110ce Correct Statistics FLUSH_WRITE_BYTES
b71c4e6 Alpine Linux Build (#990)
90ffed1 Update USERS.md with link to LinkedIn blog post (#1088)
99a3bf8 Merge pull request #1068 from daaku/c-purge-old-backups
b54c347 Use async file handle for better parallelism (#1049)
c146c9b rocksdb_create_mem_env to allow C libraries to create mem env (#1066)
6da70c5 expose more options in the c api (#1067)
6f01687 C rocksdb_create_iterators to expose NewIterators (#1069)
644f978 Fix RocksDB Lite build in db_stress
5bd4022 Add comparator, merge operator, property collectors to SST file properties (again)
7a6045a fix typo in HISTORY.md
73a847e Add per-level compression ratio property
ee221d2 Introduce XPRESS compresssion on Windows. (#1081)
874c96a Merge pull request #1083 from flabby/master
6cbffd5 Enable testing CompactFiles in db_stress
b95510d Fix DBTest.RateLimitedDelete flakiness
6356b4d Fix nullptr dereference in adaptive_table
9385fd7 Delete deprecated backup classes in Java
a2466c8 [db_stress] Make subcompaction random in crash_test
c3c389d Fix column label for L0 write sum
ec84bef New legocastle output parsing
725184b Fix db_block_cache_test in lite build
290883d Fix lite build
23089fd write_callback_test: clean test directory before running tests
792762c Split db_test.cc
40b840f Delete deprecated *BackupableDB interface for backups
6affd45 Make more tests run in parallel
47833e0 Merge branch 'master' of github.com:facebook/rocksdb
e5c614e Fixing snapshot 0 assertion
1b1adeb Fixing snapshot 0 assertion
6d436a3 DBTest.HardLimit made more deterministic
994d9bc Make parallel valgrind watch-log more readable
9d35ae6 Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more deterministic
cea8ed9 Fix backupable_db_test test cases that can't run by itself
4b6833a Rename options.compaction_measure_io_stats to options.report_bg_io_stats and include flush too.
3894603 Allow valgrind_check to run in parallel
c9d668c Fix unit tests issues on Windows (#1078)
083cadc Minor fix to Java sandcastle job definition
80b74a6 Include ldb tool in the windows build (#914)
7c14d11 Minor fix to java build job
535af52 BlockBasedTable::PrefixMayMatch() to skip index checking if we can't find a filter block.
09be5ca Minor fix to sandcastle java job definition
1aeca97 Release RocksDB 4.7
dfc3de8 Split Travis unittests Job
c2c8fe4 Add Java job for sandcastle
19ef3de Fix ManualCompactionPartial test flakiness
b345b36 Add a minimum value for the refill bytes per period value
dff4c48 BlockBasedTable::PrefixMayMatch: no need to find data block after full bloom checking
0353b85 Propagate sandcastle run error to UI
b885f33 Parallelize travis jobs
71303e0 Update db_bench_tool.cc (#1073)
63cf15b Fix option settable tests
e208575 using java7 in runtime for hdfs env (#1072)
13e6c8e Relax an assertion in Compaction::ShouldStopBefore
ae21d71 Fixed a bug in RocksDB Statistics where flush is considered as compaction
8e0e22f Fix Windows build by replacing strings.h include
5675d50 Revert travis commit
91f0f1f fix travis
a23c605 Don't run DBOptionsAllFieldsSettable under valgrind
30d72ee PrefixTest.PrefixAndWholeKeyTest should run against a different directory from prefix_test
0e3cc2c Add column family info to TableProperties::ToString()
2448f80 Make sure that if use_mmap_reads is on use_os_buffer is also on
114a1b8 Fix build errors for windows
0522990 Improve sst_dump help message
0930e5e Update comments on include/rocksdb/perf_context.h
3b977bc instructing people to use java7 for hdfs (#1063)
1518b73 Change default number of cache shard bit to be 6 and max_file_opening_threads to be 16.
ada88b6 fix wrong assignment of level0_stop_writes_trigger in spatialdb (#1061)
2391ef7 Embed column family name in SST file
ab4c623 Don't use version in the error message
d02eb8d Fix unused variable warning
9278097 Merge pull request #1056 from facebook/igorcanadi-patch-1
cc87075 No need to limit to 20 files in UpdateAccumulatedStats() if options.max_open_files=-1
8a1a603 Eliminate std::deque initialization while iterating over merge operands
f38540b WriteBatchWithIndex micro optimization
2006540 Merge pull request #1053 from adamretter/benchmark-java-comparator
f2c43a4 Stderr info logger
b55e216 Rocksdb backup can store optional application specific metadata
9b51987 Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes.
2feafa3 Change some RocksDB default options
a558830 Fixed compile warnings in posix_logger.h and coding.h
51c9464 Merge pull request #980 from adamretter/java-arm
925b5d0 Merge pull request #1054 from DCEngines/magic12
63e8f1b Formatted lines to adhere to 80 char limit
994b3bd Add support for UBsan builds to RocksDB
21700a5 to/from hex refactor
2442094 Replace kHeader by WriteBatchInternal::kHeader in few more places kHeader was moved from write_batch.cc to header file because it is being used wherever the number "12" was being used to check for record size
3bdbe89 Merge branch 'magic12' of https://github.com/dcengines/rocksdb into magic12
7871152 In all the places where log records are read, there was a check that record.size() should not be less than 12.
e7c64fb Imporve sst_file_manager comment
99ffb3d Fix perf_context::merge_operator_time_nanos calculation
07bb12d Update internal jemalloc and other versions
ad2fdaa Correct a typo in a comment
be9816b Fix data race issue when sub-compaction is used in CompactionJob
e380253 Merge pull request #1050 from yuslepukhin/support_db_test2
e7cc49c Add support for db_test2 for dev and CI runs
3996770 Add comments to perf_context skip counters
4e85b74 Make WritableFileWrapper not screw up preallocation
ec458dc Merge pull request #1047 from PraveenSinghRao/wal_filter_ex
583157f Avoid overloaded virtual function
b9d4fa8 Options settable tests to use a different special charactor
60e34ba Merge pull request #1044 from PraveenSinghRao/wal_filter_ex
136b8e0 Merge from master
2dcbb3b Addressed review comments
b1fafca Revert "Adding pin_l0_filter_and_index_blocks_in_cache feature."
5f8741a Revert "Fix failing Java unit test."
43bbb56 tools/check_format_compatible.sh to use consistent version when testing backward and forward compatibility
d7ae42b Fix failing Java unit test.
fbbb8a6 Add test for Snapshot 0
e182f03 Add unit tests for RepairDB
7d37186 travis build fixes
fbea4dc Merge pull request #1042 from SherlockNoMad/HistFix
780d2b0 Update format compatible checking tool
4f1c74a merge from master
f8c2189 Publish log numbers for column family to wal_filter, and provide log number in the record callback
4475626 Reset block cache in failing unit test.
522de4f Adding pin_l0_filter_and_index_blocks_in_cache feature.
be22271 Merge pull request #1041 from yuslepukhin/adjust_for_jemalloc
4ecc03c Fix in HistogramWindowingImpl
2ca0994 Latest versions of Jemalloc library do not require je_init()/je_unint()   calls. #ifdef in the source code and make this a default build option.
90aff0c Update --max_write_buffer_number for compaction benchmarks
7222410 Forge current file for checkpoint
33d5686 Merge pull request #1040 from bureau14/master
02e62eb Fixes warnings and ensure correct int behavior on 32-bit platforms.
9cad568 Merge pull request #1039 from bureau14/master
3d29f91 Improve documentation of the allow_os_buffer parameter.
3ff98bd Fix no compression test
b9cc42a Merge pull request #1038 from SherlockNoMad/HistFix
f76b260 Fix FB internal CI build failure
774922c Merge pull request #1026 from SherlockNoMad/Hist
17b879b Merge pull request #1037 from SherlockNoMad/BuildFix
f11b0df Fix AppVeyor build error
6b03f93 Fix the build break on Ubuntu 15.10 when gcc 5.2.1 is used
697fab8 Updates to RocksDB subcompaction benchmarking script
58379bf remove division from histogramwidowing impl
1a2cc27 ColumnFamilyOptions SanitizeOptions is buggy on 32-bit platforms.
e778c34 Merge pull request #1035 from bureau14/master
5bd3da1 Added quasardb to the USERS.md file
b2ae595 Index Reader should not be reused after DB restart
0267655 Update change log for 4.6 release
08304c0 Expose RepairDB as ldb command
fd664a2 Fix Build Error
580fede Aggregate hot Iterator counters in LocalStatistics (DBIter::Next perf regression)
54f6b9e Histogram Concurrency Improvement and Time-Windowing Support
7902528 Add multithreaded transaction test
e8e6cf0 fix: handle_fatal_signal (sig=6) in std::vector<std::string, std::allocator<std::string> >::_M_range_check | c++/4.8.2/bits/stl_vector.h:794 #174
d962023 Cleanup stale manifests outside of full purge
f71fc77 Cache to have an option to fail Cache::Insert() when full
ee8cc35 Merge pull request #938 from alexander-fenster/master
765597f Update compaction score right after CompactFiles forms a compaction
f0161c3 formatting fix
aa3f02d Improve comment in compaction.h and compaction_picker.h
2200295 optimistic transactions support for reinitialization
badd6b7 Ignore db_test2
200080e Improve snapshot handling for Transaction reinitialization
171c8e8 Update dependencies / Fix Clang
294bdf9 Change Property name from "rocksdb.current_version_number" to "rocksdb.current-super-version-number"
bf1c408 Use pure if-then check instead of assert in EraseColumnFamilyInfo
a7d4eb2 Fix a bug where flush does not happen when a manual compaction is running
68189f7 Update benchmarks used to measure subcompaction performance
dfe96c7 Fix WriteLevel0TableForRecovery file delete protection
451678c Merge pull request #1025 from SherlockNoMad/BuildFix
58ecd91 Fix Windows build
501927f [backupable db] Remove file size embedded in name workaround
ef204df Compaction always needs to be removed from level0_compactions_in_progress_ for universal compaction
e79ad9e Add Iterator Property rocksdb.iterator.version_number
19ea40f Subcompaction boundary keys should not terminate after an empty level
deb08b8 Add parsing of missing DB options
f8e90e8 Get file attributes in bulk for VerifyBackup and CreateNewBackup
12fd9b1 Change BlockBasedTableOptions.format_version default to 2
4572a2d Update current version to 4.6
74b6607 Rename iterator property "rocksdb.iterator.is.key.pinned" => "rocksdb.iterator.is-key-pinned"
6743135 Fix DB::AddFile() issue when PurgeObsoleteFiles() is called
432f3ad Add DB Property "rocksdb.current_version_number"
188bb2e Fix formatting identified by `arc lint`
0f2d2fc Refactored tests to use try-with-resources
f8e02c7 Deprecate org.rocksdb.AbstractNativeReference#dispose() and implement java.lang.AutoCloseable
0f2fdfe Fix the javadoc and the formatting of the base-classes for objects with native references
b5b1db1 Recompute compaction score after scheduling manual compaction
5ea9aa3 TransactionDB:ReinitializeTransaction
1f59541 Introduce Iterator::GetProperty() and replace Iterator::IsKeyPinned()
6778941 Merge pull request #1020 from gongsu832/master
69c471b Handle concurrent manifest update and backup creation
3373c81 Modify build_tools/build_detect_platform to detect and set -march=z10 on Linux s390x.
9905090 Merge branch 'master' of https://github.com/gongsu832/rocksdb
3492889 Merge pull request #1019 from javacruft/wip-omit-leaf-frame-pointer-archs
7ca731b build: Improve -momit-leaf-frame-pointer usage
21f17aa Modified Makefile and build_tools/build_detect_platform to compile on Linux s390x.
8800975 Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more robust
cd3fe67 Remove stale TODO
69c98f0 Reorder instance variables in backup test for proper destruction order
82f15fb Add test to make sure DropColumnFamily doesn't impact existing iterators
38201b3 Fix assert failure when DBImpl::SyncWAL() conflicts with log rolling
2568985 IOStatsContext::ToString() add option to exclude zero counters
b046916 Redo SyncPoints for flush while rolling test
291ae4c Revert "Revert "Fixed the bug when both whole_key_filtering and prefix_extractor are set.""
eef63ef Fixed CompactFiles() spuriously failing or corrupting DB
79ca039 Relax the check condition of prefix_extractor in CheckOptionsCompatibility
4b1b4b8 Merge pull request #1004 from yuslepukhin/child_attr
9ea2968 Implement ConsistentChildrenAttribute   by using default implementation for now as it works.
c7f1a8a Fix LITE build thread_local_test
0914f0c Merge pull request #1003 from yuslepukhin/fix_mutexlock_pthread_build
d37d348 This addresses build issues on Windows  https://github.com/facebook/rocksdb/issues/1002
d825fc7 Use condition variable in log roller test
6b2a047 Fix SstFileManager uninitialized data member
a3db93c Remove the SyncPoint usage in the destructor of PosixEnv
df9ba6d Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage
3943d16 Fix race conditions in auto-rolling logger
d733dd5 [build] Fix env_win.cc compiler errors
cf38e56 Fix broken appveyor build caused by D53991
351252b Merge pull request #998 from fengjian0106/master
1336052 fix ios build error
d08d502 Fix transaction locking
730a422 Improve the documentation of LoadLatestOptions
a7b6f07 Improve write_with_callback_test to sync WAL
5bcf952 Fix WriteImpl empty batch hanging issue
871cc5f fix build without gflags
c90d63a can_unlock set but not used
4437150 Fixed a segfault when compaction fails
2f084d3 Merge pull request #992 from jofusa/jdennison/options-typo-fix
7bd284c Separeate main from bench functionality to allow cusomizations
bd5f842 fixes typo in options logging
1c868d6 Fix includes for env_test
545a193 Add J to commit_prereq so comilation/execution happens in parallel
5bb7371 [build] Evaluate test names only when db_test exists
6a2b4fc Add flag to forcibly disable fallocate
92a9ccf Add a new compaction priority that picks file whose overlapping ratio is smallest
3dc3d1c Merge pull request #984 from petermattis/pmattis/comparator-iterate-upper-bound
239aaf2 Use user_comparator when comparing against iterate_upper_bound.
9081003 Fixed a dependency issue of ThreadLocalPtr
337671b Add universal compaction benchmarks to run_flash_bench.sh
3a67bff Fix an ASAN error in transaction_test.cc
c5af85e Fix a memory leak of Slice objects from org.rocksdb.WBWIRocksIterator#entry1
e84137c Remove unnessecary java.util.List expense in JNI
76e8bee Pass by pointer from/to Java from JNI not by object
0e7e6f6 Improve Javadoc
18eb563 Improve the speed and synchronization around the construction of Java/JNI objects
2a04268 Temporarily disable unstable tests in memory_test.cc
08a78b6 Merge pull request #979 from facebook/update_licenses
21e9581 Updated all copyright headers to the new format.
59b3ee6 Env function for bulk metadata retrieval
4a8cbf4 Allows Get and MultiGet to read directly from SST files.
fe93bf9 Transaction::UndoGetForUpdate
2608219 crash_test: cover concurrent memtable insert in default crash test
a76e909 Fix LITE db_test build broken by previous commit
b1887c5 Explictly fail when memtable doesn't support concurrent insert
8ed3438 Add option to run fillseq with WAL enabled in addition to WAL disabled
73a9b0f Update version to 4.5
6f71d3b Improve perf of Pessimistic Transaction expirations (and optimistic transactions)
8e6172b Add BlockBasedTableOptions::index_block_restart_interval
34a40bf Add --allow_concurrent_memtable_write in stress test and run it in crash_test
73bf330 Merge pull request #973 from yuslepukhin/moveout_testcode
f7c0f4e perf_context.cc and iostats_context.cc use different output macro (fix unity build)
9656eab This partially addresses issue https://github.com/facebook/rocksdb/issues/935   testutil.cc and testharness.cc could not be moved out at this time   as they are used by 4 benchmarks in release builds.
14a3220 Remove references to files deleted in commit abb405227848581d3e6d2ba40d94dbc0a5513902
8445e53 Add a mechanism to run all tests in sandcastle
461cec4 Merge pull request #972 from adamretter/wb-threads
9ab269a Threaded tests for WriteBatch
bf767c6 M…
@yxxyun
Copy link

yxxyun commented Feb 3, 2018

stellar/stellar-core#974

it seems stellar support this.

@ivivan
Copy link

ivivan commented Oct 3, 2018

@robcat Your circular payment example isn't circular :-)

The SendMax and Amount would need to have the same currency. For XRP, you'd probably say something like an 999999 Amount and 5 for SendMax, and select the partial payment flag. You don't care how much you receive as long as it is more than the 5 you put in. It's not well documented (at least it wasn't a year or two ago...) but the first and last step in the path are implied and redundant. For the sake of a real example:

[
	{
		"TransactionType": "Payment",
		"Flags": 458752,
		"Account": "rrrrrrrrrrrrrrrrrrrrrhoLvTp",
		"Sequence": 0,
		"Fee": "0",
		"hash": "0000000000000000000000000000000000000000000000000000000000000000",
		"Destination": "rrrrrrrrrrrrrrrrrrrrrhoLvTp",
		"Amount": {
			"value": "26.10927",
			"currency": "GCB",
			"issuer": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n"
		},
		"SendMax": "360484",
		"Paths": [
			[
				{
					"currency": "GCB",
					"issuer": "rNdwi8ain5ibXNB9A7H3zzKtSxgVzAqqAe",
					"type": 48,
					"type_hex": "0000000000000030"
				},
				{
					"account": "rNdwi8ain5ibXNB9A7H3zzKtSxgVzAqqAe",
					"type": 1,
					"type_hex": "0000000000000001"
				},
				{
					"currency": "BTC",
					"issuer": "rHSFu2BEV15GY3QmDeFznRXqgXkMhihccC",
					"type": 48,
					"type_hex": "0000000000000030"
				},
				{
					"account": "rHSFu2BEV15GY3QmDeFznRXqgXkMhihccC",
					"type": 1,
					"type_hex": "0000000000000001"
				},
				{
					"currency": "GCB",
					"issuer": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n",
					"type": 48,
					"type_hex": "0000000000000030"
				},
				{
					"account": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n",
					"type": 1,
					"type_hex": "0000000000000001"
				}
			]
		]
	},
	{
		"TransactionType": "OfferCreate",
		"Flags": 655360,
		"Account": "rrrrrrrrrrrrrrrrrrrrrhoLvTp",
		"Sequence": 0,
		"Fee": "0",
		"hash": "0000000000000000000000000000000000000000000000000000000000000000",
		"TakerPays": "402027",
		"TakerGets": {
			"value": "26.373",
			"currency": "GCB",
			"issuer": "rBfVgTnsdh8ckC19RM8aVGNuMZnpwrMP6n"
		}
	}
]

The goal is to remove the need for the OfferCreate step to complete the XRP=>XRP circuit.

The point about trustlines I was making earlier is that it is inconsistent that OfferCreates do not require a pre-existing TrustLine limit set, whereas Payments to the same account do. It makes sense for incoming payments from another account to require this be set, and it also makes sense if you want to provide rippling liquidity. But for the conversion scenario, it is just annoying :-)

Hi @donovanhide,

May I ask which gateway rHSFu2BEV15GY3QmDeFznRXqgXkMhihccC belongs to? I got a trouble with my ripple account, and I forget which BTC gateway I am using. I only know I have this address. I am new to ripple, thanks.

@nbougalis nbougalis added the Feature Request Used to indicate requests to add new features label Nov 1, 2018
@nbougalis nbougalis changed the title Enhancement: support for circular payments Support for circular payments Nov 1, 2018
@tuloski
Copy link
Author

tuloski commented Dec 25, 2019

It seems this user (r954fqRdUW2w72uBRMuB4igexVRmu95jjn) was able to find a smart trick or workaround to do circular payments by adding an external trust lines (For example this transaction: 645CD2ADB4B45376F39BDB9E895D431AF9C364BD984E9F4E223A5332950B8269).
I don't know why this kind of transactions isn't forbidden by rippled since the path passes twice from the same currency basically making it a circular payment.

@carlhua
Copy link
Contributor

carlhua commented Aug 6, 2020

This is an old issue - closing the issue. There will be significant performance implications to the ledger if implemented. We can open a new issue if needed.

@carlhua carlhua closed this as completed Aug 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request Used to indicate requests to add new features
Projects
None yet
Development

No branches or pull requests