Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wallet uses JSON RPC #2929

Merged
merged 87 commits into from
Jan 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
7c244c9
Wallet talks to index over http
raphjaph Dec 31, 2023
e0efc59
Fix test
raphjaph Jan 2, 2024
bf5d451
It compiles
raphjaph Jan 2, 2024
548ddae
Place clippy + fmt
raphjaph Jan 2, 2024
8166805
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 2, 2024
17ca387
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 3, 2024
bb9ab59
Put chain in wallet
raphjaph Jan 3, 2024
b3dcb8b
Fix wallet bug
raphjaph Jan 3, 2024
cfd83c0
Getting somewhere; need to add status json endpoint
raphjaph Jan 3, 2024
00d72a4
Add TODO
raphjaph Jan 3, 2024
2578136
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 3, 2024
a73b960
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 3, 2024
c495f1f
Get server status
raphjaph Jan 3, 2024
ab9b080
Inscribe does not use index
raphjaph Jan 4, 2024
b0771dd
Rid balance command of the index
raphjaph Jan 4, 2024
44c350e
inscriptions command no index
raphjaph Jan 4, 2024
c8ea786
Rid sats command of index
raphjaph Jan 4, 2024
903857d
Start work on send, waiting for rune json endpoint
raphjaph Jan 4, 2024
69d4ba2
output and cardinal command no index
raphjaph Jan 4, 2024
8cc344f
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 5, 2024
38571af
fix merge
raphjaph Jan 5, 2024
c1d3605
Etch no index
raphjaph Jan 5, 2024
080998f
Send no index
raphjaph Jan 5, 2024
e17e475
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 6, 2024
b274285
Stuck
raphjaph Jan 6, 2024
c6c1abe
Add in index for output JSON
raphjaph Jan 6, 2024
aef6879
Merge branch 'add-in-index-to-output' into wallet-spawns-server
raphjaph Jan 6, 2024
6a15930
Add check sync
raphjaph Jan 6, 2024
fbb2e99
Fix unsynced index test
raphjaph Jan 6, 2024
296bd38
Remove unused functions and only pass in wallet
raphjaph Jan 6, 2024
8ecb107
stashing
raphjaph Jan 7, 2024
364123b
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 8, 2024
5d08319
Remove bool
raphjaph Jan 8, 2024
32e191b
fmt
raphjaph Jan 8, 2024
cbeb7fb
quick fix
raphjaph Jan 8, 2024
d40d14c
It works, just have to fix one test
raphjaph Jan 10, 2024
42172fe
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 10, 2024
f8480ba
quick fix
raphjaph Jan 10, 2024
e6de26c
Fix preview test
raphjaph Jan 10, 2024
c297ab4
It works but tests are slow now
raphjaph Jan 10, 2024
7ef0544
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 10, 2024
e3f8114
Some cleanup
raphjaph Jan 10, 2024
5a80f92
quick fix
raphjaph Jan 11, 2024
8585a54
quick fix
raphjaph Jan 11, 2024
eab91f4
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 11, 2024
df9b15e
Place clippy
raphjaph Jan 11, 2024
39fe0d6
quick fix
raphjaph Jan 11, 2024
efeede5
Clean up some bits here and there
raphjaph Jan 11, 2024
85417cd
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 13, 2024
66755df
Move wallet into separate module
raphjaph Jan 13, 2024
8be34a5
Place clippy
raphjaph Jan 13, 2024
288d25d
Horrible state
raphjaph Jan 15, 2024
a22ca3d
Horribel state 2
raphjaph Jan 15, 2024
6815ce4
Tests pass
raphjaph Jan 16, 2024
55b091d
Split up batch stuff
raphjaph Jan 16, 2024
930bf5d
Merge branch 'master' of github.com:ordinals/ord into wallet-spawns-s…
raphjaph Jan 16, 2024
257c2d8
Fix test
raphjaph Jan 16, 2024
f59744b
Require running ord server for wallet commands
raphjaph Jan 16, 2024
0e291ab
Got one test to pass with new setup
raphjaph Jan 19, 2024
660da09
All unit tests pass
raphjaph Jan 19, 2024
a7c883e
I think all integration tests pass
raphjaph Jan 20, 2024
9633b60
Rename helper function
raphjaph Jan 20, 2024
607e539
Maybe faster now
raphjaph Jan 21, 2024
5ede49f
Placate clippy
raphjaph Jan 21, 2024
1be0799
Fix preview command
raphjaph Jan 22, 2024
4bb5963
Fixed preview test?
raphjaph Jan 22, 2024
6e29ccb
quick fix
raphjaph Jan 22, 2024
b68451c
quick fix
raphjaph Jan 22, 2024
75a119f
quick fix
raphjaph Jan 22, 2024
1ee921c
Remove preview command
raphjaph Jan 22, 2024
c16b3c6
remove unused options field
casey Jan 22, 2024
e375a7a
Sort
casey Jan 22, 2024
b986bbe
Gracefully shut down index in ctrl-c handler
casey Jan 22, 2024
5b6b5bd
Move shutdown into success branch
casey Jan 22, 2024
151cd4d
Tweak
casey Jan 22, 2024
050a3ca
check -> has
casey Jan 22, 2024
a1dc47d
Avoid clone
casey Jan 22, 2024
7755af3
Reuse bitcoin client
casey Jan 22, 2024
fe7197b
Reuse client
casey Jan 22, 2024
68aadc8
Rename modules
casey Jan 22, 2024
20ff4fa
REsuse client
casey Jan 22, 2024
2e2a9eb
Use correct create wallet
casey Jan 23, 2024
ea7ba4e
Wallet
casey Jan 23, 2024
b3cd660
Tweak
casey Jan 23, 2024
5ce1d28
Tweak
casey Jan 23, 2024
4d1da88
Tweak
casey Jan 23, 2024
fdeff81
Tweak
casey Jan 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ mp4 = "0.14.0"
ord-bitcoincore-rpc = "0.17.1"
redb = "1.5.0"
regex = "1.6.0"
reqwest = { version = "0.11.23", features = ["blocking", "json"] }
rss = "2.0.1"
rust-embed = "8.0.0"
rustls = "0.22.0"
Expand Down
4 changes: 0 additions & 4 deletions docs/src/guides/explorer.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,6 @@ To enable the JSON-API endpoints add the `--enable-json-api` or `-j` flag (see

`ord server --enable-json-api`

To test how your inscriptions will look you can run:

`ord preview <FILE1> <FILE2> ...`

Search
------

Expand Down
3 changes: 0 additions & 3 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -171,9 +171,6 @@ update-changelog:
echo >> CHANGELOG.md
git log --pretty='format:- %s' >> CHANGELOG.md

preview-examples:
cargo run preview examples/*

convert-logo-to-favicon:
convert -background none -resize 256x256 logo.svg static/favicon.png

Expand Down
113 changes: 3 additions & 110 deletions src/index.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ use {
TableDefinition, TableHandle, TableStats, WriteTransaction,
},
std::{
collections::{BTreeSet, HashMap},
collections::HashMap,
io::{BufWriter, Write},
sync::{Mutex, Once},
},
Expand Down Expand Up @@ -174,7 +174,7 @@ pub(crate) struct InscriptionInfo {
pub(crate) charms: u16,
}

trait BitcoinCoreRpcResultExt<T> {
pub(crate) trait BitcoinCoreRpcResultExt<T> {
fn into_option(self) -> Result<Option<T>>;
}

Expand Down Expand Up @@ -417,20 +417,6 @@ impl Index {
self.durability = durability;
}

pub(crate) fn check_sync(&self, utxos: &BTreeMap<OutPoint, Amount>) -> Result<bool> {
let rtx = self.database.begin_read()?;
let outpoint_to_value = rtx.open_table(OUTPOINT_TO_VALUE)?;
for outpoint in utxos.keys() {
if outpoint_to_value.get(&outpoint.store())?.is_none() {
return Err(anyhow!(
"output in Bitcoin Core wallet but not in ord index: {outpoint}"
));
}
}

Ok(true)
}

pub(crate) fn contains_output(&self, output: &OutPoint) -> Result<bool> {
Ok(
self
Expand Down Expand Up @@ -858,32 +844,6 @@ impl Index {
Ok(entries)
}

pub(crate) fn get_rune_balance(&self, outpoint: OutPoint, id: RuneId) -> Result<u128> {
let rtx = self.database.begin_read()?;

let outpoint_to_balances = rtx.open_table(OUTPOINT_TO_RUNE_BALANCES)?;

let Some(balances) = outpoint_to_balances.get(&outpoint.store())? else {
return Ok(0);
};

let balances_buffer = balances.value();

let mut i = 0;
while i < balances_buffer.len() {
let (balance_id, length) = runes::varint::decode(&balances_buffer[i..]);
i += length;
let (amount, length) = runes::varint::decode(&balances_buffer[i..]);
i += length;

if RuneId::try_from(balance_id).unwrap() == id {
return Ok(amount);
}
}

Ok(0)
}

pub(crate) fn get_rune_balances_for_outpoint(
&self,
outpoint: OutPoint,
Expand Down Expand Up @@ -925,22 +885,6 @@ impl Index {
Ok(balances)
}

pub(crate) fn get_runic_outputs(&self, outpoints: &[OutPoint]) -> Result<BTreeSet<OutPoint>> {
let rtx = self.database.begin_read()?;

let outpoint_to_balances = rtx.open_table(OUTPOINT_TO_RUNE_BALANCES)?;

let mut runic = BTreeSet::new();

for outpoint in outpoints {
if outpoint_to_balances.get(&outpoint.store())?.is_some() {
runic.insert(*outpoint);
}
}

Ok(runic)
}

pub(crate) fn get_rune_balance_map(&self) -> Result<BTreeMap<Rune, BTreeMap<OutPoint, u128>>> {
let outpoint_balances = self.get_rune_balances()?;

Expand Down Expand Up @@ -1575,28 +1519,6 @@ impl Index {
))
}

pub(crate) fn get_inscriptions(
&self,
utxos: &BTreeMap<OutPoint, Amount>,
) -> Result<BTreeMap<SatPoint, InscriptionId>> {
let rtx = self.database.begin_read()?;

let mut result = BTreeMap::new();

let satpoint_to_sequence_number = rtx.open_multimap_table(SATPOINT_TO_SEQUENCE_NUMBER)?;
let sequence_number_to_inscription_entry =
rtx.open_table(SEQUENCE_NUMBER_TO_INSCRIPTION_ENTRY)?;
for utxo in utxos.keys() {
result.extend(Self::inscriptions_on_output(
&satpoint_to_sequence_number,
&sequence_number_to_inscription_entry,
*utxo,
)?);
}

Ok(result)
}

pub(crate) fn get_inscriptions_paginated(
&self,
page_size: u32,
Expand Down Expand Up @@ -2036,11 +1958,7 @@ impl Index {

#[cfg(test)]
mod tests {
use {
super::*,
crate::index::testing::Context,
bitcoin::secp256k1::rand::{self, RngCore},
};
use {super::*, crate::index::testing::Context};

#[test]
fn height_limit() {
Expand Down Expand Up @@ -3339,31 +3257,6 @@ mod tests {
}
}

#[test]
fn unsynced_index_fails() {
for context in Context::configurations() {
let mut entropy = [0; 16];
rand::thread_rng().fill_bytes(&mut entropy);
let mnemonic = Mnemonic::from_entropy(&entropy).unwrap();
crate::subcommand::wallet::initialize("ord".into(), &context.options, mnemonic.to_seed(""))
.unwrap();
context.rpc_server.mine_blocks(1);
assert_regex_match!(
crate::subcommand::wallet::get_unspent_outputs(
&crate::subcommand::wallet::bitcoin_rpc_client_for_wallet_command(
"ord".to_string(),
&context.options,
)
.unwrap(),
&context.index
)
.unwrap_err()
.to_string(),
r"output in Bitcoin Core wallet but not in ord index: [[:xdigit:]]{64}:\d+"
);
}
}

#[test]
fn unrecognized_even_field_inscriptions_are_cursed_and_unbound() {
for context in Context::configurations() {
Expand Down
4 changes: 1 addition & 3 deletions src/index/testing.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use super::*;
use {super::*, std::ffi::OsString, tempfile::TempDir};

pub(crate) struct ContextBuilder {
args: Vec<OsString>,
Expand Down Expand Up @@ -36,7 +36,6 @@ impl ContextBuilder {
index.update().unwrap();

Ok(Context {
options,
rpc_server,
tempdir,
index,
Expand Down Expand Up @@ -65,7 +64,6 @@ impl ContextBuilder {
}

pub(crate) struct Context {
pub(crate) options: Options,
pub(crate) rpc_server: test_bitcoincore_rpc::Handle,
#[allow(unused)]
pub(crate) tempdir: TempDir,
Expand Down
34 changes: 23 additions & 11 deletions src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
#![allow(
clippy::large_enum_variant,
clippy::result_large_err,
clippy::too_many_arguments,
clippy::type_complexity,
clippy::result_large_err
clippy::type_complexity
)]
#![deny(
clippy::cast_lossless,
Expand Down Expand Up @@ -59,15 +60,14 @@ use {
cmp,
collections::{BTreeMap, BTreeSet, HashMap, HashSet, VecDeque},
env,
ffi::OsString,
fmt::{self, Display, Formatter},
fs::{self, File},
io::{self, Cursor},
mem,
net::{TcpListener, ToSocketAddrs},
net::ToSocketAddrs,
ops::{Add, AddAssign, Sub},
path::{Path, PathBuf},
process::{self, Command},
process,
str::FromStr,
sync::{
atomic::{self, AtomicBool},
Expand All @@ -77,7 +77,7 @@ use {
time::{Duration, Instant, SystemTime},
},
sysinfo::System,
tempfile::TempDir,
templates::{InscriptionJson, OutputJson, RuneJson, StatusJson},
tokio::{runtime::Runtime, task},
};

Expand All @@ -92,7 +92,7 @@ pub use self::{
runes::{Edict, Rune, RuneId, Runestone},
sat::Sat,
sat_point::SatPoint,
subcommand::wallet::transaction_builder::{Target, TransactionBuilder},
wallet::transaction_builder::{Target, TransactionBuilder},
};

#[cfg(test)]
Expand All @@ -112,7 +112,7 @@ macro_rules! tprintln {
};
}

mod arguments;
pub mod arguments;
mod blocktime;
pub mod chain;
mod config;
Expand All @@ -123,7 +123,7 @@ mod deserialize_from_str;
mod epoch;
mod fee_rate;
mod height;
mod index;
pub mod index;
mod inscriptions;
mod object;
mod options;
Expand All @@ -137,6 +137,7 @@ mod server_config;
pub mod subcommand;
mod tally;
pub mod templates;
pub mod wallet;

type Result<T = (), E = Error> = std::result::Result<T, E>;

Expand Down Expand Up @@ -200,6 +201,16 @@ fn unbound_outpoint() -> OutPoint {
}
}

pub fn parse_ord_server_args(args: &str) -> (Options, crate::subcommand::server::Server) {
match Arguments::try_parse_from(args.split_whitespace()) {
Ok(arguments) => match arguments.subcommand {
Subcommand::Server(server) => (arguments.options, server),
subcommand => panic!("unexpected subcommand: {subcommand:?}"),
},
Err(err) => panic!("error parsing arguments: {err}"),
}
}

fn gracefully_shutdown_indexer() {
if let Some(indexer) = INDEXER.lock().unwrap().take() {
// We explicitly set this to true to notify the thread to not take on new work
Expand All @@ -226,6 +237,8 @@ pub fn main() {
.unwrap()
.iter()
.for_each(|handle| handle.graceful_shutdown(Some(Duration::from_millis(100))));

gracefully_shutdown_indexer();
})
.expect("Error setting <CTRL-C> handler");

Expand All @@ -251,8 +264,7 @@ pub fn main() {
if let Some(output) = output {
output.print_json();
}
gracefully_shutdown_indexer();
}
}

gracefully_shutdown_indexer();
}
4 changes: 2 additions & 2 deletions src/options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ impl Options {

#[cfg(test)]
mod tests {
use {super::*, bitcoin::Network, std::path::Path};
use {super::*, bitcoin::Network, std::path::Path, tempfile::TempDir};

#[test]
fn rpc_url_overrides_network() {
Expand Down Expand Up @@ -569,7 +569,7 @@ mod tests {
);
}

fn parse_wallet_args(args: &str) -> (Options, subcommand::wallet::Wallet) {
fn parse_wallet_args(args: &str) -> (Options, subcommand::wallet::WalletCommand) {
match Arguments::try_parse_from(args.split_whitespace()) {
Ok(arguments) => match arguments.subcommand {
Subcommand::Wallet(wallet) => (arguments.options, wallet),
Expand Down
9 changes: 5 additions & 4 deletions src/runes/pile.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
use super::*;

pub(crate) struct Pile {
pub(crate) amount: u128,
pub(crate) divisibility: u8,
pub(crate) symbol: Option<char>,
#[derive(Debug, Deserialize, Serialize, PartialEq)]
pub struct Pile {
pub amount: u128,
pub divisibility: u8,
pub symbol: Option<char>,
}

impl Display for Pile {
Expand Down
Loading
Loading