feat: integration of frigate ephemeral scanning by sdmg15 · Pull Request #48 · bitcoindevkit/bdk-sp

sdmg15 · 2025-12-22T20:56:38Z

Description

This PR integrates the experimental Frigate scanning server with ephemeral client keys.

Notes to the reviewers

The specification and how to run a frigate server can be found at: https://github.com/sparrowwallet/frigate

Changelog notice

Checklists

All Submissions:

I've signed all my commits
I followed the conventional commit guidelines
I ran just p (fmt, clippy and test) before committing

nymius

I have tried the command without success. Could you provide me some examples to try?

If you could create an executable step by step example like the built in doc/tabconf7 would be great.

Notice it builds a container with frigate on it. You only have to expose the ports to be able to communicate with it from outside of the container.

cli/v2/src/main.rs

sdmg15 · 2026-01-23T18:35:12Z

Thanks for a quick round of review, I noticed I have some unpushed changes which I just did.
I tried the containers commands but they took too long to build and I thought I'd just test "manually" by running frigate and rpc outside of it. But I'll give it another try from there will give you the commands to run.

sdmg15 · 2026-02-06T09:35:24Z

@nymius the latest commit 6518059 is in a testable state.
I wasn't able to use the regtest playbook, due to issues mentionned in #49.

The only new command to what is inside regtest_playbook.sh is:
cargo run scan-frigate --url 127.0.0.1:57001 --start 0 assuming that that the Frigate client is running on port 57001

nymius

I fixed the issue with bdk-tx. You should rebase.

Successfully executed scanning on regtest:

$ REGTEST_ADDRESS=$(regtest-bdk unused_address | jq -r '.address' | tr -d '\n')
$ just mine 1 $REGTEST_ADDRESS
[
  "5e724d83a5e8999874ec679ed5823962f56c82bf4810be937b248dc5a08b15cb"
]
$ just mine 101 2>&1 >/dev/null
$ regtest-bdk sync
{}
$ regtest-bdk balance
{
  "satoshi": {
    "confirmed": 5000000000,
    "immature": 0,
    "trusted_pending": 0,
    "untrusted_pending": 0
  }
}
$ SP_CODE=$(regtest-sp code | jq -r '.silent_payment_code' | tr -d '\n')
$ RAW_TX=$(regtest-bdk create_sp_tx --to-sp $SP_CODE:10000 --fee_rate 5 | jq -r '.raw_tx' | tr -d '\n')
$ TXID=$(regtest-bdk broadcast --tx $RAW_TX | jq -r '.txid' | tr -d '\n')
$ just mine 1
[
  "2c28f3fa2cdd742a3e5e564f770802c558324506ea6db3ab6c0cea85ab83c0c3"
]
$ regtest-bdk sync
{}
$ regtest-bdk balance
{
  "satoshi": {
    "confirmed": 4999989228,
    "immature": 0,
    "trusted_pending": 0,
    "untrusted_pending": 0
  }
}
$ regtest-sp scan-frigate --url 127.0.0.1:57001 --start 0
2026-02-12T19:10:12.162809Z  WARN bdk_sp_oracles::frigate: Read bytes from stream 59
2026-02-12T19:10:12.183998Z  WARN bdk_sp_oracles::frigate: Read bytes from stream 156
2026-02-12T19:10:12.184062Z  INFO bdk_sp_oracles::frigate: Subscribed to silent payment address: String("sprt1qqvf42gn2770nwf8jvalczzv80ad8p9zseahnleklzzklqxkg5s80wqa7xxx58l2wvumjlmrzy9eed4vsd647jcmp6jg5tr08unyhex7u5chnkk64")
2026-02-12T19:10:12.184092Z  INFO sp_cli2: Starting frigate scanning loop...
2026-02-12T19:10:12.361669Z  WARN bdk_sp_oracles::frigate: Read bytes from stream 461
2026-02-12T19:10:12.414640Z  WARN bdk_sp_oracles::frigate: Read bytes from stream 200
2026-02-12T19:10:12.474964Z  WARN bdk_sp_oracles::frigate: Read bytes from stream 448
2026-02-12T19:10:12.475701Z  INFO sp_cli2: Progress 1
2026-02-12T19:10:12.475740Z  WARN sp_cli2: Scanning completed
$ echo $SP_CODE
sprt1qqvf42gn2770nwf8jvalczzv80ad8p9zseahnleklzzklqxkg5s80wqa7xxx58l2wvumjlmrzy9eed4vsd647jcmp6jg5tr08unyhex7u5chnkk64
$ regtest-sp balance
{
  "confirmed": {
    "immature": 0,
    "spendable": 0,
    "total": 0
  }
}
{
  "unconfirmed": {
    "total": 10000,
    "trusted": 0,
    "untrusted": 10000
  }
}

Notice the balance appears as unconfirmed because the block to which the transaction is anchored doesn't belong to the wallet local chain. I left a comment in the code mentioning how to fix that.

I think that Read bytes from stream {bytes} belongs to the DEBUG scope, rather than WARN. I would also print a one line log of the balance, after each execution of the command.

Could you give me more information about the issues that are preventing you from testing with the playbooks? Could you setup nix?

Good work!

cli/v2/src/main.rs

oracles/src/frigate/mod.rs

cli/v2/src/main.rs

sdmg15 · 2026-03-11T17:36:41Z

Added a new playbook 59a1128. It's important to note that for this work we need to update Frigate to 1.3.2

nymius

I've checked the frigate_playbook.sh execution and it works great. I'm glad you were able to replicate it.

I recommend you to rebase and check each commit individually and:

execute git show --check and fix any issue arising.
execute just p and implement fixes and apply formatting.
rewrite commits to use conventional commit format. Look at commit history to understand the conventions and scopes used.
squash the commits into a single one.

nymius · 2026-03-16T20:52:01Z

cli/v2/src/main.rs

                );
            }
        }
+        Commands::ScanFrigate { rpc_args, start } => {


For start you could implement something similar than ScanCbf logic:

let sync_point = if let (Some(height), Some(hash)) = (height, hash) { HeaderCheckpoint::new(height, hash) } else if wallet.birthday.height <= wallet.chain().tip().height() { let height = wallet.chain().tip().height(); let hash = wallet.chain().tip().hash(); HeaderCheckpoint::new(height, hash) } else { let checkpoint = wallet .chain() .get(wallet.birthday.height) .expect("should be something"); let height = checkpoint.height(); let hash = checkpoint.hash(); HeaderCheckpoint::new(height, hash) };

nymius · 2026-03-16T20:53:12Z

cli/v2/src/main.rs

            }
        }
+        Commands::ScanFrigate { rpc_args, start } => {
+            // The implementation done here differ from what mentionned in the section https://github.com/sparrowwallet/frigate/tree/master?tab=readme-ov-file#blockchainsilentpaymentssubscribe


Suggested change

// The implementation done here differ from what mentionned in the section https://github.com/sparrowwallet/frigate/tree/master?tab=readme-ov-file#blockchainsilentpaymentssubscribe

// The implementation done here differs from what is mentioned in the section

// https://github.com/sparrowwallet/frigate/tree/master?tab=readme-ov-file#blockchainsilentpaymentssubscribe

// This implementation is doing a one time scanning only. So instead of calling

// `blockchain.scripthash.subscribe` on each script from the wallet, we just subscribe

// and read the scanning result from the stream. On each result received we update the

// wallet state and once scanning progress reaches 1.0 (100%) we stop.

nymius · 2026-03-16T20:53:56Z

cli/v2/src/main.rs

+            let mut client = FrigateClient::connect(&rpc_args.url)
+                .await
+                .unwrap()
+                .with_timeout(tokio::time::Duration::from_secs(600));


10 minutes is a lot of time. I would reduce this to 60 seconds.

nymius · 2026-03-16T20:54:21Z

cli/v2/src/main.rs

+                                raw_blk.push_str("00");
+
+                                // Push dummy coinbase
+                                let dummy_coinbase = "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff1b03951a0604f15ccf5609013803062b9b5a0100072f425443432f20000000000000000000";


I would make a const of this hex value.

nymius · 2026-03-16T20:54:47Z

oracles/src/frigate/mod.rs

+const UNSUBSCRIBE_RPC_METHOD: &str = "blockchain.silentpayments.unsubscribe";
+const GET_RPC_METHOD: &str = "blockchain.transaction.get";
+const VERSION_RPC_METHOD: &str = "server.version";
+const _SUBSCRIBE_OWNED_OUTPUTS: &str = "blockchain.scripthash.subscribe";


I would remove this if unused.

nymius · 2026-03-16T21:02:21Z

oracles/src/frigate/mod.rs

+        let req = RequestPayload {
+            method: GET_RPC_METHOD.to_string(),
+            params: serde_json::json!(params),
+            id: serde_json::Value::from(51),


Same question about ID than before.

nymius · 2026-03-16T21:02:27Z

oracles/src/frigate/mod.rs

+        let req = RequestPayload {
+            method: VERSION_RPC_METHOD.to_string(),
+            params: serde_json::json!(params),
+            id: serde_json::Value::from(71),


Also about ID.

nymius · 2026-03-16T21:02:37Z

oracles/src/frigate/mod.rs

+        let req = RequestPayload {
+            method: SUBSCRIBE_RPC_METHOD.to_string(),
+            params: serde_json::json!(params),
+            id: serde_json::Value::from(91),


Also about ID.

nymius · 2026-03-16T21:02:52Z

oracles/src/frigate/mod.rs

+        let req = RequestPayload {
+            method: UNSUBSCRIBE_RPC_METHOD.to_string(),
+            params: serde_json::json!(params),
+            id: serde_json::Value::from(189),


nymius · 2026-03-16T21:03:37Z

Cargo.lock

+ "rand 0.8.5",
+ "rand_core 0.6.4",


Why are you repeating dependencies here? Remove this from the commit, please.

sdmg15 force-pushed the feat/frigate branch from c72dc84 to ceb8420 Compare January 12, 2026 20:21

nymius reviewed Jan 23, 2026

View reviewed changes

cli/v2/src/main.rs Outdated Show resolved Hide resolved

cli/v2/src/main.rs Outdated Show resolved Hide resolved

cli/v2/src/main.rs Outdated Show resolved Hide resolved

nymius assigned sdmg15 Jan 23, 2026

nymius added the enhancement New feature or request label Jan 23, 2026

sdmg15 force-pushed the feat/frigate branch from ceb8420 to 732ca71 Compare January 23, 2026 18:31

sdmg15 marked this pull request as ready for review January 23, 2026 18:32

sdmg15 requested a review from nymius February 6, 2026 09:28

sdmg15 force-pushed the feat/frigate branch from c4132e8 to 6518059 Compare February 6, 2026 09:30

nymius reviewed Feb 12, 2026

View reviewed changes

cli/v2/src/main.rs Outdated Show resolved Hide resolved

oracles/src/frigate/mod.rs Outdated Show resolved Hide resolved

cli/v2/src/main.rs Outdated Show resolved Hide resolved

sdmg15 added 3 commits February 28, 2026 21:16

integration of frigate ephemeral scanning

a3f47a8

feat: improve listening to tcp stream

231cd36

add wallet checkpoint for balance updates

676d135

sdmg15 force-pushed the feat/frigate branch from 6518059 to 676d135 Compare March 2, 2026 10:36

sdmg15 added 2 commits March 4, 2026 19:17

feat: add default timeout long requests

af183df

\add frigate non nix playbook

59a1128

sdmg15 force-pushed the feat/frigate branch from d1faf60 to 59a1128 Compare March 11, 2026 17:35

nymius requested changes Mar 16, 2026

View reviewed changes

-            // The implementation done here differ from what mentionned in the section https://github.com/sparrowwallet/frigate/tree/master?tab=readme-ov-file#blockchainsilentpaymentssubscribe
+// The implementation done here differs from what is mentioned in the section
+// https://github.com/sparrowwallet/frigate/tree/master?tab=readme-ov-file#blockchainsilentpaymentssubscribe
+// This implementation is doing a one time scanning only. So instead of calling
+// `blockchain.scripthash.subscribe` on each script from the wallet, we just subscribe
+// and read the scanning result from the stream. On each result received we update the
+// wallet state and once scanning progress reaches 1.0 (100%) we stop.

Conversation

sdmg15 commented Dec 22, 2025

Description

Notes to the reviewers

Changelog notice

Checklists

All Submissions:

Uh oh!

nymius left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdmg15 commented Jan 23, 2026

Uh oh!

sdmg15 commented Feb 6, 2026

Uh oh!

nymius left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdmg15 commented Mar 11, 2026

Uh oh!

nymius left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nymius left a comment •

edited

Loading