Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upd gha workflows #37

Closed
wants to merge 284 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
284 commits
Select commit Hold shift + click to select a range
8b1a77e
add coinbase and move core to gold
forgxyz Jul 18, 2023
6a063b3
tx final incr strat
forgxyz Jul 18, 2023
c5119c6
coinbase
forgxyz Jul 18, 2023
04f0f2d
adj test
forgxyz Jul 18, 2023
bc14b1a
Merge pull request #4 from FlipsideCrypto/add_coinbase
forgxyz Jul 18, 2023
eab0fbf
alter tag on bronze load models
forgxyz Jul 18, 2023
86dea1a
tags
forgxyz Jul 18, 2023
e42d690
Merge pull request #5 from FlipsideCrypto/update_tags
forgxyz Jul 18, 2023
04bc4b5
add addl block filter
forgxyz Jul 18, 2023
5e75ab7
upd block test
forgxyz Jul 18, 2023
61fd3df
add block test, adj tests
forgxyz Jul 18, 2023
52abd60
explicitly identify txid
forgxyz Jul 18, 2023
c3c0d09
factor out tx index
forgxyz Jul 18, 2023
65ee913
join condition
forgxyz Jul 18, 2023
b295153
group by
forgxyz Jul 18, 2023
37a2035
tag index model
forgxyz Jul 18, 2023
73a7cdf
blocks rewards and gold folders
forgxyz Jul 18, 2023
728df16
coinbase in txs
forgxyz Jul 18, 2023
9e286a1
add incr to inputs
forgxyz Jul 18, 2023
3bbf869
inputs_final
forgxyz Jul 18, 2023
9246af7
Merge pull request #6 from FlipsideCrypto/filter_adjs
forgxyz Jul 18, 2023
7078d98
typo
forgxyz Jul 18, 2023
25a14c1
adj jobs to 1hr
forgxyz Jul 18, 2023
13f6ffc
Merge pull request #7 from FlipsideCrypto/prerelease
forgxyz Jul 19, 2023
a2bd3b0
typo
forgxyz Jul 19, 2023
e1202f4
adj run schedule
forgxyz Jul 19, 2023
9759f6c
disable miner reward for now
forgxyz Jul 19, 2023
9b2b115
disable miner reward for now (core)
forgxyz Jul 19, 2023
ac0c430
fix alias and ref
forgxyz Jul 19, 2023
4aecdfc
Merge pull request #8 from FlipsideCrypto/incr_fix
forgxyz Jul 19, 2023
82e391e
broaden incr for inputs final
forgxyz Jul 19, 2023
8776f2e
del outputs filter
forgxyz Jul 19, 2023
35948bd
use txs for rewards
forgxyz Jul 19, 2023
eabf6c9
add desc
forgxyz Jul 19, 2023
d2476ab
alter txs final
forgxyz Jul 19, 2023
302458a
set view order
forgxyz Jul 19, 2023
56a23f3
update desc todos
forgxyz Jul 19, 2023
0cf1a45
Merge pull request #10 from FlipsideCrypto/core_view_order
forgxyz Jul 19, 2023
bb258a7
move tx array to end of blocks view
forgxyz Jul 19, 2023
2c4042a
comma
forgxyz Jul 19, 2023
554b829
adjust yml
forgxyz Jul 19, 2023
9ab354b
Merge pull request #11 from FlipsideCrypto/yml_adj
forgxyz Jul 19, 2023
b89f6a0
add help links
forgxyz Jul 19, 2023
93b508a
Enable
forgxyz Jul 19, 2023
5e8fe07
sequence test
forgxyz Jul 19, 2023
e32ff80
Merge pull request #12 from FlipsideCrypto/docs_overview
forgxyz Jul 19, 2023
454c5f8
qualify
forgxyz Jul 19, 2023
0497d82
Merge pull request #13 from FlipsideCrypto/bronze_dedupe_change
forgxyz Jul 19, 2023
ef0f6fb
model tag
forgxyz Jul 19, 2023
db06285
test updates and txs lookback
forgxyz Jul 20, 2023
3721f5b
Merge pull request #14 from FlipsideCrypto/inputs_lookback
forgxyz Jul 20, 2023
79c51c3
incr miner
forgxyz Jul 20, 2023
06253fd
Merge pull request #9 from FlipsideCrypto/adj_rewards
forgxyz Jul 20, 2023
a8d6a02
add coalesce
forgxyz Jul 20, 2023
22dbf9c
Merge pull request #15 from FlipsideCrypto/rewards_coalesce
forgxyz Jul 20, 2023
e36180a
add ez to hyperlink
forgxyz Jul 21, 2023
e7c13f8
urls
forgxyz Jul 21, 2023
3b58782
Merge pull request #16 from FlipsideCrypto/docs_update_ez_rewards
forgxyz Jul 21, 2023
d5a4ea7
silence nextblockhash temp
forgxyz Jul 23, 2023
967d07b
add pending block retry
forgxyz Jul 24, 2023
1571601
add notnull back
forgxyz Jul 24, 2023
2341a85
rm all
forgxyz Jul 24, 2023
e6ad1f6
Merge pull request #17 from FlipsideCrypto/AN-3598/blocks_retro
forgxyz Jul 24, 2023
30108b7
upd cluster on txs and inputs final
forgxyz Jul 25, 2023
530df2f
typo
forgxyz Jul 25, 2023
bcc53e7
Merge pull request #18 from FlipsideCrypto/upd_final_clustering
forgxyz Jul 25, 2023
4355adb
add requirements file
forgxyz Jul 28, 2023
fcfcbb2
upd slack msg
forgxyz Jul 31, 2023
659dc43
Update dbt_test_alert.py
forgxyz Jul 31, 2023
08180e8
!
forgxyz Aug 1, 2023
3c683d7
Merge pull request #20 from FlipsideCrypto/upd_slack_msg
forgxyz Aug 1, 2023
af4dc46
upd next_block_hash severity
forgxyz Aug 10, 2023
544ed3f
Merge pull request #21 from FlipsideCrypto/AN-3723/upd_test
forgxyz Aug 10, 2023
4a4d8c1
del+ins
forgxyz Aug 16, 2023
92f9ebb
Merge pull request #22 from FlipsideCrypto/quickfix_081829/bronze_incr
forgxyz Aug 16, 2023
62deee9
txs del+ins
forgxyz Aug 16, 2023
5d5a21f
Merge pull request #23 from FlipsideCrypto/quickfix_081829-2/bronze_incr
forgxyz Aug 16, 2023
2e26eed
upd_adhoc_title
forgxyz Aug 17, 2023
fedbeea
Merge pull request #24 from FlipsideCrypto/upd_adhoc
forgxyz Aug 17, 2023
0122eb9
add retry for missing txs
forgxyz Aug 18, 2023
8c9fb32
Merge pull request #25 from FlipsideCrypto/AN-3773/enhance_retry
forgxyz Aug 18, 2023
2909b9f
add block hash
forgxyz Aug 19, 2023
d576e87
Merge pull request #26 from FlipsideCrypto/hotfix_081823_retry
forgxyz Aug 19, 2023
2eac9c8
14 day pending block window
forgxyz Aug 21, 2023
27c9c25
Merge pull request #27 from FlipsideCrypto/082123_alter_pending_block…
forgxyz Aug 21, 2023
4368cba
90 day temp
forgxyz Aug 21, 2023
59c68de
Merge pull request #28 from FlipsideCrypto/082123_alter_pending_block…
forgxyz Aug 21, 2023
09b7ffa
back to 3 days
forgxyz Aug 22, 2023
0d09ce5
Merge pull request #29 from FlipsideCrypto/082123_alter_pending_block…
forgxyz Aug 22, 2023
cb4b3db
add qualify to take latest hash
forgxyz Aug 22, 2023
b96d43a
Merge pull request #30 from FlipsideCrypto/AN-3823_upd_streamline_rea…
forgxyz Aug 22, 2023
b7a9f7b
activate block hash test and walkback realtime blocks qualify
forgxyz Aug 22, 2023
67ebc2b
Merge pull request #31 from FlipsideCrypto/AN-3823_adj_txs_blockfork
forgxyz Aug 22, 2023
60adb3f
blocks completeness
forgxyz Aug 28, 2023
95272ff
txs completeness model
forgxyz Aug 28, 2023
1b100d1
inputs, outputs and workflow
forgxyz Aug 28, 2023
3e49168
add tests to obs. update bronze txs to avoid empty arrays
forgxyz Aug 29, 2023
b2b137a
Merge pull request #32 from FlipsideCrypto/AN-3710/observability
forgxyz Aug 29, 2023
5853267
add OBSERV_FULL_TEST
forgxyz Aug 29, 2023
0c43385
Merge pull request #33 from FlipsideCrypto/quickfix_add_var
forgxyz Aug 29, 2023
c095958
schema reorg miner rewards
forgxyz Aug 29, 2023
3da052c
add depr warning to correct yml file
forgxyz Aug 29, 2023
df4bc0d
change to TEST_WAREHOUSE var
forgxyz Aug 30, 2023
6121255
Merge pull request #35 from FlipsideCrypto/quickfix_test_warehouse
forgxyz Aug 30, 2023
a41eda7
Merge pull request #34 from FlipsideCrypto/AN-3826/schema_reorg
forgxyz Aug 31, 2023
7ebe968
add deployment script - copied file from near
forgxyz Aug 31, 2023
914d661
add desc line
forgxyz Aug 31, 2023
81bfe15
Merge pull request #36 from FlipsideCrypto/add_deployment_gha
forgxyz Aug 31, 2023
f545516
add select tags
forgxyz Sep 7, 2023
5a1f6aa
upd pending blocks to get hash from complete if null in silver
forgxyz Sep 7, 2023
20576ef
Merge pull request #38 from FlipsideCrypto/upd_pending_blocks
forgxyz Sep 7, 2023
2014f84
Update dbt_project.yml
forgxyz Sep 8, 2023
64bd4d9
Merge pull request #37 from FlipsideCrypto/AN-3913/select_tags
eric-laurello Sep 8, 2023
735df22
upd _missing_transactions to incl null
forgxyz Sep 8, 2023
e679ae4
ephemeral
forgxyz Sep 8, 2023
00cacad
typo
forgxyz Sep 8, 2023
b77b631
Merge pull request #39 from FlipsideCrypto/enhance_missing_tx
forgxyz Sep 8, 2023
e6a5d55
add null check to incr logic
forgxyz Sep 8, 2023
46854a8
upd lookback in silver txs
forgxyz Sep 8, 2023
c1089b5
add block num restriction on txs
forgxyz Sep 11, 2023
b781c68
add output incr back to txs final
forgxyz Sep 11, 2023
049611c
add block miner reward lookback
forgxyz Sep 11, 2023
1e9e8e3
del block miner change
forgxyz Sep 11, 2023
394e380
Merge pull request #41 from FlipsideCrypto/upd_incr_logic
forgxyz Sep 11, 2023
d4ba901
update schedules
forgxyz Sep 25, 2023
9afdb1a
upd schedules and disable block hash
forgxyz Sep 25, 2023
0de6337
Merge pull request #42 from FlipsideCrypto/latency_upd_schedule_20m
eric-laurello Sep 25, 2023
9711744
labels view, and minor other tweak moving script
forgxyz Sep 26, 2023
b3c4300
newline
forgxyz Sep 26, 2023
1feb8af
Merge pull request #43 from FlipsideCrypto/AN-3925/labels
forgxyz Sep 26, 2023
f5d7d14
adding btc entity clusters
gronders Sep 27, 2023
ad5d128
fix cluster yml
gronders Sep 27, 2023
959c39d
cluster yml fix
gronders Sep 27, 2023
1d1ebf4
fix cluster yml files
gronders Sep 27, 2023
dda139e
upd to table, add tags, move sql out into views, upd core view name, …
forgxyz Sep 27, 2023
a0bb655
add weekly scheduled job
forgxyz Sep 27, 2023
3c59f26
newline
forgxyz Sep 27, 2023
8bd0122
Merge pull request #44 from FlipsideCrypto/entity_cluster
gronders Sep 27, 2023
ad69b41
rem inserted timestamp from core view
forgxyz Sep 27, 2023
5537cde
Merge pull request #45 from FlipsideCrypto/upd_core_view
forgxyz Sep 27, 2023
61f99eb
add date filter for just non lowered btc labels
forgxyz Oct 4, 2023
2bde691
Merge pull request #46 from FlipsideCrypto/labels_del_old
forgxyz Oct 4, 2023
ae3a3bb
agg output value
forgxyz Oct 13, 2023
2113b35
Merge pull request #47 from FlipsideCrypto/AN-3924/block_miner_reward…
forgxyz Oct 13, 2023
fc9f0f4
upd fact txs tests
forgxyz Oct 17, 2023
c8b667d
Merge pull request #48 from FlipsideCrypto/adj_gap_test
forgxyz Oct 17, 2023
a70d31e
add incremental predicates to optimize data loads
desmond-hui Oct 20, 2023
ec61375
Merge pull request #49 from FlipsideCrypto/AN-4062/add-incremental-pr…
desmond-hui Oct 20, 2023
ee26001
take output value from txs
forgxyz Oct 23, 2023
ce4c5ef
del output cte
forgxyz Oct 23, 2023
caeca7d
Merge pull request #50 from FlipsideCrypto/AN-4031/output_val
forgxyz Oct 24, 2023
07594c2
upd streamline schedule
forgxyz Oct 30, 2023
fc5500e
Merge pull request #51 from FlipsideCrypto/upd_schedule
forgxyz Oct 30, 2023
d79f73f
changes to bitcoin cluster refs
gronders Oct 31, 2023
166ea4a
change core table ref
gronders Oct 31, 2023
32c79d1
price data from mc, gecko, paprika. todo - confirm gold view cols
forgxyz Nov 1, 2023
4cad0a0
updated clustering process
gronders Nov 1, 2023
0ce8ef2
more deletes for updated process
gronders Nov 1, 2023
ec9fcae
remove null test
gronders Nov 1, 2023
ca2ce3e
changes to group_id name
gronders Nov 1, 2023
c76d81f
core view and gap test on hour
forgxyz Nov 1, 2023
6f3dfda
removing silver tests
gronders Nov 1, 2023
4e02314
incremental fixes
gronders Nov 1, 2023
498fd2b
remove newlines
gronders Nov 1, 2023
8e20a96
upd incr logic
forgxyz Nov 1, 2023
f452545
Merge pull request #52 from FlipsideCrypto/entity_clusters
gronders Nov 1, 2023
38ba0ab
add ez view
forgxyz Nov 1, 2023
c9f8245
keep ohlc in silver, just close in core view
forgxyz Nov 1, 2023
8ee3528
fix coinpaprika open close logic and add silver yml files
forgxyz Nov 2, 2023
de88074
Merge Main
forgxyz Nov 2, 2023
7942df2
upd overview md
forgxyz Nov 2, 2023
f172252
recency test
forgxyz Nov 2, 2023
927d971
db tag
forgxyz Nov 2, 2023
07ecc01
Merge pull request #54 from FlipsideCrypto/An-4128-HISTORICAL-PRICES
forgxyz Nov 2, 2023
dc68765
upd overview
forgxyz Nov 2, 2023
a3d0860
Merge pull request #55 from FlipsideCrypto/quickfix_docs
forgxyz Nov 2, 2023
04f3072
gha to snowflake tasks
eric-laurello Nov 6, 2023
c815c85
Merge pull request #56 from FlipsideCrypto/snowflake-tasks-scheduling
eric-laurello Nov 6, 2023
b5bedfd
incremental bitcoin clustering code
gronders Nov 7, 2023
4940f0e
gha to snowflake tasks
eric-laurello Nov 6, 2023
f09065d
incremental bitcoin clustering code
gronders Nov 7, 2023
0765232
changes to sources
gronders Nov 7, 2023
001abd2
del reward view
forgxyz Nov 7, 2023
2b00c9e
Merge pull request #58 from FlipsideCrypto/deprecate_view
forgxyz Nov 7, 2023
1c0ac20
Merge branch 'main' into incremental_clusters
gronders Nov 9, 2023
926155f
upd price gap test to 24h
forgxyz Nov 9, 2023
4c10449
severity warn
forgxyz Nov 9, 2023
31ef0b9
2 days
forgxyz Nov 9, 2023
dc63f75
Merge pull request #59 from FlipsideCrypto/upd_test
forgxyz Nov 9, 2023
5ea186c
upd tags core vs non core and workflow schedule csv
forgxyz Nov 10, 2023
43062ec
stagger non-core to start after core finishes
forgxyz Nov 10, 2023
be1302a
Merge pull request #60 from FlipsideCrypto/upd_schedule_core_tags
eric-laurello Nov 10, 2023
4c105e9
changes to insert date, full refresh, refs
gronders Nov 10, 2023
9563d6f
updated incrementals and keys
gronders Nov 14, 2023
5cf4b96
keys to full refresh
gronders Nov 14, 2023
ccdc44b
upd ins ts logic, correct new cols, upd tags
forgxyz Nov 14, 2023
ba07317
upd ins ts logic, correct new cols, upd tags
forgxyz Nov 14, 2023
7de56fc
upd inserted timestamp scan range
forgxyz Nov 14, 2023
9678635
clusters label table
gronders Nov 14, 2023
15c69c0
gha workflow and upd label yml
forgxyz Nov 15, 2023
f14f5d0
block full refresh
gronders Nov 15, 2023
b83bd63
Merge pull request #57 from FlipsideCrypto/incremental_clusters
gronders Nov 15, 2023
7ddee7c
remove quotes on var to fix bug
gronders Nov 16, 2023
5220a27
interval -> dateadd
eric-laurello Nov 16, 2023
29b6152
Merge pull request #61 from FlipsideCrypto/bitcoin_cluster_fixes
gronders Nov 16, 2023
9efebb5
added a select
gronders Nov 16, 2023
3f17606
Merge pull request #62 from FlipsideCrypto/select_fix
gronders Nov 16, 2023
470d3db
silver - core
forgxyz Dec 11, 2023
2d837c7
silver - ez, price, label
forgxyz Dec 11, 2023
e11c050
gold - core
forgxyz Dec 11, 2023
8bde434
gold - ez, price
forgxyz Dec 11, 2023
b591fa8
upd coalesce on ts
forgxyz Dec 11, 2023
a13697e
Merge pull request #64 from FlipsideCrypto/AN-4122-COL-ADDS
forgxyz Dec 11, 2023
434e720
sysdate
forgxyz Dec 15, 2023
7731bad
Merge pull request #65 from FlipsideCrypto/quickfix_cluster_new_col
forgxyz Dec 15, 2023
f86b2ea
1.7 update
WHYTEWYLL Jan 26, 2024
1fc7355
update labels
WHYTEWYLL Jan 26, 2024
93d5773
Merge pull request #66 from FlipsideCrypto/AN-4442-flow-dbt1.7
WHYTEWYLL Jan 30, 2024
09e7847
delete colapse, fix description
WHYTEWYLL Jan 30, 2024
3a1cadb
deleted colapse
WHYTEWYLL Jan 30, 2024
ff06f49
Merge pull request #67 from FlipsideCrypto/update_labels
WHYTEWYLL Jan 31, 2024
bc268f4
qualify instead of exclude
forgxyz Feb 14, 2024
0cbae70
add lower() to partition
forgxyz Feb 14, 2024
eb2bf9a
del cluster
forgxyz Feb 14, 2024
8253e01
Merge pull request #68 from FlipsideCrypto/QUICKFIX-LABELS
forgxyz Feb 14, 2024
5da515a
upd incremental
forgxyz Feb 14, 2024
f4bf125
upd to use underlying modified timestamp
forgxyz Feb 14, 2024
1774543
Merge pull request #69 from FlipsideCrypto/quickfix-labels-2
forgxyz Feb 14, 2024
0bf72c3
prior dup labels were deleted, can drop qualify. UPD Tests
forgxyz Feb 14, 2024
d0b4025
upd core
forgxyz Feb 14, 2024
7986d02
Merge pull request #70 from FlipsideCrypto/quickfix-labels-3
forgxyz Feb 14, 2024
191fd6b
bitcoin stats table
WHYTEWYLL Feb 22, 2024
0fcec5a
add decimals
WHYTEWYLL Feb 22, 2024
3ce0681
overview
WHYTEWYLL Feb 22, 2024
e924226
typo
WHYTEWYLL Feb 22, 2024
f877fe9
remove limit use ez prices in gold
eric-laurello Feb 22, 2024
61ed2a1
Merge pull request #71 from FlipsideCrypto/AN-4439/stats
WHYTEWYLL Feb 26, 2024
66c7c8f
fix inc logic
eric-laurello Mar 1, 2024
bb63356
Merge pull request #72 from FlipsideCrypto/fix/stats
eric-laurello Mar 1, 2024
915c977
udf decode hex
forgxyz Mar 7, 2024
b36288d
add decoded coinbase to miner reward table
forgxyz Mar 7, 2024
d8eac23
add UDF_
forgxyz Mar 7, 2024
cd2355b
add to gold view
forgxyz Mar 7, 2024
9619da8
del old return line
forgxyz Mar 7, 2024
3d21031
del coalesce from gov view
forgxyz Mar 7, 2024
9116928
upd gov yml
forgxyz Mar 8, 2024
ee7c05b
Merge pull request #73 from FlipsideCrypto/DECODE-COINBASE
forgxyz Mar 8, 2024
86e661c
where cond
WHYTEWYLL Mar 8, 2024
80e400e
Merge pull request #75 from FlipsideCrypto/genesis_test
WHYTEWYLL Mar 9, 2024
5c6a5bd
upd gha
forgxyz Mar 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
del output cte
  • Loading branch information
forgxyz committed Oct 23, 2023
commit ce4c5ef4bc164e2e15042b9fe8140234f4cbfac4
16 changes: 0 additions & 16 deletions models/silver/core/silver__transactions_final.sql
Original file line number Diff line number Diff line change
@@ -31,22 +31,6 @@ inputs AS (
FROM
{{ ref('silver__inputs_final') }}

{% if is_incremental() %}
WHERE
_inserted_timestamp >= (
SELECT
MAX(_inserted_timestamp) _inserted_timestamp
FROM
{{ this }}
)
{% endif %}
),
outputs AS (
SELECT
*
FROM
{{ ref('silver__outputs') }}

{% if is_incremental() %}
WHERE
_inserted_timestamp >= (