Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eng 824/auctioneer #1839

Draft
wants to merge 74 commits into
base: itamarreif/auctioneer/eng-824
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
43349db
chore(proto): move optimistic block protos to v1 (#1707)
itamarreif Nov 19, 2024
942ffc9
fix: update audit ignore list (#1805)
Fraser999 Nov 19, 2024
9b6ecc7
chore(docs): updates to data verification and inclusion proof specs (…
noot Nov 19, 2024
3830a3d
fix(ci): remove unnecessary checks from ibc-tests (#1813)
quasystaty1 Nov 19, 2024
471ee91
feat(cli): allowed fee-assets query (#1816)
quasystaty1 Nov 19, 2024
66704f2
fix(ci, core): delete unused generated code, generate overlooked code
SuperFluffy Nov 19, 2024
837ae4e
set msrv to 1.81, remove bin entry
SuperFluffy Nov 20, 2024
f00c9fb
remove unused dependencies
SuperFluffy Nov 21, 2024
15fe94a
clean up logging in main, ensure root spans exist
SuperFluffy Nov 21, 2024
b9ea05e
remove unused associated const
SuperFluffy Nov 21, 2024
b4c0577
rename auctioneer::inner::Auctioneer -> auctioneer::Inner and fix som…
SuperFluffy Nov 21, 2024
9a33b04
have bundle stream match other stream impls
SuperFluffy Nov 21, 2024
5e63066
import traits as _
SuperFluffy Nov 21, 2024
93a2d1e
remove explicit retries for opening streams; rely on tonic's reconnec…
SuperFluffy Nov 22, 2024
92b2223
core: impl Protobuf for FilteredSequencerBlock
SuperFluffy Nov 22, 2024
3d47389
run sequencer streams off of a single channel
SuperFluffy Nov 22, 2024
195f25f
remove needless pin projection
SuperFluffy Nov 22, 2024
df1eae6
run rollup streams off a single channel
SuperFluffy Nov 22, 2024
1dd405f
flatten optimistic executioner into auctioneer
SuperFluffy Nov 24, 2024
9ac1596
have auctioneer shutdown events outside it's long running task
SuperFluffy Nov 24, 2024
2ef11e0
don't short circuit in the auctioneer select loop
SuperFluffy Nov 24, 2024
5a69451
shutdown all still active auctions
SuperFluffy Nov 24, 2024
dcc40dd
remember to shutdown all constituent tasks during shutdown
SuperFluffy Nov 24, 2024
bd7ceef
move running and starting runstates into their own modules
SuperFluffy Nov 24, 2024
3a0355a
remove unnecessary exclusive ref
SuperFluffy Nov 25, 2024
172bf53
run all startup initialization futures concurrently
SuperFluffy Nov 25, 2024
bf83e2b
auctioneer is unbiased toward all events
SuperFluffy Nov 25, 2024
2501b8f
treat handling of bundle events like all other events
SuperFluffy Nov 25, 2024
453aeda
use the channel singleton to pass a sequencer service client to aucti…
SuperFluffy Nov 25, 2024
c857188
use the same rollup ID everywhere
SuperFluffy Nov 25, 2024
b78e186
make auctioneer run states private to the inner business logic impl
SuperFluffy Nov 25, 2024
51e4382
impl display for auction IDs and use that instead of explicit base64
SuperFluffy Nov 26, 2024
c18d87c
abort auctions using its cancellation token
SuperFluffy Nov 26, 2024
ca5281e
move tracking of current blocks into the auction manager
SuperFluffy Nov 27, 2024
7a0fd7c
replace the joinset of live auctions by a single auction task
SuperFluffy Nov 27, 2024
66b7f2d
submit auction winners without a retry
SuperFluffy Nov 27, 2024
17bb3e7
fetch pending nonces on a schedule and uncouple from auction winner s…
SuperFluffy Nov 28, 2024
951f610
include a restart mechanism for all streams after they have concluded
SuperFluffy Nov 28, 2024
d27c9f1
remove auctioneer runstates
SuperFluffy Nov 29, 2024
aa02a77
move auctions, manager into the auctioneer module
SuperFluffy Nov 29, 2024
19dbdae
track the running auction inside auctioneer
SuperFluffy Nov 29, 2024
9a3a1b7
turn auction::Running into a future
SuperFluffy Nov 29, 2024
444e655
clippy and toml formatting
SuperFluffy Nov 29, 2024
3116bd2
feat(core)!: provide a newtype for block hashes
SuperFluffy Nov 29, 2024
752fc20
update auctioneer to make use of the block hash type
SuperFluffy Nov 29, 2024
1387377
re-instrument all handlers in primary auctioneer event loop
SuperFluffy Nov 30, 2024
c667efc
track result of cancelled auctions
SuperFluffy Nov 30, 2024
8ef55fc
starting new auctions doesn't need cloned blocks
SuperFluffy Nov 30, 2024
c33ba78
auction factory does not need mutable access or instrumentation
SuperFluffy Nov 30, 2024
d4b29fc
flatten all auction construction directly into the factory
SuperFluffy Nov 30, 2024
8d9f490
rename Auction to Worker, Running to Auction
SuperFluffy Nov 30, 2024
07aebdf
swap the module relationship between auction future and worker task
SuperFluffy Nov 30, 2024
0deb18e
instrument all auction handlers, separate auction ID from block hashes
SuperFluffy Dec 2, 2024
e16952b
remove connect timeouts
SuperFluffy Dec 2, 2024
37568f8
instrument grpc at the request/response level
SuperFluffy Dec 2, 2024
60e9da1
emit events for failed stream connection requests; also bring back co…
SuperFluffy Dec 2, 2024
946fdbd
REMOVE FROM PR: emit span events in tracing subscriber
SuperFluffy Dec 2, 2024
a940b2e
add astria-auctioneer to shortnames
SuperFluffy Dec 3, 2024
e5e85bb
Merge branch 'superfluffy/delete-old-generated-code' into ENG-824/auc…
SuperFluffy Dec 3, 2024
739746f
update all imports to use optimistic, not optimisticblock
SuperFluffy Dec 3, 2024
78fa646
unset running auction after completion
SuperFluffy Dec 4, 2024
b074a8b
return the connection error in restarting stream
SuperFluffy Dec 4, 2024
9c6f926
fix allocation rule never taking bids with zero-bids
SuperFluffy Dec 4, 2024
070a9e7
use Arcs when passing bundles to running auction; instrument first pr…
SuperFluffy Dec 4, 2024
6bff3e5
make the auction bundle channel unbounded
SuperFluffy Dec 4, 2024
5d66615
replace the mpsc command channel by two oneshot channels to trigger a…
SuperFluffy Dec 4, 2024
f6ee3a5
separate the trigger to start the auction timer from the bids
SuperFluffy Dec 4, 2024
2ac6504
small refactor: start_processing_bids -> start_bids
SuperFluffy Dec 4, 2024
0a8a595
add some todos to the auction
SuperFluffy Dec 4, 2024
c4ea203
not receiving winning bids does not constitute an error for auctionee…
SuperFluffy Dec 4, 2024
82e401a
skip_all not only skip(self)
SuperFluffy Dec 4, 2024
779b36f
cancel auctions instead of aborting them
SuperFluffy Dec 11, 2024
e17fe69
some clippy changes to auctioneer
SuperFluffy Dec 11, 2024
2edf112
compare bundle parent hash against executed block hash, not parent of…
SuperFluffy Dec 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .cargo/audit.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,10 @@ ignore = [
# `proc-macro-error` is Unmaintained. It is a transient dependency of borsh crates, so cannot
# easily be replaced.
"RUSTSEC-2024-0370",
# `instant` is Unmaintained. It is a transient dependency of `isahc`, `wiremock` and `ethers`, so
# cannot easily be replaced.
"RUSTSEC-2024-0384",
# `derivative` is Unmaintained. It is a transient dependency of many crates including several
# penumbra ones, so cannot easily be replaced.
"RUSTSEC-2024-0388",
]
59 changes: 36 additions & 23 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 5 additions & 37 deletions charts/ibc-test.just
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,7 @@ run-without-native tag=defaultTag:
# Execute the transfer from Celestia to the Rollup
just ibc-test _do-ibc-transfer {{defaultNamespace}} {{sequencer_sudo_address}}

# Add transfer/channel-0/utia as fee-asset
docker run --rm --network host $ASTRIA_CLI_IMAGE sequencer sudo fee-asset add --private-key {{sequencer_sudo_pkey}} --asset transfer/channel-0/utia --sequencer-url {{sequencer_rpc_url}} --sequencer.chain-id {{sequencer_chain_id}}
# check that sequencer balance updated correctly

EXPECTED_BALANCE=$(echo "1 * {{transfer_amount}}" | bc)
for i in {1..50}
do
Expand All @@ -110,6 +107,11 @@ run-without-native tag=defaultTag:
exit 1
fi

# Add transfer/channel-0/utia as allowed fee-asset
docker run --rm --network host $ASTRIA_CLI_IMAGE sequencer sudo fee-asset add --private-key {{sequencer_sudo_pkey}} --asset transfer/channel-0/utia --sequencer-url {{sequencer_rpc_url}} --sequencer.chain-id {{sequencer_chain_id}}

# TODO: query allowd fee asset verifying succefull addition

[no-cd]
run tag=defaultTag:
#!/usr/bin/env bash
Expand Down Expand Up @@ -149,23 +151,6 @@ run tag=defaultTag:
# Execute the transfer from Celstia to sequencer with compat address
just ibc-test _do-ibc-transfer {{defaultNamespace}} {{compat_address}}

# check that celestia balance updated correctly
for i in {1..50}
do
current_celestia_balance=$(just ibc-test get-celestia-balance)
echo "check $i, balance: $current_celestia_balance, expected: $expected_celestia_balance"
if (( $expected_celestia_balance == $current_celestia_balance )); then
expected_celestia_balance_found="1"
break
else
sleep 1
fi
done
if [[ -z $expected_celestia_balance_found ]]; then
echo "expected celestia balance was not found after withdraw; IBC transfer from Celestia to the Rollup failed"
exit 1
fi

# check that sequencer balance updated correctly
ASTRIA_CLI_IMAGE="{{cli_image}}{{ if tag != '' { replace(':#', '#', tag) } else { '' } }}"
EXPECTED_BALANCE=$(echo "1 * {{transfer_amount}}" | bc)
Expand Down Expand Up @@ -283,23 +268,6 @@ run-timeout tag=defaultTag:
# Execute the transfer from Celstia to sequencer with compat address
just ibc-test _do-ibc-transfer {{defaultNamespace}} {{compat_address}}

# check that celestia balance updated correctly
for i in {1..50}
do
current_celestia_balance=$(just ibc-test get-celestia-balance)
echo "check $i, balance: $current_celestia_balance, expected: $expected_celestia_balance"
if (( $expected_celestia_balance == $current_celestia_balance )); then
expected_celestia_balance_found="1"
break
else
sleep 1
fi
done
if [[ -z $expected_celestia_balance_found ]]; then
echo "expected celestia balance was not found after withdraw; IBC transfer from Celestia to the Rollup failed"
exit 1
fi

# check that sequencer balance updated correctly
ASTRIA_CLI_IMAGE="{{cli_image}}{{ if tag != '' { replace(':#', '#', tag) } else { '' } }}"
EXPECTED_BALANCE=$(echo "1 * {{transfer_amount}}" | bc)
Expand Down
28 changes: 10 additions & 18 deletions crates/astria-auctioneer/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,12 @@
name = "astria-auctioneer"
version = "0.0.1"
edition = "2021"
rust-version = "1.76"
rust-version = "1.81"
license = "MIT OR Apache-2.0"
readme = "README.md"
repository = "https://github.com/astriaorg/astria"
homepage = "https://astria.org"

[[bin]]
name = "astria-auctioneer"

[dependencies]
astria-build-info = { path = "../astria-build-info", features = ["runtime"] }
astria-core = { path = "../astria-core", features = ["serde", "client"] }
Expand All @@ -21,20 +18,16 @@ telemetry = { package = "astria-telemetry", path = "../astria-telemetry", featur
"display",
] }

async-trait = { workspace = true }
axum = { workspace = true }
base64 = { workspace = true }
bytes = { workspace = true }
futures = { workspace = true }
hex = { workspace = true }
humantime = { workspace = true }
itertools = { workspace = true }
pbjson-types = { workspace = true }
pin-project-lite = { workspace = true }
prost = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
sha2 = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = [
"macros",
"rt-multi-thread",
Expand All @@ -46,20 +39,19 @@ tokio-util = { workspace = true, features = ["rt"] }
tracing = { workspace = true, features = ["attributes"] }
tryhard = { workspace = true }
tonic = { workspace = true }
tokio-stream = { workspace = true, features = ["net"] }
tokio-stream = { workspace = true, features = ["sync"] }

pin-project-lite = "0.2.15"
tower-http = { version = "0.4.4", features = ["map-response-body", "trace"] }
tower = { version = "0.5.1", features = ["util"] }
hyper.workspace = true
http = "0.2.11"
http-body = "0.4.5"

[dev-dependencies]
astria-core = { path = "../astria-core", features = ["client"] }
config = { package = "astria-config", path = "../astria-config", features = [
"tests",
] }
insta = { workspace = true, features = ["json"] }
tempfile = { workspace = true }
test_utils = { package = "astria-test-utils", path = "../astria-test-utils", features = [
"geth",
] }
tokio-test = { workspace = true }
wiremock = { workspace = true }

[build-dependencies]
astria-build-info = { path = "../astria-build-info", features = ["build"] }
32 changes: 0 additions & 32 deletions crates/astria-auctioneer/src/auction/allocation_rule.rs

This file was deleted.

Loading
Loading