Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Consistent trie sort order #5615

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Consistent trie sort order #5615

wants to merge 3 commits into from

Conversation

sffc
Copy link
Member

@sffc sffc commented Sep 29, 2024

@sffc sffc mentioned this pull request Sep 30, 2024
Draft
@sffc
Copy link
Member Author

sffc commented Nov 24, 2024

Got around to finishing this up so I can run benchmarks.

get/basic/SimpleAscii   time:   [105.14 ns 105.18 ns 105.21 ns]
                        change: [+6.8413% +7.1727% +7.4843%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  3 (3.00%) high severe
get/basic/PerfectHash   time:   [90.225 ns 90.258 ns 90.296 ns]
                        change: [+1.7558% +1.9749% +2.1696%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  1 (1.00%) high mild
  3 (3.00%) high severe
get/basic/ExtendedCapacity
                        time:   [89.067 ns 89.088 ns 89.110 ns]
                        change: [+1.3389% +1.6927% +1.9896%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) low severe
  1 (1.00%) low mild
  2 (2.00%) high mild
  3 (3.00%) high severe
get/basic/ZeroMap/usize time:   [95.301 ns 95.618 ns 95.992 ns]
                        change: [-1.5754% -1.2333% -0.8898%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 18 outliers among 100 measurements (18.00%)
  1 (1.00%) high mild
  17 (17.00%) high severe
get/basic/ZeroMap/u8    time:   [78.991 ns 79.087 ns 79.177 ns]
                        change: [-2.6764% -2.4349% -2.1911%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
get/basic/HashMap       time:   [90.831 ns 91.230 ns 91.755 ns]
                        change: [+0.3983% +0.7403% +1.1134%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  5 (5.00%) high mild
  9 (9.00%) high severe
get/basic/ZeroHashMap/usize
                        time:   [158.72 ns 158.81 ns 158.90 ns]
                        change: [-1.5309% -0.9817% -0.5194%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
get/basic/ZeroHashMap/u8
                        time:   [124.12 ns 124.26 ns 124.40 ns]
                        change: [-0.1612% +0.0956% +0.3490%] (p = 0.46 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

get/subtags_10pct/SimpleAscii
                        time:   [5.9496 µs 5.9590 µs 5.9659 µs]
                        change: [+28.580% +28.795% +28.984%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  2 (2.00%) low severe
  7 (7.00%) low mild
  5 (5.00%) high severe
get/subtags_10pct/PerfectHash
                        time:   [5.6711 µs 5.6753 µs 5.6801 µs]
                        change: [+8.0707% +8.2644% +8.4582%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  3 (3.00%) high severe
get/subtags_10pct/ExtendedCapacity
                        time:   [5.9406 µs 5.9572 µs 5.9828 µs]
                        change: [+8.4296% +8.6755% +8.9396%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
get/subtags_10pct/ZeroMap/usize
                        time:   [7.3466 µs 7.3562 µs 7.3694 µs]
                        change: [-2.8745% -2.2913% -1.7970%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
get/subtags_10pct/ZeroMap/u8
                        time:   [6.5400 µs 6.5556 µs 6.5755 µs]
                        change: [+0.1971% +0.6800% +1.1398%] (p = 0.00 < 0.05)
                        Change within noise threshold.
get/subtags_10pct/HashMap
                        time:   [2.6496 µs 2.6599 µs 2.6695 µs]
                        change: [+1.2850% +1.7402% +2.1718%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
get/subtags_10pct/ZeroHashMap/usize
                        time:   [4.5215 µs 4.5294 µs 4.5383 µs]
                        change: [+3.3001% +3.6223% +3.9405%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  8 (8.00%) high mild
  1 (1.00%) high severe
get/subtags_10pct/ZeroHashMap/u8
                        time:   [3.4526 µs 3.4569 µs 3.4620 µs]
                        change: [-0.2127% +0.0074% +0.2560%] (p = 0.95 > 0.05)
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) high mild
  7 (7.00%) high severe

get/subtags_full/SimpleAscii
                        time:   [114.89 µs 114.98 µs 115.07 µs]
                        change: [+27.052% +27.684% +28.347%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  4 (4.00%) high mild
  11 (11.00%) high severe
get/subtags_full/PerfectHash
                        time:   [88.376 µs 88.452 µs 88.552 µs]
                        change: [+4.9593% +5.2156% +5.5133%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe
get/subtags_full/ExtendedCapacity
                        time:   [92.270 µs 92.347 µs 92.462 µs]
                        change: [+5.0040% +5.3257% +5.5932%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
get/subtags_full/ZeroMap/usize
                        time:   [135.66 µs 135.73 µs 135.80 µs]
                        change: [-0.4141% -0.2089% -0.0154%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
get/subtags_full/ZeroMap/u8
                        time:   [129.85 µs 130.26 µs 130.79 µs]
                        change: [-0.5113% -0.2881% -0.0444%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
get/subtags_full/HashMap
                        time:   [25.779 µs 25.801 µs 25.822 µs]
                        change: [-0.5017% -0.2349% -0.0021%] (p = 0.06 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe
get/subtags_full/ZeroHashMap/usize
                        time:   [42.574 µs 42.625 µs 42.678 µs]
                        change: [-0.1563% -0.0269% +0.1045%] (p = 0.70 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
get/subtags_full/ZeroHashMap/u8
                        time:   [34.343 µs 34.374 µs 34.427 µs]
                        change: [-0.5629% -0.4074% -0.2451%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

Main notable thing there is the +28% on SimpleAscii getter performance. So, this is probably not a good idea as the default behavior.

However, modeling ZeroTrie as essentially 3 different branching functions might not be a bad idea:

  1. Regular binary search
  2. Modified binary search with optional case-insensitive lookup
  3. Perfect hash function

@sffc
Copy link
Member Author

sffc commented Nov 25, 2024

Here's how IgnoreCase stacks up, by the way:

get/basic/IgnoreCase    time:   [101.88 ns 102.48 ns 103.33 ns]
Found 30 outliers among 100 measurements (30.00%)
  18 (18.00%) low severe
  12 (12.00%) high severe

get/subtags_10pct/IgnoreCase
                        time:   [6.4376 µs 6.4419 µs 6.4473 µs]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe

(it returns a MixedCase error on subtags_full)

So it seems that the comparison function in this PR is slightly faster than the comparison function in IgnoreCase. (5.8089 µs vs 6.4419 µs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sffc