8000 8000
Skip to content

[core] Shared slot count factory for codegen sized listener storage - #18057

Merged
bdraco merged 12 commits into
devfrom
ble_listener_slot_factory
Aug 4, 2026
Merged

[core] Shared slot count factory for codegen sized listener storage#18057
bdraco merged 12 commits into
devfrom
ble_listener_slot_factory

Conversation

@bdraco
@bdraco bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member

What does this implement/fix?

Lifts the duplicated slot count pattern behind the codegen sized StaticVector listener storage into one shared factory. cg.slot_counter(define) returns a single request function; a consumer's to_code calls it once per slot, and the first request schedules a FINAL priority job that emits the define with the requested count. No requests, no define, so the guarded storage compiles out. There is no emit job for callers to wire up or forget.

The factory lives in esphome/cpp_helpers.py (exported through codegen) because nothing in it is BLE specific; bk72xx_ble and ln882h_ble drop their hand rolled copies and call it, keeping their public request_scan_listener_slot names and define names unchanged. ble_device_base uses the same factory for the neutral ESPHOME_BLE_DEVICE_BASE_LISTENER_COUNT, replacing the per tracker emit jobs in bk72xx_ble_tracker and rp2_ble_tracker.

wifi and ethernet carry the same hand rolled count machinery with a paired USE_* define; migrating them is left as a follow up since it also touches their C++ guards.

Codegen tests build real tracker and controller only configs and assert on the emitted defines; unit tests pin the factory itself, including a slot requested while FINAL jobs are already draining.

Chained on #18056.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • New developer-facing feature (adds functionality for component developers; no end-user configuration change)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected) — policy
  • Developer breaking change (an API change that could break external components) — policy
  • Undocumented C++ API change (removal or change of undocumented public methods that lambda users may depend on) — policy
  • Code quality improvements to existing code or addition of tests
  • Other

Related issue or feature (if applicable):

Pull request in esphome.io with documentation (if applicable):

Pull request in developers.esphome.io with developer documentation (if applicable):

Test Environment

  • ESP32
  • ESP32 IDF
  • ESP8266
  • RP2040/RP2350
  • BK72xx
  • RTL87xx
  • LN882x
  • nRF52840

Example entry for config.yaml:

# Example config.yaml

bk72xx_ble_tracker:

Checklist:

  • The code change is tested and works locally.
  • Tests have been added to verify that the new code works (under tests/ folder).

If user exposed functionality or configuration variables are added/changed:

@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

@bluetoothbot review

1 similar comment
@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

@bluetoothbot review

@esphome
esphome Bot commented Aug 4, 2026
Copy link
Copy Markdown
Contributor

To use the changes in this PR:

# Clone the repository:
git clone https://github.com/esphome/esphome
cd esphome

# Checkout the PR branch:
git fetch origin pull/18057/head:ble_listener_slot_factory
git checkout ble_listener_slot_factory

# Install the development version:
script/setup

# Activate the development version:
source venv/bin/activate

Now you can run esphome as usual to test the changes in this PR.


(Added by the PR bot)

@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

@bluetoothbot review

1 similar comment
@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

@bluetoothbot review

@bluetoothbot
bluetoothbot commented Aug 4, 2026
Copy link
Copy Markdown
Contributor

Previous review — superseded by a newer review below.

@bluetoothbot bluetoothbot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

No blocking issues found — ready to merge.

@codecov
codecov Bot commented Aug 4, 2026
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.10%. Comparing base (c6a3784) to head (69c4d72).
⚠️ Report is 6 commits behind head on dev.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              dev   #18057      +/-   ##
==========================================
+ Coverage   86.91%   87.10%   +0.18%     
==========================================
  Files          61       62       +1     
  Lines       14534    14599      +65     
  Branches     2189     2200      +11     
==========================================
+ Hits        12632    12716      +84     
+ Misses       1593     1575      -18     
+ Partials      309      308       -1     
Files with missing lines Coverage Δ
esphome/codegen.py 100.00% <ø> (ø)
esphome/cpp_helpers.py 88.48% <100.00%> (+2.25%) ⬆️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

@bluetoothbot review

@bdraco
bdraco force-pushed the ble_listener_slot_factory branch from 5e62c66 to b0ef638 Compare August 4, 2026 05:16
@github-actions
github-actions Bot commented Aug 4, 2026
Copy link
Copy Markdown
Contributor

Memory Impact Analysis

Components: esp32_ble_tracker
Platform: esp32-idf

Metric Target Branch This PR Change
RAM 66,380 bytes 66,380 bytes ➡️ +0 bytes (0.00%)
Flash 1,072,787 bytes 1,072,787 bytes ➡️ +0 bytes (0.00%)

Note: This analysis measures static RAM and Flash usage only (compile-time allocation).
Dynamic memory (heap) cannot be measured automatically.
⚠️ You must test this PR on a real device to measure free heap and ensure no runtime memory issues.

This analysis runs automatically when components change. Memory usage is measured from a representative test configuration.

@bdraco
bdraco force-pushed the ble_listener_slot_factory branch from b0ef638 to 1047b01 Compare August 4, 2026 05:50
@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

@bluetoothbot review

@bluetoothbot
bluetoothbot commented Aug 4, 2026
Copy link
Copy Markdown
Contributor

Previous review — superseded by a newer review below.

@bluetoothbot bluetoothbot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

No blocking issues found — ready to merge.

@bluetoothbot
bluetoothbot commented Aug 4, 2026
Copy link
Copy Markdown
Contributor

Previous review — superseded by a newer review below.

@bluetoothbot bluetoothbot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

No blocking issues found — ready to merge.

@bdraco
bdraco force-pushed the ble_listener_slot_factory branch from 1047b01 to 9b37520 Compare August 4, 2026 06:12
@bluetoothbot
bluetoothbot commented Aug 4, 2026
Copy link
Copy Markdown
Contributor

Previous review — superseded by a newer review below.

@bluetoothbot bluetoothbot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

No blocking issues found — ready to merge.

@bdraco
bdraco marked this pull request as ready for review August 4, 2026 17:52
@bdraco
bdraco requested a review from a team as a code owner August 4, 2026 17:52
Copilot AI lite review requested due to automatic review settings August 4, 2026 17:52
@esphome
esphome Bot commented Aug 4, 2026
Copy link
Copy Markdown
Contributor

👋 Hi there! This PR modifies 8 file(s) with codeowners.

@Bl00d-B0b, @esphome/core - As codeowner(s) of the affected files, your review would be appreciated! 🙏

Note: Automatic review request may have failed, but you're still welcome to review.

Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR centralizes the “count slots during codegen, then emit a #define at FINAL priority” pattern into a shared cg.slot_counter(DEFINE) factory, reducing duplicated listener-count plumbing across BLE components while preserving existing define names and public request function names.

Changes:

  • Added slot_counter() / get_slot_count() to esphome/cpp_helpers.py and re-exported them via esphome/codegen.py.
  • Migrated LibreTiny BLE controllers/trackers and ble_device_base away from bespoke CORE.data + FINAL emit jobs to the shared factory.
  • Added unit + component tests validating define emission behavior (including FINAL-time request edge cases) and end-to-end behavior for BK72xx configs.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit_tests/test_cpp_helpers.py Adds unit tests covering slot_counter behavior (emit, no-emit, FINAL-time request edge cases, late-request error).
tests/component_tests/helpers.py Adds a helper to read rendered CORE define values for component tests.
tests/component_tests/ble_device_base/test_slot_counter.py End-to-end component tests asserting listener-count define emission/absence via real configs and a mock BLEHub path.
tests/component_tests/ble_device_base/config/bk72xx_tracker.yaml Minimal tracker config used to assert count define is emitted.
tests/component_tests/ble_device_base/config/bk72xx_controller_only.yaml Minimal controller-only config used to assert count define is not emitted.
esphome/cpp_helpers.py Implements the shared slot_counter factory and get_slot_count backed by CORE.data.
esphome/components/rp2_ble_tracker/init.py Removes the tracker-local FINAL emit job in favor of the shared slot-counter mechanism.
esphome/components/ln882h_ble/init.py Replaces hand-rolled scan listener counting/emission with cg.slot_counter(...).
esphome/components/esp32_ble_tracker/init.py Switches neutral listener feature-coupling to rely on cg.get_slot_count(...) (define emission handled elsewhere).
esphome/components/ble_device_base/init.py Converts listener counting to cg.slot_counter(...) and defines a shared LISTENER_COUNT_DEFINE.
esphome/components/bk72xx_ble/init.py Replaces hand-rolled scan listener counting/emission with cg.slot_counter(...).
esphome/components/bk72xx_ble_tracker/init.py Removes the tracker-local FINAL emit job in favor of the shared slot-counter mechanism.
esphome/codegen.py Re-exports slot_counter and get_slot_count for component use via cg.*.

Comment thread tests/component_tests/helpers.py Outdated
@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

@bluetoothbot review

@bluetoothbot
Copy link
Copy Markdown
Contributor

PR Review — [core] Shared slot count factory for codegen sized listener storage

Approve — re-verification against the dev baseline turns up nothing new, and the one open comment thread is closed. Merge-ready.

What holds up under checking this round:

  • Callers fully migrated, no dead state. get_listener_count() / KEY_BLE_LISTENER_COUNT had exactly three call sites on dev (esp32_ble_tracker:311, bk72xx_ble_tracker:61, rp2_ble_tracker:62) and the diff removes all three. I also checked each trimmed module for leftover CORE. / CoroPriority use after the import line is dropped — rp2_ble_tracker, bk72xx_ble_tracker, bk72xx_ble, ln882h_ble, and ble_device_base each used those names only on the removed lines, so no dangling references and no unused imports.

  • Factory is safe across runs. slot_counter() is invoked at module import (request_scan_listener_slot = cg.slot_counter(...)), but the closure captures only the define string — all state is fetched lazily from CORE.data, which clears between runs. That is exactly the pattern CLAUDE.md asks for instead of module-level mutable globals.

  • Emit happens once. The count == 1 guard sits on shared CORE.data state, not on the per-factory closure, so two modules calling slot_counter() on the same define still schedule a single job. That actually removes a latent hazard: Define.__hash__ is (name, value) and value is a fresh IntLiteral per call, so a duplicate emit would have produced two set entries with the same name rather than deduplicating.

  • Ordering is unchanged. _add_ble_features is still CoroPriority.FINAL and now only reads cg.get_slot_count() for the ESP_BT_DEVICE feature coupling; the counts it reads are filled during 7292 to_code, so both jobs sitting at FINAL creates no dependency. CoroPriority.FINAL is -1000 and blocked tasks lose one point per yield (esphome/coroutine.py:358), so the post-emission ValueError cannot fire on a build that is not already deadlocking.

  • Defines and guards line up. All three names (ESPHOME_BLE_DEVICE_BASE_LISTENER_COUNT, BK72XX_BLE_SCAN_LISTENER_COUNT, LN882H_BLE_SCAN_LISTENER_COUNT) are present in esphome/core/defines.h, and every C++ use is #ifdef-guarded, so the no-request case genuinely compiles the StaticVector storage out.

  • Tests are isolated and the new fixture directory is harmless. Both test trees have an autouse CORE.reset(); each test uses a distinct define name. Adding config/ under tests/component_tests/ble_device_base/ flips the autouse config_path fixture to point at that directory, and the pre-existing test_scan_parameter_validation.py is pure schema validation that never reads CORE.config_path.

  • The new YAML fixtures validate. I ran esphome config locally on both configs (reconstructed on dev, where the component code is unchanged): bk72xx_tracker.yaml and bk72xx_controller_only.yaml both report Configuration is valid!. generic-bk7252 matches the board the repo's existing bk72xx_ble / bk72xx_ble_tracker validate-only tests already build on.

  • Docstring comment is genuinely resolved. get_define_value now states plainly that a value-less define renders as the string "None" while an absent one returns the None object, which is what the implementation does.

  • No blocking or important findings.

  • One standing observation, not a finding: ln882h_ble.request_scan_listener_slot still has no in-tree caller (no ln882h tracker exists yet). That predates this PR — the hand-rolled version had the same shape — and this change only moves it onto the factory.

  • Caveat on verification: the PR branch is not checked out in this environment, so the above is from the diff plus the dev baseline plus the config validation runs, not from executing the new test files.



Checklist

  • Refactor is behaviour-preserving (define names, counts, FINAL ordering)
  • No stale callers of the removed get_listener_count / KEY_BLE_LISTENER_COUNT
  • No leftover CORE/CoroPriority use after the trimmed imports
  • Factory holds no cross-run state (import-time closure captures only the define name)
  • Emit job scheduled exactly once; post-emission request raises instead of undercounting
  • Shared factory lives in a neutral module; no undeclared component coupling
  • cg.add_define entries mirrored in esphome/core/defines.h and C++ uses are #ifdef-guarded
  • New/changed branches covered by tests (rewired register_ble_device call site pinned)
  • Test isolation — autouse CORE.reset in both trees; distinct define names per test
  • New config/ directory does not disturb the pre-existing test in that folder
  • New test YAML validates (esphome config run locally on both files)
  • No hardcoded secrets or unsafe operations
  • Diff matches PR description; no scope creep

Automated review by Kōan (Claude) HEAD=69c4d72 4 min 41s

@bluetoothbot bluetoothbot left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

No blocking issues found — ready to merge.

@bdraco
bdraco commented Aug 4, 2026
Copy link
Copy Markdown
Member Author

Thanks

@bdraco
bdraco enabled auto-merge (squash) August 4, 2026 19:39
@bdraco
bdraco merged commit 68640f8 into dev Aug 4, 2026
38 checks passed
@bdraco
bdraco deleted the ble_listener_slot_factory branch August 4, 2026 19:46
bdraco added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 4, 2026


The shared cg.slot_counter factory now owns emitting
ESPHOME_BLE_DEVICE_BASE_LISTENER_COUNT from ble_device_base itself, and
get_listener_count() no longer exists, so the tracker's local FINAL
coroutine and its CORE.add_job call are gone (same migration esphome#18057
applied to the bk72xx and rp2 trackers).
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 4, 2026
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 4, 2026
Rebased onto esphome#18057: listener counting via the core slot-counter factory; tracker carries no counting code.
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 4, 2026
Rebased onto esphome#18057: listener counting via the core slot-counter factory; tracker carries no counting code.
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 4, 2026
Rebased onto esphome#18057: listener counting via the core slot-counter factory; tracker carries no counting code.
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 5, 2026
Rebased onto esphome#18057: listener counting via the core slot-counter factory; tracker carries no counting code.
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 5, 2026
Rebased onto esphome#18057: listener counting via the core slot-counter factory; tracker carries no counting code.
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 5, 2026
Rebased onto esphome#18057: listener counting via the core slot-counter factory; tracker carries no counting code.
Bl00d-B0b added a commit to Bl00d-B0b/esphome that referenced this pull request Aug 5, 2026
Rebased onto esphome#18057: listener counting via the core slot-counter factory; tracker carries no counting code.
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 6, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

0