8000
Skip to content

[core] Convert components, devices, and areas vectors to static allocation - #10020

Merged
jesserockz merged 2 commits into
devfrom
static_comp_areas
Aug 4, 2025
Merged

[core] Convert components, devices, and areas vectors to static allocation#10020
jesserockz merged 2 commits into
devfrom
static_comp_areas

Conversation

@bdraco
@bdraco bdraco commented Aug 2, 2025 •
edited
Loading
Copy link
Copy Markdown
Member

What does this implement/fix?

This PR extends the static allocation optimization from #10018 to the components_, devices_, and areas_ vectors in the Application class. These vectors are now allocated with a fixed size at compile time based on the actual counts from the YAML configuration, eliminating heap allocations and fragmentation.

Key improvements:

  • Converts std::vector to StaticVector for deterministic memory usage
  • Eliminates heap allocation overhead and fragmentation
  • Provides compile-time memory allocation based on YAML configuration
  • Reduces flash usage by removing dynamic allocation code

Memory impact:

  • Small configs: ~264 bytes flash saved, minimal net RAM change
  • Large configs: ~924 bytes flash saved, significant heap memory freed
  • All configs: Eliminates heap fragmentation and provides predictable memory usage

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Code quality improvements to existing code or addition of tests
  • Other

Related issue or feature (if applicable):

Pull request in esphome-docs with documentation (if applicable):

  • N/A (internal optimization, no user-facing changes)

Test Environment

  • ESP32
  • ESP32 IDF
  • ESP8266
  • RP2040
  • BK72xx
  • RTL87xx
  • nRF52840

Example entry for config.yaml:

# No configuration changes required - this is an internal optimization
# The defines are automatically generated based on your existing configuration

# Example showing areas and devices that benefit from this optimization:
esphome:
  name: my-device
  area: Living Room

area:
  - id: bedroom
    name: "Bedroom"
  - id: kitchen
    name: "Kitchen"

device:
  - id: lamp1
    name: "Bedroom Lamp"
    area_id: bedroom
  - id: lamp2
    name: "Kitchen Light"
    area_id: kitchen

Checklist:

  • The code change is tested and works locally.
  • Tests have been added to verify that the new code works (under tests/ folder).

If user exposed functionality or configuration variables are added/changed:

Implementation Details

Changes Made:

  1. Added compile-time defines in core/config.py:

    • ESPHOME_COMPONENT_COUNT - based on len(CORE.component_ids)
    • ESPHOME_DEVICE_COUNT - based on devices in YAML
    • ESPHOME_AREA_COUNT - based on areas in YAML

  2. Enhanced StaticVector in core/helpers.h:

    • Added empty() method
    • Added reverse iterator support (rbegin(), rend())
    • Added <iterator> include for std::reverse_iterator

  3. Converted vectors in core/application.h:

    • std::vector<Component *> → StaticVector<Component *, ESPHOME_COMPONENT_COUNT>
    • std::vector<Device *> → StaticVector<Device *, ESPHOME_DEVICE_COUNT>
    • std::vector<Area *> → StaticVector<Area *, ESPHOME_AREA_COUNT>

  4. Updated member ordering in application.h:

    • Reordered members by size to minimize padding
    • Moved StaticVectors to the end (largest members with inline arrays)
    • Preserved important comments (partitioned vector design)

  5. Added default counts in core/defines.h for static analysis:

    • ESPHOME_COMPONENT_COUNT 50
    • ESPHOME_DEVICE_COUNT 10
    • ESPHOME_AREA_COUNT 10

Testing Results:

Small Device (ESP32-IDF):

  • Flash: 517,226 → 516,962 bytes (-264 bytes)
  • Static RAM: 16,060 → 16,124 bytes (+64 bytes)
  • Heap free: 302,732 → 302,808 bytes (+76 bytes)
  • Net RAM saved: 12 bytes

Large Device (ESP32-IDF):

  • Flash: 1,095,150 → 1,094,226 bytes (-924 bytes)
  • Static RAM: 35,156 → 35,436 bytes (+280 bytes)
  • Heap measurements show variability but trend toward more free heap

The optimization successfully reduces flash usage and eliminates heap fragmentation while maintaining compatibility with all existing configurations.

@codecov-commenter
codecov-commenter commented Aug 2, 2025 •
edited
Loading
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.19%. Comparing base (4f58e1c) to head (015bb6f).

Files with missing lines Patch % Lines
esphome/core/config.py 33.33% 2 Missing ⚠️
Additional details and impacted files 
@@           Coverage Diff           @@
##              dev   #10020   +/-   ##
=======================================
  Coverage   60.19%   60.19%           
=======================================
  Files          51       51           
  Lines       10305    10305           
  Branches     1368     1368           
=======================================
  Hits         6203     6203           
  Misses       3745     3745           
  Partials      357      357

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bdraco
bdraco commented Aug 2, 2025
View reviewed changes
Comment thread esphome/core/application.h
@bdraco
bdraco marked this pull request as ready for review August 2, 2025 08:23
Copilot AI review requested due to automatic review settings August 2, 2025 08:23
@bdraco
bdraco requested a review from a team as a code owner August 2, 2025 08:23
Copilot AI reviewed Aug 2, 2025
View reviewed changes
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR converts the components_, devices_, and areas_ vectors in the Application class from std::vector to StaticVector for compile-time memory allocation. This optimization eliminates heap allocations and fragmentation by determining the required sizes from YAML configuration at compile time.

  • Replaces dynamic vectors with static allocation based on YAML configuration counts
  • Adds compile-time defines for component, device, and area counts in config generation
  • Enhances StaticVector with additional STL-compatible methods (empty, reverse iterators)

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
esphome/core/helpers.h Adds missing STL methods to StaticVector (empty(), reverse iterators)
esphome/core/defines.h Adds default count definitions for static analysis tools
esphome/core/config.py Replaces reserve calls with compile-time count definitions
esphome/core/application.h Converts vectors to StaticVector and reorganizes member layout

Comment thread esphome/core/application.h
Comment thread esphome/core/application.h
Comment thread esphome/core/application.h
This was referenced Aug 2, 2025
Merged
Merged
@jesserockz
jesserockz merged commit d86e1e2 into dev Aug 4, 2025
32 checks passed
@jesserockz
jesserockz deleted the static_comp_areas branch August 4, 2025 01:51
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 6, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@jesserockz jesserockz jesserockz approved these changes

Assignees

No one assigned

Labels

code-quality core

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bdraco @codecov-commenter @jesserockz
0