This hopefully addresses memory leak that occurs when not using a Home Hub and connections constantly drop and re-establish.
To do: must add back logic for notifications using new hap linked-list structure, and then delete static hap array code.
Also adds: homeSpan.controllerListBegin(), homeSpan.controllerListEnd()
Also adds: Controller::isAdmin(), Controller::getID(), Controller::getLTPK()
To accomplish this, needed to move Controller definition from HAP.h to HomeSpan.h
Also, converted Controller from struct to class to ensure Controller data is protected from being modified now that it is exposed through condstant iterators.
Eliminated the need to instantiate a static HKDF structure within HAPClient since there is no storage within HKDF itself, just a single function call (create).
homeSpan.setPairingCode() operates silently if called from within a sketch, but will report error to Serial Monitor and HALT sketch if code is invalid.
Otherwise, if called from 'S', or from AP, or as default during start-up, diagnostics are printed as usual and sketch will NOT halt if invalid code is provided
* Converted various homeSpan.set() methods from void to Span& to enable chaining
* Update Reference.md
* Update Reference.md
* Update src.ino
* Update Pixel.ino
changed nPixels from uint8_t to int to allow for longer pixel strands (library is coded for int, so there is no need to limit to uint8_t in the example).
* Converted static HTTP Buffer to dynamic TempBuffer
Saved about 8K in RAM!
* Upgraded TempBuffer logic to use .get() to return pointer
Next up: update Network.cpp to use client.available() and reduce fixed memory usage.
* Update Network.cpp
* Replaced appropriate snprintf() with asprintf() and used *m in sscanf
Optimized use of heap memory instead of stack memory for temporary variables, and makes code easier to read, provided all heap usage is free() at end of each function.
Also removed char *hostName as a member variable of homeSpan. It's never actually used since MDNS constructs its own copy of hostName as needed.
* Removed duplicate if(POST /pairings)
* modified HAP::receivedEncrypted() to use TempBuffer instead of fixed stack array
* Added HAPClient::listControllers()
Will be used for pairings list request
* Completed adding LIST to HAPClient::postPairingsURL()
HomeSpan now correctly responds to pairing list requests
* Refactored TLV
* Added support for zero-length TLV
* Added SEPARATOR as a formal kTLVType (and updated listControllers() to use)
* Added `uint8_t *buf(tagType tag, uint8_t *src, int len);` to load buffer needing external memcpy (and updated listControllers() to use)
* Simplified `uint8_t *TLV<tagType, maxTags>::buf(tagType tag, int len)`
Also updated HAP.cpp to use new `uint8_t *TLV<tagType, maxTags>::buf(tagType tag, uint8_t *src, int len)`
* Refactored Controller Structure and Add/Remove/List/Save Functions
Changed fixed-size array `struct Controller[MAX_CONTROLLERS]` to a dynamic linked-list of Controllers. Re-coded all related functions. MAX_CONTROLLERS no sets the size of any buffers but is only used to limit the ultimate size of the linked-list.
Saved about 1K of RAM since most of the time there are only 2 Controllers defined (as opposed to the 16 allowed).
* Support Non-English WiFi name
* Preventing possibility of buffer overwrite.
* Update README.md
* Update README.md
* Bumped release to 1.8.1
* Added homeSpan.setWebLogCallback(String (*f)())
Allows extension of initial table produced in WebLog, where f is a function that returns a String. The returned String is copied into the WebLog HTML just before the </table> tag is printed in the initial table.
* Change webLogCallback() to pass String as reference
* Verbose WiFi Reconnect
Add the ability to turn off WiFi reconnect messages.
* Update Reference.md
* Update Logging.md
* Update README.md
* Added homeSpan.setWifiCallbackAll(int n)
Adds a second type of WiFi Callback that is called every time WiFi is established OR re-established after a disconnect. Passes the number of times WiFi has been connected as an argument.
* Update README.md
* Update Reference.md
* Update README.md
* Made conforming changes to setVerboseWiFiReconnect
* Update Reference.md
* Update README.md
* Update Reference.md
* Update Logging.md
* Changed Accessory Limit from 41 to 150 and added Memory Warning
Low Memory Watermark is compared to DEFAULT_LOW_MEM_THRESHOLD (currently set at 80,000 bytes) after HAP initialization but before WIFI connection. If Low Memory Watermark is below DEFAULT_LOW_MEM_THRESHOLD, a WARNING is issued.
Note: memory is based on heap with MALLOC_CAP_DEFAULT capabilities
* Update Reference.md
* Made vTaskDelay(5) in pollTask conditional of using autoPoll()
vTaskDelay(5) is only needed if polling is in its own task. If calling poll() directly, there is no need for any delay.
* Update Logging.md
* Add hint for UTF8 usage in names
* Moved vTaskDelay(5) from pollTask() to homeSpan.autoPoll()
Adding the vTaskDelay(5) to the code that spawns the separate task is more consistent with how Arduino-ESP32 spawns the main loop() task.
* Update HomeSpan.h
Forgot braces in for-loop of autoPoll()
* Update 07-AccessoryNames.ino
fix typo
* Update Reference.md
* Add MAC Address output to RemoteDevice.ino
Prompts user with reminder of which MAC Address to use in Main Device.
* Added new homeSpan.rebootCallback() function
An optional callback used to track the number of times the device reboots, where a reboot is defined as any restart where the uptime was less than a specified number of milliseconds (default=5000)
* Update src.ino
* Update Reference.md
* Update src.ino
* Update README.md
* Update Reference.md
* Update README.md
* Update 07-AccessoryNames.ino
* Update HAPClient::sendEncrypted() to send individual frames to save memory
Rather than use a large temporary buffer to store all encrypted frames to transmit all at once, each encrypted frame is transmitted as soon as created, which means the temporary buffer only needs to be large enough to store a single encrypted frame.
For a device with a large number of Accessories, this can save 15-20 KB of temporary storage, allowing for more Accessories.
Note this change does not reduce the size of the unencrypted message that is initially produced and can also be quite large. Potential for future project.
* Changed storage of required and optional Characteristics from unordered_set to vector
Significantly reduces memory usage - unordered_sets take up much more memory and using std::find from <algorithm> is as efficient as using find() method of an unordered_set.
To do: Remove all uses of unordered_set across HomeSpan wherever possible
* Changed unordered_set to vector when checking for duplicate AID
This eliminated the use of unordered_set everywhere in HomeSpan
* Added error checking for NVS storage
Also added NVS storage diagnostic to 'm' command.
* Fixing HTML syntax error in weblogs.
* PSRAM: Adds custom allocator and HS_MALLOC/HS_CALLOC macros
Forces all heap requests to be drawn from PSRAM, unless the device does not contain PSRAM.
To do: Provide easy-to-use NEW() macro; add custom allocator to unordered sets; auto-shrink vectors after updateDatabase()
* Added ps_new() macro
A more generic version of `new(*ptr)` that allocates from PSRAM if present, otherwise is that same a normal `new`
* Added HS_REALLOC macro for generalized realloc() when using PSRAM
When using PSRAM, changing the number of Accessories no longer creates any change to heap usage on internal RAM. The entire Accessory will be stored in PSRAM.
* Address WebLog size issue that limited number of entries
Added logic to sequentially transmit chunks of the HTML response String whenever it grows beyond 1024 bytes. This ensures the number of WebLog entries does not create a very large HTML String that eats up all the free heap space. WebLog entries are now only limited by storage of the entries themselves. Tested successfully with 500 entries.
* Change NVS storage for Characteristics to always use nvs_set_u64()
Reduces NVS consumption by from 3 to only 1 records when storing numerical values.
Also: Fixed memory problem in getCharacteristics by replacing dynamically-sized stack buffers with TempBuffer (must do this everywhere), which was causing stack probllem when combination number of Accessories and number of Characteristics got very large.
* Override new operator for SpanAccessory, SpanService, and SpanCharacteristic
New Operator now uses PSRAM if available, else uses Internal RAM. Obviates the need for the ps_new() macro, though will keep definition since it may be useful for any class that does not already override malloc() to use PSRAM.
* Streamlined NVS recall for String values
Avoids need for intermediate variable
* Create MaxAccessories.ino
Demonstrates use of custom partition and shows how PSRAM is used in background when available
* Create partitions.csv
* Changed various static and dynamic stack-based buffers to TempBuffer
Also added casting overload to TempBuffer so that get() method is no longer needed
* Eliminated .get() from TempBuffer objects where possible
Stylistic only. Cannot remove .get() when a TempBuffer is being re-cast. Also can't remove .get() when cast can't be determined, such as when used in Serial.printf() [potential to do: explore if this can be addressed, though this is stylistic only and has no impact on code]
* Added Characteristic Constants
Added constants to Characteristics that describe states.
Example: Characteristic::SecuritySystemTargetState::ARM_STAY
* Correction to time server URL doc
* Addressed issue with new Characteristic enum constants
* Update MaxAccessories.ino
* Added ability to turn off SpanPoint encryption
New class-level method SpanPoint::setEncryption(boolean encrypt). Default is to use encryption unless method is called with encrypt=false.
With encryption, 7 connections max; without encryption, 20 connections max.
Serial Monitor provides warnings if connections are exceeded.
* Update NOW.md
* Update README.md
* Update README.md
* Update README.md
* Added new TLV8 class
To do: Refactor HAP.cpp to use new TLV8 class
* Starting to integrate new TLV8 into postPairVerify
* Completed re-factoring of postPairVerify using new TLV8
Next: re-factor postPairSetup
* Preparing for re-factoring postPairSetup
* Starting re-factoring of postPairSetup
* added TLV length function
Checks that iterator exists (i.e. does not equal end()) and returns length, else returns -1. Can be used for checking for both the presence, and length, of a TLV record
* Completed initial re-factoring of postPairSetup - must test!
* Completed postPairSetup and fully tested - also changes TLV8 from vector to forward_list
Changed TLV8 from vector to forward_list to ensure iterators are always retained after adding new elements.
Nest: refactor postPairings
* Completed and fully tested refactoring of TLV8
Next up: remove old TLV code and clean up
* Removed all old TLV code
* Cleaning up postverify
* Added new initializer to TempBuffer
Allows TempBuffer to be initialized by concatenation of multiple existing buffers.
Constructor contains pairs of (T *buf), nElements, where last argument MUST BE NULL to signify end of variable argument list.
Will work with any type T, and nElements is in units of elements of type T.
* Start re-working SRP6A's use in Pair-Setup to make it local instead of global
* Initial integration of local SRP6A into Pair Setup
* Moved definitions of N3072 and other constants into SRP6A structure
* In process of updated Pair-Setup M3...
Cleaning up code by replacing calls to mbedTLS within Pair-Setup with arguments passed to/from SRP6A. Much better and better encapsulates SRP6A
* Completed Pair-Setup M3 update
* Completed refactoring of SRP6A and integration into all steps of Pair-Setup
* Remove include HAP.h from SRP
SRP now only requires Utils (for TempBuffer).
Also, moved HS_ALLOC definitions into Utils so it can be re-used by others files
* Initial test of StreamBuffer added to WebLog
* Moved Streamer from Utils.h into separate Streamer.cpp
But declared Streamer inside HomeSpan.h instead of separate Streamer.h since it should only be used when HomeSpan.h is used.
Also, changed class structure so that it is "final" by removing virtual tags and making all protected variables private.
* Begin integration of hapStream into HAP.ccp
* Moved StreamBuf parameter calls into HapOut
* Moved StreamBuf into HapOut as private nested class
* Added getSize() to HapOut and changed default behavior to NEVER output anything
Note: getSize() must be called before flush(), which resets byte counter
* Converted sprintfAttributes to printfAttributes and added encryption to HapOut
Success!
* Added SHA-384 hash computation to HapOut
Hashing of entire database is now done piecemeal within HapOut, preventing the need to load the entire database into a large buffer just to obtain a unique hash.
Modified updateDatabase() to use HapOut.getHash()
* Deleted sprintfAttributes for Span, SpanAccessory, and SpanService
These now use the new printfAttributes with HapOut
* Converted sprintfNotify and eventNotify to use HapOut
Successfully tested with MultiTest script
* Added back weblogCallback() functionality that uses hapOut
* Converted all remaining sprintfAttributes to printfAttributes
This removes last of large-sized JSON buffers - everything has now been converted to using hapOut.
Next up: Must add prettyPrint to hapOut to support 'd' command
* Moved prettyPrint into HapOut
This completes re-work of all JSON buffer memory management to use HAPOut.
Next up: Stress test with Max Accessories to see how many Accessories we can now handle.
* Changed HapOut delay from 1ms to 50ms for WiFi transmits. Seems to be much more stable.
* Update MaxAccessories.ino
* fix for PSRAM with new hapOut
Force hapOut to use internal memory only.
Tested on ESP32-S2 with/without PSRAM. However, crashes on ESP32-S3 when using PSRAM (?!)
* Changed hapOut delay back to 1ms - works fine
Identified issue with ESP32-S3 crashing when using PSRAM. Problem was not the code, but that the sketch was using pins that are needed to run the Octal PSRAM that comes with the Espressif S3 DevKit.
* Fixed CSS in weblog
Missing conversion from String to c_str() when using hapOut.
* Added homeSpan.getWebLog()
* Update src.ino
* Update src.ino
* started work on TLV streaming with << operator
Goal - eliminate sendEncrypted() completely by using hapOut in tlvRepond().
Next up: use function call instead of << operator so logic can be moved into TLV.h
* moved tlv streaming into tlv class
* refactored tlvRespond() to use hapOut instead of sendEncrypted()
To do:
1) update putPrepareURL() to use hapOut instead of sendEncrypted()
2) remove sendEncrypted() since it is no longer needed
3) try moving MAllocator template into separate file (memory.h?) so it can be called from any other module
4) remove #include "HomeSpan.h" from TLV8
* Moved all PSRAM definitions into separate PSRAM.h file
Now allows #include "HomeSpan.h" to be removed from TLV dependencies
* updated putPrepareURL and deleted receiveEncrypted()
Must test with Door example, which uses putPrepare
* Modified range checks so that "NAN" gets picked up as error for FLOAT Characteristics
* Truncate all STRING-based Characteristics to 64 characters when adding to JSON
HAP default max length of STRING Characteristics is 64 characters
* Added homeSpan.getAutoPollTask() to return task handler of autoPoll
Returns NULL if autoPoll has not been used.
Also added low stack size for both main Arduino loop task and HomeSpan autoPoll task to 'm' command
* Added optional triggerType to homeSpan.setControlPin()
Default is SpanButton::TRIGGER_ON_LOW. Follows same options as SpanButton, including the ability to provide your own triggerType function of form:
boolean f(int pin)
* Cleaned up some public/private permissioning inside PushButton and SpanButton
Ensure users can access constants like TRIGGER_ON_HIGH, TRIGGER_ON_LOW, etc. using any of these equivalent fully-qualified names:
* PushButton::TRIGGER_ON_HIGH
* SpanButton::TRIGGER_ON_HIGH
* SpanToggle::TRIGGER_ON_HIGH
* Update Reference.md
* Update Reference.md
* Added CustomNVSPartition Example
To do: Add to list of Other Examples in documentation.
* Update CustomNVSPartition.ino
* Update Tutorials.md
* Create ExternalReference.ino
* Update Tutorials.md
* Update Logging.md
* Update Reference.md
* New Services and Characteristics List (#751)
* auto-generation of service list
* Update makeServiceList
* Update makeServiceList
* Update test.md
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update Span.h
* Update test.md
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Create ServiceListHead.md
* Update ServiceListHead.md
* Update ServiceListHead.md
* Update test.md
* update
* update
* Update ServiceList.md
* update
* Update ServiceList.md
* update
* removed bold font for Characteristic Constants in ServiceList.md
* fix heavy_check_mark
* add <span> to ServiceList to prevent linebreaks before checkmark
* fixed typo in ServiceLabelNameSpace description
* Create ServistList_Expanded.md
* Update ServistList_Expanded.md
* Update ServiceList.md
* added ServiceList_Expanded
* added code to create pre-expanded Services and Characteristics List
* Update Tutorials.md
* Update Tutorials.md
* Added ConfiguredName to Switch and Lightbulb
* Moved error/warning output to end of 'i' output
* Add configuredName() to all applicable Services and deprecated Name()
* reformat ServiceList to remove expandable details
Expandable details are not as practical to navigate for a page this size. Easier to see everything laid out all at once.
* Re-organized Services List into Service Groups
* Update ServiceList.md
* converted Service Group names to upper case
* Added linebreak after each Service
* updated some descriptions
* Delete ServiceList_Expanded.md
* Revised Example 11 (Service Names) to use ConfiguredName Characteristic
Complete re-work of Example 11 - fully tested for iOS 17.2.1
* Documentation update (#765)
* Update Stepper.md
* Update README.md
* Update FAQ.md
* Update TVServices.md
* Get Arduino-ESP32 Version from <esp_arduino_version.h> instead of <core_version.h>
This works fine for all versions after 2.0.3, but returns 2.0.0 for versions 2.0.2 and 2.0.1, as well as 2.0.0. Not an issue since current version is 2.0.14.
* Update Solutions.md
* Added Stepper_UNIPOLAR and Stepper_UNL2003
ALSO: Re-arranged include files so that StepperControl.h is now part of HomeSpan.h. No need to include anything except for HomeSpan.h to use any of the Stepper Motor classes. Must update documentation accordingly.
NOTE: Always including StepperControl.h DOES NOT increase size of compile if no stepper motors are instantiated.
To do: Consider doing the same for Pixel, PWM, and RMT as well.
* Removed #include "extras/PwmPin.h" from all Tutorial Examples
Since PwmPin.h is already included in HomeSpan.h, there is no reason to separately include it in user sketches.
* Removed all #include "extras/*.h" from "Other Examples"
Including any extras/*.h files in a HomeSpan sketch is no longer necessary since these are already included in HomeSpan.h.
* Update PWM.md
* Update RMT.md
* Update Pixels.md
* Update RMT.md
* Update Stepper.md
* Update 17-LinkedServices.ino
* Update Stepper.md
* Update Stepper_TB6612.md
* Update Stepper_A3967.md
* Update Stepper.md
* Update Stepper.md
* Update Stepper.md
* Update StepperControl.h
* Update Stepper_A3967.md
* Create Stepper_ULN2003A.md
* Update Stepper_ULN2003A.md
* Update Stepper_ULN2003A.md
* Update Stepper_TB6612.md
* Update Stepper_A3967.md
* Create Stepper_UNIPOLAR
* Create Stepper_UNIPOLAR.md
* Update Stepper_UNIPOLAR.md
* Delete Stepper_UNIPOLAR
* Update Stepper_UNIPOLAR.md
* Update Stepper_UNIPOLAR.md
* Update Stepper_UNIPOLAR.md
* Update Stepper_UNIPOLAR.md
* Added option to specify NULL for 4th parameter of homeSpan.enableWebLog()
If 4th parameter (the statusURL) is NULL, web logging will NOT be enabled BUT time server will still acquire time if server address and time zone were specified as 2nd and 3rd parameters.
Note that not having web logging enabled only impacts ability to acquire weblog from HTTP request. It has no effect on the actual logging of WEBLOG() messages and thus homeSpan.getWebLog() works as expected.
* Allow user-defined data when receiving weblogs from callback (#770)
* Adding userData to getweblogs callback.
Allows user-specific data to be available in the callback that returns weblog HTML data.
* Adding userData to getweblogs callback.
Allows user-specific data to be available in the callback that returns weblog HTML data.
Return HTTP response code and content type only when no weblog callback defined
* Small clean-up and formatting of comments
* Bump version to 1.9.0
* Updated copyright dates to 2024
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update Reference.md
* Update Logging.md
* Update README.md
* Update Logging.md
* Update README.md
* Update HomeSpan.h
* Update README.md
* Update FeatherPins.h
Changed F21 from pin 35 to pin 38 for ESP32-S3 to avoid conflicting with Octal PSRAM.
* Update README.md
* Update README.md
* Update Logging.md
* Update library.properties
---------
Co-authored-by: Yitao Jiang <jytscientist@hotmail.com>
Co-authored-by: Francois <franklamoureux@hotmail.com>
Co-authored-by: Michael Geramb <mail@michael-geramb.at>
Co-authored-by: frankonski <frankonski@users.noreply.github.com>
OTA pasword now stored in NVS. Use 'O' command to change from default. Note password is stored as MD5 hash, and is therefore unrecoverable. Changes to password DO NOT take effect until next reboot. Password CAN be changed even if OTA has not been enabled for sketch. Blank passwords are not allowed.
Stored password can only be erased with 'E' command.
Program now halts if more than 41 Accessories are defined. This allows for a bridge to have 40 Accessories. HAP limit is 150, but there is not enough memory in ESP32 to robustly handle more (testing up to 50 LightBulb Services worked, but was sluggish).
1) Used `const char *` instead of `char *` where appropriate, including the need to create a dummy blank string for us in certain places.
2) Set initialization of WiFiClient to 0 instead of NULL, since WiFiClient is not a pointer (probably don't need to set it to anything since WiFiClient overrides the boolean operator anyway).
3) Cleaned up some of the messaging and logic when WiFi tries to connect so that users know to wait a bit.
4) Only remaining warning messages are for casting SpanService to (void *), which I think i unavailable (and is not forbidden).
To Do: Go through examples and check for warnings (will likely need to convert `char *` to `const char *` in many places.
Convention is that every *.cpp references, in this order:
1) Any required core librries
2) Its own *.h (i.e. Foo.cpp -> Foo.h)
3) All other *.h files needed by the cpp, UNLESS those *.h files are already in its own *.h file. Must include all *.h files that are not in its own *.h file even if it is others that are referenced.
This change allows the user to set the log-level with homeSpan.setLogLevel() instead of needing to change in Settings.h. Next up: Update Example that introduces VERBOSE to instead use setLogLevel(). Consider adding an interactive 'L' command to change this during run-time.
Also, added "#pragma once" to every *.h file to prevent duplication of definitions. TO DO: Review all #include to ensure sanity across all files.
Move all configuration code into Network.cpp, eliminating any calls or connection to HAP.cpp. Network.cpp is now standalone for HomeSpan.cpp to call directly to handle ALL apects of WiFI and Setup Code initialization.
Also move TempBuffer from HAP.h to Utils.h so it can be used by Network.h
This is no longer needed since PushButtons can be easily emulated from within a Service loop() as per Example 14. Library is now fully up-to-date: TimedResets and SpanEvents have been fully removed.
Completed new framework for using loop() methods and deleted all code related to new SpanEvent() and event() loops. Re-worked Example 13 to utilize new framework and validated it functions as expected.
Service loops() called for only those Services with over-ridden loop() methods. TO DO: update event logic to follow new framework. Vector should point to all CHARACTERISTICS that are updated to setVal()
This check all timed write PID/TTL entries and clears all those that are expired. This completes the implementation of Timed Write functionality as required for PUT /prepare requests.
checkEvents() now allows for multiple characteristics to be updated by service->notify(). This change included the use of vector<SpanBuf> to dynamically create the the temporary storage needed. To do: re-work similar routines to use vector<SpanBuf> instead of needing to perform two passes each time to gauge size of SpanBuf array needed.
putCharacteristics and checkNotifications now both call the same eventNotify routing. This is being done to prepare for services that will stream notifications back to client (like a temperature sensor, or pushbutton). TO DO: rename checkNotifications to check TimedResets, and rename SpanPut to something more generic.
Gregg