diff --git a/components/m5stack-tab5/Kconfig b/components/m5stack-tab5/Kconfig index d6b8e04ed7..67f02663fa 100644 --- a/components/m5stack-tab5/Kconfig +++ b/components/m5stack-tab5/Kconfig @@ -56,4 +56,22 @@ menu "M5Stack Tab5 Configuration" PPA-only, and a rotated flush with no PPA available is dropped with an error instead of being rotated in software. + config M5STACK_TAB5_DPI_NUM_FBS + int "Number of DPI frame buffers" + default 1 + range 1 3 + help + How many frame buffers the MIPI-DSI DPI panel driver allocates in PSRAM + (esp_lcd_dpi_panel_config_t::num_fbs). Each one is 720 x 1280 RGB565 = + 1.8 MB of PSRAM, so every buffer beyond the first costs 1.8 MB. + With 1 (the default) LVGL renders into separate draw buffers and the BSP + copies each flushed area into the one frame buffer the panel scans out. + With 2 or 3 the application can render straight into the frame buffers + and switch whole frames instead of copying them: see + M5StackTab5::frame_buffers() and + DisplayConfig::render_to_frame_buffers_enabled. LVGL's DIRECT mode uses + at most two of them (buffers 1 and 0); a third only serves an + application that drives the frame buffers itself. Extra buffers without + DIRECT mode buy nothing: LVGL keeps copying into the scanned-out buffer. + endmenu diff --git a/components/m5stack-tab5/README.md b/components/m5stack-tab5/README.md index a5e5be261a..62cfa80e8c 100644 --- a/components/m5stack-tab5/README.md +++ b/components/m5stack-tab5/README.md @@ -101,6 +101,54 @@ auto controller_type = tab5.get_display_controller(); const char* controller_name = tab5.get_display_controller_name(); ``` +## Rendering into the panel's frame buffers + +The DPI panel scans a frame buffer in PSRAM out to the display continuously. By +default the BSP creates one (`CONFIG_M5STACK_TAB5_DPI_NUM_FBS` = 1): LVGL +renders into separate draw buffers and every flush copies the area into that +frame buffer. With 2 (or 3) frame buffers an application can render straight +into them and show a finished frame by switching the panel to its buffer, with +no copy: + +- `frame_buffers()` returns the panel's frame buffers (after + `initialize_lcd()`), each `frame_buffer_size()` bytes, cache-line aligned. + `esp_lcd_panel_draw_bitmap()` with a pointer inside one of them makes the + panel switch to that buffer instead of copying it. +- `present_frame(frame_buffer)` switches the panel to a finished frame and + waits until the panel scans it out (at most one refresh: about 17 ms, 21 ms + on the ILI9881 panel), so the previous buffer is free to draw into: + tear-free double buffering. The wait uses the DMA's frame-done event; + `on_color_trans_done` cannot serve, because for a frame buffer the driver + calls it before the switch. Tear-free therefore depends on that wait + succeeding. On a timeout (default 100 ms) the switch stays queued and the + call returns `false` rather than stalling: the caller carries on and the + next frame may tear. After three consecutive timeouts (a frame-done event + that never arrives, e.g. a panel / ESP-IDF combination whose callback does + not fire) `present_frame()` disables its wait itself, with a warning, so + the UI is not held at one frame per timeout for good; frames may then tear + until `set_vsync_enabled(true)` re-arms it. `set_vsync_enabled(false)` skips + the wait deliberately. The DIRECT-mode flush ignores the return value for + the same reason: it never blocks LVGL on a missing event. +- `initialize_display({.render_to_frame_buffers_enabled = true})` sets LVGL up in DIRECT + mode on two of them, with the BSP's flush calling `present_frame()` at the end + of each frame. No draw buffers and no rotation scratch buffer are allocated. + The display rotation must stay at 0 (LVGL cannot rotate in DIRECT mode). + +```cpp +// menuconfig: M5Stack Tab5 Configuration -> Number of DPI frame buffers = 2 +auto &tab5 = espp::M5StackTab5::get(); +tab5.initialize_lcd(); +tab5.initialize_display({.render_to_frame_buffers_enabled = true}); +``` + +Measured on a Tab5 in an LVGL application (full-screen redraw, 720 x 1280 +RGB565): 75.6 ms with full-frame double draw buffers copied into one frame buffer, +48.6 ms rendering into two frame buffers. Memory: two frame buffers take 3.7 MB of +PSRAM, against 5.5 MB for one frame buffer plus two full-frame draw buffers (and +the 1.8 MB rotation scratch buffer is not needed either). The frame buffers are +cache-line aligned, so they also satisfy a raised `LV_DRAW_BUF_ALIGN` (e.g. 128 +for the PPA draw unit), which a plain `heap_caps_malloc()` draw buffer may not. + ## Example The [example](./example) shows how to use the `espp::M5StackTab5` hardware abstraction component to initialize and use various subsystems of the Tab5. diff --git a/components/m5stack-tab5/example/README.md b/components/m5stack-tab5/example/README.md index b633c27642..2248e654d3 100644 --- a/components/m5stack-tab5/example/README.md +++ b/components/m5stack-tab5/example/README.md @@ -71,6 +71,16 @@ those tasks than the defaults assume: your own project on this example (or reproduce its functionality), keep this override in your `sdkconfig` / `sdkconfig.defaults`. +### Rendering into the panel's frame buffers + +Set **M5Stack Tab5 Configuration -> Number of DPI frame buffers** to 2 and +enable **M5Stack Tab5 Example Configuration -> Render LVGL straight into the +panel's frame buffers** to run the example with +`initialize_display({.render_to_frame_buffers_enabled = true})`: LVGL renders in DIRECT +mode into the panel's frame buffers instead of into two full-frame PSRAM draw +buffers that are copied on every flush. The rotate button does nothing in this +mode (LVGL cannot rotate in DIRECT mode). See the component README. + ## Hardware Requirements - M5Stack Tab5 development board diff --git a/components/m5stack-tab5/example/main/Kconfig.projbuild b/components/m5stack-tab5/example/main/Kconfig.projbuild new file mode 100644 index 0000000000..165e769875 --- /dev/null +++ b/components/m5stack-tab5/example/main/Kconfig.projbuild @@ -0,0 +1,19 @@ +menu "M5Stack Tab5 Example Configuration" + + config M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS + bool "Render LVGL straight into the panel's frame buffers" + depends on M5STACK_TAB5_DPI_NUM_FBS > 1 + default n + help + Initialize the display with + DisplayConfig::render_to_frame_buffers_enabled: LVGL renders in DIRECT + mode straight into the DPI panel's frame buffers and a finished + frame is shown by switching the panel to it, instead of copying + LVGL's draw buffers into the one frame buffer the panel scans + out. Saves the two draw buffers and the rotation scratch buffer + (PSRAM) and the copy on every flush. Needs + CONFIG_M5STACK_TAB5_DPI_NUM_FBS >= 2 (M5Stack Tab5 Configuration). + The rotate button does nothing in this mode: LVGL cannot rotate in + DIRECT mode. + +endmenu diff --git a/components/m5stack-tab5/example/main/gui.cpp b/components/m5stack-tab5/example/main/gui.cpp index 6cf1fe2036..77dbfbaf16 100644 --- a/components/m5stack-tab5/example/main/gui.cpp +++ b/components/m5stack-tab5/example/main/gui.cpp @@ -417,16 +417,27 @@ void Gui::init_circle_layer() { } bool Gui::update(std::mutex &m, std::condition_variable &cv) { + [[maybe_unused]] uint32_t next_ms = 16; { std::lock_guard lock(mutex_); - lv_task_handler(); + next_ms = lv_task_handler(); // keep the audio volume label in sync with the live BSP state, so the // first press of a volume button doesn't appear to jump from a stale // default (the values are set by app_main after the Gui is constructed) update_audio_label(); } std::unique_lock lock(m); +#if CONFIG_M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS + // The flush already waits for the panel's frame-done event, which paces a + // frame that has something to draw; here only sleep until LVGL's next timer + // is due (at least 1 ms so an idle LVGL never spins). lv_timer_handler() + // returns LV_NO_TIMER_READY when nothing is scheduled, which would sleep + // forever: cap that case at the display refresh period. + const uint32_t sleep_ms = next_ms == LV_NO_TIMER_READY ? LV_DEF_REFR_PERIOD : next_ms; + cv.wait_for(lock, std::chrono::milliseconds(std::max(sleep_ms, 1))); +#else cv.wait_for(lock, std::chrono::milliseconds(16)); +#endif return false; // don't stop the task } @@ -541,6 +552,11 @@ bool Gui::draw_page_active() { } void Gui::next_rotation() { +#if CONFIG_M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS + // LVGL renders into the panel's frame buffers in DIRECT mode, which cannot + // rotate. + logger_.warn("Rotation is not available while rendering into the frame buffers"); +#else std::lock_guard lock(mutex_); clear_circles_impl(); auto rotation = lv_display_get_rotation(lv_display_get_default()); @@ -553,6 +569,7 @@ void Gui::next_rotation() { lv_obj_set_size(circle_layer_, width, height); lv_obj_align(circle_layer_, LV_ALIGN_CENTER, 0, 0); lv_obj_invalidate(circle_layer_); +#endif } void Gui::cycle_brightness() { diff --git a/components/m5stack-tab5/example/main/m5stack_tab5_example.cpp b/components/m5stack-tab5/example/main/m5stack_tab5_example.cpp index 7f29258638..1b9717261d 100644 --- a/components/m5stack-tab5/example/main/m5stack_tab5_example.cpp +++ b/components/m5stack-tab5/example/main/m5stack_tab5_example.cpp @@ -94,12 +94,21 @@ extern "C" void app_main(void) { // repaints at once rather than wiping progressively. Partial updates still // only render/flush their dirty area, so the larger buffer costs nothing // there (just PSRAM). - logger.info("Initializing display..."); + logger.info("Initializing display ({} DPI frame buffers)...", tab5.frame_buffers().size()); +#if CONFIG_M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS + // LVGL renders straight into the panel's frame buffers (DIRECT mode): no + // draw buffers, no rotation scratch buffer, and no copy per flush. + if (!tab5.initialize_display({.render_to_frame_buffers_enabled = true})) { + logger.error("Failed to initialize display!"); + return; + } +#else auto pixel_buffer_size = tab5.display_width() * tab5.display_height(); if (!tab5.initialize_display(pixel_buffer_size)) { logger.error("Failed to initialize display!"); return; } +#endif // make the filter we'll use for the IMU to compute the orientation static constexpr float angle_noise = 0.001f; diff --git a/components/m5stack-tab5/include/m5stack-tab5.hpp b/components/m5stack-tab5/include/m5stack-tab5.hpp index c87733ce50..5d98880402 100644 --- a/components/m5stack-tab5/include/m5stack-tab5.hpp +++ b/components/m5stack-tab5/include/m5stack-tab5.hpp @@ -1,5 +1,6 @@ #pragma once +#include #include #include #include @@ -213,11 +214,41 @@ class M5StackTab5 : public BaseComponent { /// \return true if the LCD was successfully initialized, false otherwise bool initialize_lcd(); + /// How initialize_display() sets LVGL up + struct DisplayConfig { + /// Size of each of LVGL's two draw buffers, in pixels. They are allocated + /// in PSRAM, as is the rotation scratch buffer of the same size. Ignored + /// with render_to_frame_buffers_enabled. + size_t pixel_buffer_size{1280 * 720 / 10}; + /// Let LVGL render in DIRECT mode straight into the DPI panel's frame + /// buffers (see frame_buffers()) instead of into separate draw buffers. + /// No draw buffers and no rotation scratch buffer are allocated, and a + /// finished frame is shown by switching the panel to its frame buffer + /// instead of copying it. Needs initialize_lcd() first and + /// CONFIG_M5STACK_TAB5_DPI_NUM_FBS >= 2. + /// The BSP's flush shows each frame with present_frame(), so LVGL never + /// draws into the buffer the panel is still scanning out (no tearing). + /// \note The display rotation must stay at 0: LVGL cannot rotate in DIRECT + /// mode. Pixels written with write_lcd_lines() are lost at the next + /// frame switch. + /// \note If you install your own LVGL flush callback, call + /// lv_display_flush_ready() once for every call of it (LVGL waits + /// for it before flushing the next area), or register a wait + /// callback with lv_display_set_flush_wait_cb(); the transfer-done + /// interrupt no longer does it. + bool render_to_frame_buffers_enabled{false}; + }; + /// Initialize the LVGL display /// \param pixel_buffer_size The size of the pixel buffer /// \return true if the display was successfully initialized, false otherwise bool initialize_display(size_t pixel_buffer_size = 1280 * 720 / 10); + /// Initialize the LVGL display + /// \param config How to set LVGL up (draw buffers, or the panel's frame buffers) + /// \return true if the display was successfully initialized, false otherwise + bool initialize_display(const DisplayConfig &config); + /// Initialize the GT911 multi-touch controller /// \param callback The touchpad callback /// \return true if the touchpad was successfully initialized, false otherwise @@ -303,6 +334,90 @@ class M5StackTab5 : public BaseComponent { /// \return A shared pointer to the display driver const std::shared_ptr &display_driver() const { return display_driver_; } + /// Most frame buffers the DPI panel driver supports (ESP-IDF's limit) + static constexpr size_t max_frame_buffers = 3; + + /// Get the DPI panel's frame buffers + /// \return The CONFIG_M5STACK_TAB5_DPI_NUM_FBS frame buffers initialize_lcd() + /// got from the panel driver, each frame_buffer_size() bytes + /// (display_width() x display_height() RGB565 pixels) in PSRAM and + /// cache-line aligned; empty until initialize_lcd() succeeded + /// \note The panel scans one of them out all the time. Passing a pointer + /// inside one of them to esp_lcd_panel_draw_bitmap() makes the panel + /// switch to that buffer (after a cache write-back) instead of copying + /// it. This lets an application render (e.g. LVGL in DIRECT mode) + /// straight into the frame buffers; see also + /// DisplayConfig::render_to_frame_buffers_enabled. + /// \note If you install your own LVGL flush callback, call + /// lv_display_flush_ready() once for every call of it (LVGL waits + /// for it before flushing the next area), or register a wait + /// callback with lv_display_set_flush_wait_cb(); the transfer-done + /// interrupt no longer does it. + std::span frame_buffers() const { + // frame_buffers_ is written before initialize_lcd() opens the gate + if (!lcd_initialized_.load(std::memory_order_acquire)) { + return {}; + } + return {frame_buffers_.data(), num_frame_buffers_}; + } + + /// Get the size of one DPI frame buffer + /// \return The size of one frame buffer in bytes + static constexpr size_t frame_buffer_size() { + return display_width_ * display_height_ * sizeof(Pixel); + } + + /// Show a whole frame that is already in one of the panel's frame buffers, + /// and wait until the panel scans it out + /// + /// The tear-free way to switch frame buffers, e.g. at the end of an LVGL + /// DIRECT-mode frame. esp_lcd_panel_draw_bitmap() on a frame buffer only + /// writes the cache back and records the buffer as the next one to scan + /// out; the panel's DMA switches to it when it starts its next frame. + /// Until then it is still reading the previous buffer, so drawing into that + /// one at once would tear. This waits for the DMA's frame-done event (the + /// moment it restarts on the new buffer), after which the previous buffer + /// is free. (on_color_trans_done cannot be used for this: for a frame + /// buffer the driver calls it synchronously, before the switch.) + /// + /// \param frame_buffer A pointer inside one of frame_buffers() (its start, or any + /// address in it: the buffer that contains it is shown) + /// \param timeout_ms How long to wait for the switch + /// \return true once the panel scans the frame out, or at once if the wait + /// is disabled or unavailable (see set_vsync_enabled()); false if + /// the panel has only one frame buffer (CONFIG_M5STACK_TAB5_DPI_NUM_FBS + /// must be >= 2: with one there is nothing to switch, and no frame + /// that is safe to draw into afterwards), \p frame_buffer is not + /// inside one of frame_buffers(), a previous transfer is wedged, or + /// the wait timed out (the switch is still queued; carry on rather + /// than stall). Callers are serialized: one presenter at a time. + /// \note Waits at most one panel refresh period (about 17 ms; 21 ms on the + /// ILI9881 panel) when the panel is running - two on ESP-IDF before + /// v5.5.5 / v6.0.3, whose frame-done event is the DSI bridge's VSYNC + /// rather than the DMA's re-arm (see present_frame() in video.cpp). + /// Do not call it from an ISR. + bool present_frame(const void *frame_buffer, uint32_t timeout_ms = 100); + + /// Enable or disable the wait in present_frame() + /// \param enabled false: present_frame() queues the switch and returns at + /// once (faster, but the next frame can tear). present_frame() + /// disables the wait itself after a few consecutive timeouts (a + /// frame-done event that never arrives); pass true to arm it again. + /// Re-arming starts a new outage episode: the consecutive-timeout + /// count and the once-per-episode timeout warning are reset, so a + /// later stall is logged again. + void set_vsync_enabled(bool enabled) { + vsync_timeouts_in_a_row_ = 0; + if (enabled) { + vsync_timeout_logged_.store(false, std::memory_order_relaxed); + } + vsync_enabled_.store(enabled, std::memory_order_relaxed); + } + + /// Whether present_frame() waits for the frame-buffer switch + /// \return true if the wait is enabled (the default) + bool vsync_enabled() const { return vsync_enabled_.load(std::memory_order_relaxed); } + /// Write lines to the LCD /// \param xs The x start coordinate /// \param ys The y start coordinate @@ -1106,9 +1221,19 @@ class M5StackTab5 : public BaseComponent { // buffer + draw_bitmap copy (which doubles the PSRAM traffic and can starve // the DSI scan-out DMA into FIFO underruns / on-screen streaking). Null when // unavailable, in which case flush() falls back to the scratch-buffer path. - void *dpi_framebuffer_{nullptr}; + // Null when the PPA direct-to-framebuffer path is unavailable. Moves with + // present_frame() to the buffer the panel scans out (written under + // panel_op_mutex_, read on the LVGL task by flush()). + std::atomic dpi_framebuffer_{nullptr}; size_t dpi_framebuffer_bytes_{0}; + // The DPI panel's frame buffers (frame_buffers()), filled by initialize_lcd() + std::array frame_buffers_{}; + size_t num_frame_buffers_{0}; + // initialize_display() set LVGL up in DIRECT mode on the frame buffers + // (DisplayConfig::render_to_frame_buffers_enabled) + bool direct_render_enabled_{false}; + // Display controller detection DisplayController display_controller_{DisplayController::UNKNOWN}; @@ -1143,6 +1268,26 @@ class M5StackTab5 : public BaseComponent { // Usually ISR context; the DPI driver's CPU copy path calls it synchronously // on the issuing task, so it picks the semaphore API by xPortInIsrContext(). void complete_transfer(BaseType_t *higher_priority_task_woken); + // Frame-done ISR (on_frame_buf_complete, on_refresh_done before ESP-IDF + // v5.5.5 / v6.0.3): the DMA has restarted on the frame buffer queued last. + // Gives frame_done_sem_ for present_frame(). + static bool notify_frame_done(esp_lcd_panel_handle_t panel, esp_lcd_dpi_panel_event_data_t *edata, + void *user_ctx); + // Given once per panel refresh by notify_frame_done() (only with more than + // one frame buffer); present_frame() waits on it + SemaphoreHandle_t frame_done_sem_{nullptr}; + // Serializes present_frame() end to end (queue the switch + wait for the + // frame-done event): the event has one token and one waiter per switch. + std::mutex present_mutex_; + std::atomic vsync_enabled_{true}; + // A stalled panel logs one present_frame() timeout, not one per frame + std::atomic vsync_timeout_logged_{false}; + // Consecutive present_frame() timeouts (only present_frame() touches it, + // under present_mutex_); at kVsyncTimeoutsBeforeDisable the wait is + // switched off so a frame-done event that never comes cannot hold the UI + // at one frame per timeout for good. + static constexpr int kVsyncTimeoutsBeforeDisable = 3; + int vsync_timeouts_in_a_row_{0}; // DSI write helpers void dsi_write_command(uint8_t cmd, std::span params, uint32_t flags); diff --git a/components/m5stack-tab5/src/video.cpp b/components/m5stack-tab5/src/video.cpp index 4f40f06b35..fe97e069dc 100644 --- a/components/m5stack-tab5/src/video.cpp +++ b/components/m5stack-tab5/src/video.cpp @@ -57,18 +57,27 @@ static_assert(LV_COLOR_DEPTH == 16 && sizeof(M5StackTab5::Pixel) == sizeof(uint1 "The Tab5 video path (DPI panel config, PPA rotation color modes, buffer " "sizing) is RGB565-only; configure LVGL with LV_COLOR_DEPTH 16"); -// Number of framebuffers the DPI panel is created with (esp_lcd_dpi_panel_config_t::num_fbs, -// used for every controller variant below). The direct-to-framebuffer PPA -// rotation in flush() caches THE single framebuffer pointer at init and writes -// into it unconditionally; with more than one framebuffer the driver flips -// which buffer is scanned out, and rotating into a fixed one would -// intermittently update a non-visible buffer. That optimization is therefore -// only implemented for exactly one framebuffer: raising this value requires -// first teaching the direct-to-framebuffer path to track the active -// framebuffer (or removing it in favor of the always-correct scratch-buffer -// path), and a static_assert in initialize_lcd() enforces that at compile -// time rather than letting the stale-pointer bug ship. -static constexpr uint8_t kNumDpiFramebuffers = 1; +// Number of frame buffers the DPI panel is created with (num_fbs, the same for +// every controller variant below). More than one lets an application render +// straight into them and switch whole frames (see frame_buffers()). +static constexpr uint8_t kNumDpiFramebuffers = CONFIG_M5STACK_TAB5_DPI_NUM_FBS; +static_assert(kNumDpiFramebuffers >= 1 && kNumDpiFramebuffers <= M5StackTab5::max_frame_buffers, + "CONFIG_M5STACK_TAB5_DPI_NUM_FBS must be 1..3 (the DPI panel driver's limit)"); + +// Whether this ESP-IDF's DPI panel raises its frame-done event from the DMA +// transfer-done interrupt after re-arming on the queued frame buffer +// (on_frame_buf_complete: v5.5.5+, v6.0.3+). Earlier versions raise +// on_refresh_done from the DSI bridge's VSYNC interrupt on chips that have +// one (the P4), which is not synchronized with the re-arm; present_frame() +// then waits for two events. The DMA2D / frame-buffer draw_bitmap ordering +// bug of v6.0.0 is handled separately in initialize_lcd(). +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(6, 0, 3) || \ + (ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 5) && \ + ESP_IDF_VERSION < ESP_IDF_VERSION_VAL(6, 0, 0)) +static constexpr bool kFrameDoneIsBufferComplete = true; +#else +static constexpr bool kFrameDoneIsBufferComplete = false; +#endif // Alignment the PPA requires for its output buffer in external (PSRAM) memory: // both the buffer pointer and the buffer size must be multiples of the data @@ -199,7 +208,7 @@ bool M5StackTab5::initialize_lcd() { // (display_driver_ / display_controller_ need no reset here: display_driver_ // is .reset() unconditionally below and both are rewritten together before // the gate can open.) - dpi_framebuffer_ = nullptr; + dpi_framebuffer_.store(nullptr, std::memory_order_release); dpi_framebuffer_bytes_ = 0; if (!ioexp_0x43_) { @@ -393,6 +402,39 @@ bool M5StackTab5::initialize_lcd() { } } + // Keep the panel's frame buffers for frame_buffers() and for + // DisplayConfig::render_to_frame_buffers_enabled. (The driver reads only the first + // kNumDpiFramebuffers of the pointers passed.) + num_frame_buffers_ = 0; + frame_buffers_ = {}; + ret = esp_lcd_dpi_panel_get_frame_buffer(lcd_handles_.panel, kNumDpiFramebuffers, + &frame_buffers_[0], &frame_buffers_[1], + &frame_buffers_[2]); + if (ret != ESP_OK) { + logger_.error("Failed to get the DPI panel frame buffers: {}", esp_err_to_name(ret)); + frame_buffers_ = {}; + return false; + } + // DIRECT mode hands LVGL buffers 1 and 0 and present_frame() switches + // between them, so every one of the first kNumDpiFramebuffers pointers + // must be a real, distinct buffer; a null or duplicate would make two + // "buffers" the same memory. + for (size_t i = 0; i < kNumDpiFramebuffers; ++i) { + if (frame_buffers_[i] == nullptr) { + logger_.error("DPI panel frame buffer {} of {} is null", i, kNumDpiFramebuffers); + frame_buffers_ = {}; + return false; + } + for (size_t j = 0; j < i; ++j) { + if (frame_buffers_[j] == frame_buffers_[i]) { + logger_.error("DPI panel frame buffers {} and {} are the same buffer", j, i); + frame_buffers_ = {}; + return false; + } + } + } + num_frame_buffers_ = kNumDpiFramebuffers; + espp::display_drivers::Config display_config{ .panel_io = nullptr, .write_command = std::bind_front(&M5StackTab5::dsi_write_command, this), @@ -471,6 +513,29 @@ bool M5StackTab5::initialize_lcd() { return false; } } + if constexpr (kNumDpiFramebuffers > 1) { + // Frame-done event for present_frame(): the DMA has finished a frame and + // restarted on the frame buffer queued last (cur_fb_index), i.e. a queued + // switch has happened. ESP-IDF v5.5.5 / v6.0.3 / v6.1 renamed this + // callback from on_refresh_done (now deprecated) to on_frame_buf_complete + // and added a separate on_vsync; pick whichever this ESP-IDF has. + auto set_frame_done_callback = [](auto &callbacks, auto callback) { + if constexpr (requires { callbacks.on_frame_buf_complete; }) { + callbacks.on_frame_buf_complete = callback; + } else { + callbacks.on_refresh_done = callback; + } + }; + // Created before the callback is registered, so the ISR never sees null. + if (frame_done_sem_ == nullptr) { + frame_done_sem_ = xSemaphoreCreateBinary(); + } + if (frame_done_sem_ != nullptr) { + set_frame_done_callback(cbs, &M5StackTab5::notify_frame_done); + } else { + logger_.warn("Could not create the frame-done semaphore; present_frame() will not wait"); + } + } ret = esp_lcd_dpi_panel_register_event_callbacks(lcd_handles_.panel, &cbs, this); if (ret != ESP_OK) { logger_.error("Failed to register panel event callback: {}", esp_err_to_name(ret)); @@ -505,7 +570,21 @@ bool M5StackTab5::initialize_lcd() { // on which the underrun streaking this call addresses was reproduced, and // M5Stack's own Tab5 demo firmware ships with DMA2D enabled on production // units that are predominantly ST71xx. - if (display_controller_ != DisplayController::ILI9881) { + // + // ESP-IDF v6.0.0 only: its DPI draw_bitmap() checks the DMA2D hook before the + // "pointer is inside a frame buffer" test (fixed in v6.0.1), so with DMA2D on + // and 2+ frame buffers a frame-buffer switch would become a full-frame copy + // into the buffer on screen. Keep the CPU path there. +#if ESP_IDF_VERSION == ESP_IDF_VERSION_VAL(6, 0, 0) + constexpr bool dma2d_breaks_fb_switch = kNumDpiFramebuffers > 1; +#else + constexpr bool dma2d_breaks_fb_switch = false; +#endif + if constexpr (dma2d_breaks_fb_switch) { + logger_.info("ESP-IDF v6.0.0 with {} frame buffers: keeping CPU draw_bitmap copies (DMA2D " + "would turn a frame-buffer switch into a copy)", + kNumDpiFramebuffers); + } else if (display_controller_ != DisplayController::ILI9881) { ret = esp_lcd_dpi_panel_enable_dma2d(lcd_handles_.panel); if (ret != ESP_OK) { // Not fatal: draw_bitmap falls back to the (slower) CPU copy. @@ -522,22 +601,15 @@ bool M5StackTab5::initialize_lcd() { // with the PPA (see flush()). The panel scans this buffer out continuously; // the espp draw/flush path itself never writes it with the CPU. // - // This optimization is only valid with a single DPI framebuffer: caching one - // fixed pointer assumes the panel scans that same buffer forever. With - // num_fbs > 1 the driver flips between buffers and the PPA could rotate into - // one that is not being scanned out. Multi-framebuffer operation is - // intentionally unsupported until this path learns to track the active - // framebuffer, and the static_assert below turns raising - // kNumDpiFramebuffers without doing that work into a compile-time error - // instead of an intermittent visual glitch. - static_assert(kNumDpiFramebuffers == 1, - "The direct-to-framebuffer PPA rotation caches a single framebuffer pointer; " - "with multiple DPI framebuffers it must track the active one instead"); + // With several DPI framebuffers the driver scans whichever one was last + // handed to draw_bitmap(): nothing switches buffers except present_frame(), + // which keeps dpi_framebuffer_ pointing at the buffer it switched to, so the + // PPA rotation always targets the scanned-out buffer whatever the count. + // (An application that presents its own frames does not use the LVGL + // partial-mode flush at the same time, and DIRECT mode never rotates.) { - void *fb = nullptr; - if (esp_lcd_dpi_panel_get_frame_buffer(lcd_handles_.panel, kNumDpiFramebuffers, &fb) == - ESP_OK && - fb != nullptr) { + void *fb = frame_buffers_[0]; + if (fb != nullptr) { const size_t fb_bytes = static_cast(display_width_) * display_height_ * sizeof(Pixel); // The PPA requires its output buffer pointer and size to be aligned to // the data cache line (see kPpaOutBufferAlignment). The esp_lcd driver @@ -550,7 +622,7 @@ bool M5StackTab5::initialize_lcd() { // explores a configuration where the macro folds to 1, trivializing the // checks. On real configurations both checks are meaningful. if ((reinterpret_cast(fb) % cache_align) == 0 && (fb_bytes % cache_align) == 0) { - dpi_framebuffer_ = fb; + dpi_framebuffer_.store(fb, std::memory_order_release); dpi_framebuffer_bytes_ = fb_bytes; } else { logger_.warn("DPI framebuffer not cache-line aligned; PPA will rotate via scratch buffer"); @@ -639,22 +711,94 @@ static size_t third_buffer_bytes = 0; static ppa_client_handle_t g_ppa_client = nullptr; bool M5StackTab5::initialize_display(size_t pixel_buffer_size) { + return initialize_display(DisplayConfig{.pixel_buffer_size = pixel_buffer_size}); +} + +bool M5StackTab5::initialize_display(const DisplayConfig &config) { + if (display_ && direct_render_enabled_ != config.render_to_frame_buffers_enabled) { + logger_.error("The LVGL display is already initialized {} the frame buffers", + direct_render_enabled_ ? "on" : "without"); + return false; + } + const Display::LvglConfig lvgl_config{ + .width = display_width_, + .height = display_height_, + .flush_callback = std::bind_front(&M5StackTab5::flush, this), + // Forward LVGL rotation changes to the display driver so panels that can + // rotate in hardware (ST7121) do so via MADCTL instead of the PPA / + // software rotation in flush(). + .rotation_callback = std::bind_front(&M5StackTab5::on_display_rotation, this), + .rotation = rotation}; + const Display::OledConfig oled_config{ + .set_brightness_callback = + [this](float brightness) { this->brightness(brightness * 100.0f); }, + .get_brightness_callback = [this]() { return this->brightness() / 100.0f; }}; + + if (config.render_to_frame_buffers_enabled) { + if (num_frame_buffers_ < 2) { + logger_.error("render_to_frame_buffers_enabled needs initialize_lcd() first and " + "CONFIG_M5STACK_TAB5_DPI_NUM_FBS >= 2 ({} frame buffers)", + num_frame_buffers_); + return false; + } + // DIRECT mode renders straight into the panel's frame buffers, whose rows + // are exactly display_width_ pixels apart. LVGL lays its draw buffers out + // with its own stride rule (LV_DRAW_BUF_STRIDE_ALIGN): 720 RGB565 pixels + // are 1440 bytes, a multiple of neither 64 nor 128, so a raised stride + // alignment would silently skew every row. Refuse rather than draw skewed. + // The stride comes from LVGL's draw-buffer handlers, which lv_init() + // installs; espp::Display calls lv_init() itself, but only below, and + // before that the query returns 0. lv_init() is idempotent. + lv_init(); + const uint32_t lvgl_stride = + lv_draw_buf_width_to_stride(display_width_, LV_COLOR_FORMAT_RGB565); + if (lvgl_stride != display_width_ * sizeof(Pixel)) { + logger_.error( + "render_to_frame_buffers_enabled needs LVGL's row stride ({} bytes) to equal the " + "panel's ({} bytes); set LV_DRAW_BUF_STRIDE_ALIGN so it does", + lvgl_stride, display_width_ * sizeof(Pixel)); + return false; + } + // LVGL cannot rotate in DIRECT mode (it renders in place, and the PPA / + // software rotation of flush() has no separate draw buffer to rotate + // from), so the display must come up unrotated and stay so. + if (rotation != DisplayRotation::LANDSCAPE || + requested_rotation_.load(std::memory_order_seq_cst) != DisplayRotation::LANDSCAPE) { + logger_.error( + "render_to_frame_buffers_enabled needs the display rotation to be 0 (LANDSCAPE); " + "LVGL cannot rotate in DIRECT mode"); + return false; + } + logger_.info("Initializing LVGL display in DIRECT mode on the DPI frame buffers"); + if (!display_) { + // LVGL renders its first frame into vram0. The panel scans frame buffer 0 + // out from the start, so hand LVGL buffer 1 first: the first frame is + // drawn off screen like every later one. (With 3 frame buffers LVGL uses + // buffers 1 and 0.) + display_ = std::make_shared>( + lvgl_config, oled_config, + Display::StaticMemoryConfig{ + .pixel_buffer_size = display_width_ * display_height_, + .vram0 = static_cast(frame_buffers_[1]), + .vram1 = static_cast(frame_buffers_[0]), + }, + Logger::Verbosity::WARN); + // espp::Display sets the buffers up for PARTIAL rendering; render whole + // frames in place instead (nothing has been rendered yet). + lv_display_set_render_mode(display_->get_lvgl_display(), LV_DISPLAY_RENDER_MODE_DIRECT); + direct_render_enabled_ = true; + } + // No PPA client and no rotation scratch buffer: flush() does not rotate in + // DIRECT mode, and LVGL's buffers are the frame buffers themselves. + logger_.info("LVGL display initialized"); + return true; + } + + const size_t pixel_buffer_size = config.pixel_buffer_size; logger_.info("Initializing LVGL display with pixel buffer size: {} pixels", pixel_buffer_size); if (!display_) { display_ = std::make_shared>( - Display::LvglConfig{.width = display_width_, - .height = display_height_, - .flush_callback = std::bind_front(&M5StackTab5::flush, this), - // Forward LVGL rotation changes to the display driver so - // panels that can rotate in hardware (ST7121) do so via - // MADCTL instead of the PPA / software rotation in flush(). - .rotation_callback = - std::bind_front(&M5StackTab5::on_display_rotation, this), - .rotation = rotation}, - Display::OledConfig{ - .set_brightness_callback = - [this](float brightness) { this->brightness(brightness * 100.0f); }, - .get_brightness_callback = [this]() { return this->brightness() / 100.0f; }}, + lvgl_config, oled_config, Display::DynamicMemoryConfig{ .pixel_buffer_size = pixel_buffer_size, .double_buffered = true, @@ -981,6 +1125,116 @@ void M5StackTab5::write_lcd_lines(int xs, int ys, int xe, int ye, const uint8_t draw_and_wait(xs, ys, xe + 1, ye + 1, data); } +bool M5StackTab5::present_frame(const void *frame_buffer, uint32_t timeout_ms) { + // frame_buffers() is empty until the lcd_initialized_ gate is open + // (acquire), so this also refuses a panel that is not fully initialized. + // Any pointer inside a frame buffer selects that buffer, as the driver's + // draw_bitmap does. + const auto fbs = frame_buffers(); + const auto *p = static_cast(frame_buffer); + const auto owner = std::find_if(fbs.begin(), fbs.end(), [p](void *fb) { + const auto *base = static_cast(fb); + return base != nullptr && p >= base && p < base + frame_buffer_size(); + }); + if (frame_buffer == nullptr || owner == fbs.end()) { + logger_.error("present_frame() needs a pointer inside one of the panel's frame buffers"); + return false; + } + // With a single frame buffer there is nothing to switch to and no frame + // that is safe to draw into afterwards: say so instead of returning a + // "switched" that would hide the tearing. (Rate-limited: an application may + // call this once per frame.) The DIRECT-mode flush never gets here with one + // buffer, initialize_display() refuses that configuration up front. + if (fbs.size() < 2) { + logger_.error_rate_limited("present_frame() needs CONFIG_M5STACK_TAB5_DPI_NUM_FBS >= 2 (one " + "frame buffer: nothing to switch)"); + return false; + } + // One presenter at a time: the frame-done semaphore has a single token and + // a single waiter per switch, so a second caller must not steal this one's + // event. (The LVGL flush is the only presenter in DIRECT mode; this guards + // an application that drives the frame buffers itself from another task.) + std::lock_guard present_lock(present_mutex_); + { + // Queue the switch: for a frame buffer the driver writes the cache back, + // makes it the next one to scan out (cur_fb_index) and calls + // on_color_trans_done synchronously, so draw_and_wait() returns at once. + // The DMA takes the new buffer up when it restarts at the end of the + // frame it is sending. Serialized with write_lcd_lines() like every other + // panel draw; the frame-done wait below is outside the mutex, so a direct + // write is not held up for a refresh. + std::lock_guard lock(panel_op_mutex_); + // Like every panel operation: wait for the transfer in flight (an + // asynchronous LVGL flush copy or PPA rotation) before issuing one, so + // the token accounting in wait_for_transfer_idle() / draw_and_wait() + // stays exact. The driver completes a frame-buffer draw synchronously + // through on_color_trans_done, which gives the token from task context. + if (!wait_for_transfer_idle()) { + logger_.error("present_frame(): a previous transfer is wedged; not switching"); + return false; + } + if (!draw_and_wait(0, 0, display_width_, display_height_, frame_buffer)) { + logger_.error("present_frame(): draw_bitmap failed"); + return false; + } + // The panel now scans (or is about to scan) this buffer: point the PPA + // rotation path at it (see initialize_lcd()). Only if that path was + // enabled at init, i.e. the buffers passed its alignment check. + if (dpi_framebuffer_.load(std::memory_order_relaxed) != nullptr) { + dpi_framebuffer_.store(*owner, std::memory_order_release); + } + } + if (!vsync_enabled_.load(std::memory_order_relaxed) || frame_done_sem_ == nullptr) { + return true; + } + // Drop a frame-done event that came before this point: it may have + // restarted the DMA on the previous buffer. Any later one restarts it on + // the new buffer. (If the switch itself landed between draw_bitmap() and + // here, this waits one refresh longer, which is safe.) + xSemaphoreTake(frame_done_sem_, 0); + // How many events prove the switch happened. With on_frame_buf_complete + // (ESP-IDF v5.5.5 / v6.0.3 and later) the driver raises the event from the + // DMA transfer-done interrupt right after it re-armed on cur_fb_index, so + // one event after the draw is the switch. Before that the P4 raises + // on_refresh_done from the DSI bridge's VSYNC interrupt, and the DMA fetch + // of a frame completes a porch BEFORE that frame's VSYNC: the first VSYNC + // after the draw may belong to a frame whose re-arm preceded the draw + // (still the old buffer). The DMA re-arms exactly once between two + // consecutive VSYNCs, so the second VSYNC after the draw is past a re-arm + // that saw the new index; one refresh of extra latency on those versions. + constexpr int kFrameDoneEventsForSwitch = kFrameDoneIsBufferComplete ? 1 : 2; + bool switched = true; + for (int i = 0; i < kFrameDoneEventsForSwitch; ++i) { + if (xSemaphoreTake(frame_done_sem_, pdMS_TO_TICKS(timeout_ms)) != pdTRUE) { + switched = false; + break; + } + } + if (switched) { + vsync_timeouts_in_a_row_ = 0; + return true; + } + // A panel that stopped refreshing must not freeze the caller: the switch + // stays queued and happens whenever the DMA restarts. Each timeout costs + // the caller timeout_ms, so a frame-done event that never comes (a panel + // or ESP-IDF combination whose callback does not fire) would otherwise hold + // the UI at 1000 / timeout_ms frames per second for good. After a few + // consecutive timeouts the wait is switched off: present_frame() then + // queues the switch and returns at once (frames may tear, as with + // set_vsync_enabled(false)); set_vsync_enabled(true) arms it again. + if (!vsync_timeout_logged_.exchange(true, std::memory_order_relaxed)) { + logger_.warn("present_frame(): no frame-done event within {} ms (logged once)", timeout_ms); + } + if (++vsync_timeouts_in_a_row_ >= kVsyncTimeoutsBeforeDisable) { + vsync_timeouts_in_a_row_ = 0; + vsync_enabled_.store(false, std::memory_order_relaxed); + logger_.warn("present_frame(): {} consecutive timeouts; the frame-done wait is now disabled " + "(frames may tear); set_vsync_enabled(true) to re-arm it", + kVsyncTimeoutsBeforeDisable); + } + return false; +} + void M5StackTab5::brightness(float brightness) { brightness = std::clamp(brightness, 0.0f, 100.0f); if (backlight_) { @@ -1036,6 +1290,39 @@ void M5StackTab5::flush(lv_display_t *disp, const lv_area_t *area, uint8_t *px_m // own thread. flush() runs on the LVGL (GUI) thread, so reading it here is // safe; the camera task reads the cached atomic instead. camera_display_rotation_.store(static_cast(rotation), std::memory_order_relaxed); + + if (direct_render_enabled_) { + // DIRECT mode (DisplayConfig::render_to_frame_buffers_enabled): LVGL has rendered + // the dirty areas in place, into the frame buffer px_map points to, and + // calls this once per area. When the last area is done, present_frame() + // switches the panel to that buffer (no copy). LVGL then renders the next + // frame into the other buffer. + static bool rotation_warned = false; // flush() only runs on the LVGL task + if (rotation != LV_DISPLAY_ROTATION_0 && !rotation_warned) { + logger_.warn("Display rotation is not supported when rendering into the frame buffers"); + rotation_warned = true; + } + if (lv_display_flush_is_last(disp)) { + // The return value is deliberately not acted on: on a timeout the switch + // is still queued and LVGL carries on (that frame may tear); after a few + // consecutive timeouts present_frame() disables its wait itself. + // Waits until the panel scans px_map out, so LVGL never renders the + // next frame into the buffer the panel is still reading (see + // present_frame()). Two costs are inherent to DIRECT mode with two + // buffers and deliberately not optimized here: LVGL copies the previous + // frame's dirty areas between the buffers on the CPU before each render + // (refr_sync_areas), and the switch writes the whole frame's cache back + // - the sync copy dirties lines outside this frame's areas, so a + // per-area write-back would have to union the previous frame's areas. + // This flush never leaves a transfer in flight (lvgl_flush_pending_ is + // not set): the switch completes synchronously and LVGL is released + // right here. + present_frame(px_map); + } + lv_display_flush_ready(disp); + return; + } + // When the panel itself applies the rotation (ST7121, 180 degrees via the // display driver's MADCTL scan flip - see on_display_rotation()), the // logical frame is written to the framebuffer unrotated and at unrotated @@ -1056,7 +1343,10 @@ void M5StackTab5::flush(lv_display_t *disp, const lv_area_t *area, uint8_t *px_m offsetx2 = rotated_area.x2; offsety1 = rotated_area.y1; offsety2 = rotated_area.y2; - if (g_ppa_client != nullptr && dpi_framebuffer_ != nullptr) { + // The scanned-out framebuffer (present_frame() moves it when it switches + // buffers; see initialize_lcd()). + void *const target_framebuffer = dpi_framebuffer_.load(std::memory_order_acquire); + if (g_ppa_client != nullptr && target_framebuffer != nullptr) { // Hardware rotation via the PPA, writing the rotated block DIRECTLY into // the DPI panel's framebuffer at the rotated offset (the PPA output // supports placing a block inside a larger picture). This halves the @@ -1084,7 +1374,7 @@ void M5StackTab5::flush(lv_display_t *disp, const lv_area_t *area, uint8_t *px_m srm.in.block_w = ww; srm.in.block_h = hh; srm.in.srm_cm = PPA_SRM_COLOR_MODE_RGB565; - srm.out.buffer = dpi_framebuffer_; + srm.out.buffer = target_framebuffer; srm.out.buffer_size = dpi_framebuffer_bytes_; srm.out.pic_w = display_width_; srm.out.pic_h = display_height_; @@ -1295,6 +1585,21 @@ void IRAM_ATTR M5StackTab5::complete_transfer(BaseType_t *higher_priority_task_w } } +bool IRAM_ATTR M5StackTab5::notify_frame_done(esp_lcd_panel_handle_t panel, + esp_lcd_dpi_panel_event_data_t *edata, + void *user_ctx) { + auto *tab5 = static_cast(user_ctx); + if (tab5 == nullptr || tab5->frame_done_sem_ == nullptr) { + return false; + } + // ISR context (the DMA transfer-done interrupt, or on chips with a DSI + // bridge VSYNC interrupt and ESP-IDF < v6.0.3, that one): only give the + // semaphore. Returning true asks the driver to yield to the woken task. + BaseType_t higher_priority_task_woken = pdFALSE; + xSemaphoreGiveFromISR(tab5->frame_done_sem_, &higher_priority_task_woken); + return higher_priority_task_woken == pdTRUE; +} + void M5StackTab5::dsi_write_command(uint8_t cmd, std::span params, uint32_t /*flags*/) { if (!lcd_handles_.io) {