Show a live picture on a camera tile, run an action from the alert button, roll long media titles, and keep full-page tiles in their own colour (0.2.91)

Live picture (issue #4): a camera or image tile on a Guition takes
Display -> Live picture with a pace of 15 or 30 s. The camera tiles of a
page share one download: the screen asks ESP Screens for them together
(esphome.screen_camera with tiles, size and bg) and the app serves one
BMP, a strip of squares with the corners rounded over each tile's own
colour; every tile draws its square out of it with LVGL's image offset
(LV_IMAGE_ALIGN_TOP_LEFT: the default centres a taller source). A third
online_image on the Guition holds the strip; it loads after the alert's
picture, a cover and the camera full screen, never under a card, in
standby or under a finger. The strip is served whole every time: ESPHome's
http_request logs a 304 as a failed request. Firmware 0.2.77.

Alert button: esp_screens_show_alert takes action and data; the app
performs the action once when the button's report (esphome.screen_alert,
ok) comes from any screen. show_alert keeps its seven fields, so every
firmware from 0.2.31 has it.

Media card and full-page media tile: a title or artist line that is too
wide rolls by (LV_LABEL_LONG_SCROLL_CIRCULAR) instead of ending in dots.

Full-page tiles are white or their own pastel like every other tile; the
tint in the state colour of firmware 0.2.62 to 0.2.76 is gone.

Host harness (27 checks) and bench Guition + local add-on on the Yellow:
docs/TEST_RESULTS_0291.md. CYD firmware 1,633,728 B, 89.0 %.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

screen_manager/app/camera_feed.py

@@ -11,6 +11,7 @@ that loads a link gets the last snapshot at once, however long the camera takes
 in the screen's main loop, where touch and drawing wait with it), and the load starts fetching the next snapshot.
 """
 import asyncio
+from email.utils import formatdate
 import hashlib
 import io
 import logging
@@ -33,6 +34,16 @@ COVER_DOMAINS = ('media_player',)
 COVER_MIN_FIRMWARE = (0, 2, 64)
 COVER_SIZES = (48, 320)  # the smallest and largest cover a screen may ask for, in pixels
 COVER_RADIUS_SHARE = 12  # the corner is a twelfth of the size, at least 4 px (media_card.h radius_for)
+# Live pictures on camera tiles (app 0.2.91, firmware 0.2.77): a tile with "display": "live" shows a small square of its
+# camera in the icon's place. The camera tiles of a page share one image: the screen asks for them together and the
+# app serves one strip of squares, top to bottom in the screen's order, each cut to the middle of its snapshot with
+# the corners rounded over that tile's own colour. A camera is fetched again only when its tile's pace (15 or 30 s)
+# has passed since the last fetch, so a page loading every 15 s leaves a 30 s camera alone in between.
+LIVE_MIN_FIRMWARE = (0, 2, 77)
+LIVE_SIZES = (24, 160)   # a square's side, in pixels
+LIVE_MAX_TILES = 6       # one page
+LIVE_REFRESH = (15, 30)  # the paces a tile may choose, in seconds
+LIVE_RADIUS_SHARE = 6    # a rounder corner than a cover's: the square is small
 PORT = 8098
 # A camera nobody loaded a picture of for this long is forgotten, its last snapshot with it.
 WATCH_SECONDS = 30
@@ -73,6 +84,30 @@ def can_show_cover(screen):
     return bool(screen) and screen.get('board') in BOXES and (screen_firmware(screen) or (0, 0, 0)) >= COVER_MIN_FIRMWARE
 
 
+def can_show_live(screen):
+    """A paired Guition with firmware that draws live pictures on camera tiles."""
+    return bool(screen) and screen.get('board') in BOXES and (screen_firmware(screen) or (0, 0, 0)) >= LIVE_MIN_FIRMWARE
+
+
+def live_request(request):
+    """(entities, size, grounds) a screen asked live pictures for, or None when it is not a request the app can serve:
+    `tiles` the camera tiles of its page as a comma-separated list, `size` the square's side within LIVE_SIZES and
+    `bg` one colour of six hex digits per tile, the tile's own, behind the corners."""
+    tiles, grounds = request.get('tiles'), request.get('bg')
+    if not isinstance(tiles, str) or not isinstance(grounds, str):
+        return None
+    entities, colours = tiles.split(','), grounds.split(',')
+    try:
+        size = int(request.get('size'))
+    except (TypeError, ValueError):
+        return None
+    if not 1 <= len(entities) <= LIVE_MAX_TILES or len(colours) != len(entities) or len(set(entities)) != len(entities):
+        return None
+    if not LIVE_SIZES[0] <= size <= LIVE_SIZES[1] or not all(supported(e) for e in entities) or not all(re.fullmatch(r'[0-9A-Fa-f]{6}', c) for c in colours):
+        return None
+    return entities, size, [int(c, 16) for c in colours]
+
+
 def cover_request(request):
     """(size, background) a screen asked for, or None when the request is not a cover request the app can serve:
     `size` in pixels within COVER_SIZES, `bg` six hex digits of the colour behind the corners."""
@@ -148,9 +183,59 @@ def encode_cover(raw, size, background):
     return out.getvalue()
 
 
+def _square(image, size, background, radius_share):
+    """The middle of `image` as a square of `size` pixels with rounded corners over `background` (0xRRGGBB)."""
+    from PIL import Image, ImageDraw
+    width, height = image.size
+    side = min(width, height)
+    if width != height:
+        left, top = (width - side) // 2, (height - side) // 2
+        image = image.crop((left, top, left + side, top + side))
+    if image.size != (size, size):
+        image = image.resize((size, size), Image.Resampling.LANCZOS)
+    radius = max(4, size // radius_share)
+    mask = Image.new('L', (size, size), 0)
+    ImageDraw.Draw(mask).rounded_rectangle((0, 0, size - 1, size - 1), radius=radius, fill=255)
+    ground = Image.new('RGB', (size, size), tuple((background >> shift) & 0xFF for shift in (16, 8, 0)))
+    ground.paste(image, mask=mask)
+    return ground
+
+
+def _opaque(source, background):
+    """An RGB image from any picture Home Assistant hands out, transparency landing on `background`."""
+    from PIL import Image, ImageOps
+    image = ImageOps.exif_transpose(source)
+    if image.mode in ('RGBA', 'LA', 'P', 'PA'):
+        image = image.convert('RGBA')
+        ground = Image.new('RGB', image.size, tuple((background >> shift) & 0xFF for shift in (16, 8, 0)))
+        ground.paste(image, mask=image.getchannel('A'))
+        return ground
+    return image if image.mode == 'RGB' else image.convert('RGB')
+
+
+def encode_live(raws, size, grounds):
+    """The strip a page's camera tiles share: one square of `size` pixels per tile, top to bottom, each the middle of
+    its snapshot with rounded corners over that tile's colour (`grounds`, 0xRRGGBB each); a tile without a snapshot
+    (None) gets a plain square of its colour. A 24-bit BMP of `size` by `size` times the number of tiles."""
+    from PIL import Image
+    strip = Image.new('RGB', (size, size * len(raws)))
+    for n, (raw, background) in enumerate(zip(raws, grounds)):
+        if raw is None:
+            square = Image.new('RGB', (size, size), tuple((background >> shift) & 0xFF for shift in (16, 8, 0)))
+        else:
+            with Image.open(io.BytesIO(raw)) as source:
+                source.draft('RGB', (size * 2, size * 2))
+                square = _square(_opaque(source, background), size, background, LIVE_RADIUS_SHARE)
+        strip.paste(square, (0, n * size))
+    out = io.BytesIO()
+    strip.save(out, 'BMP')
+    return out.getvalue()
+
+
 class Watch:
     """One camera or media player: its last picture, the sizes made of it and the fetch on its way. For a media
-    player `picture` is the address the raw picture came from, so a changed picture is fetched again."""
+    player `picture` is the address the raw picture came from, so a changed picture is fetched again. `fetched_at`
+    is when the last fetch started: a live tile fetches again only when its own pace has passed."""
 
     def __init__(self, now):
         self.used = now
@@ -159,16 +244,18 @@ class Watch:
         self.frames = {}
         self.failures = 0
         self.retry_at = 0.0
+        self.fetched_at = 0.0
         self.task = None
 
 
 class Link:
-    __slots__ = ('entity', 'box', 'still', 'etag', 'used', 'lifetime', 'cover')
+    __slots__ = ('entity', 'box', 'still', 'etag', 'used', 'lifetime', 'cover', 'live')
 
-    def __init__(self, entity, box, now, still=None, cover=None):
+    def __init__(self, entity, box, now, still=None, cover=None, live=None):
         self.entity, self.box, self.used = entity, box, now
         self.still = still
         self.cover = cover  # (size, background) of a media player's cover, else None
+        self.live = live    # (entities, size, grounds, paces) of a page's live camera tiles, else None
         self.etag = f'"{hashlib.sha1(still).hexdigest()[:16]}"' if still else ''
         self.lifetime = STILL_SECONDS if still else LINK_SECONDS
 
@@ -180,6 +267,7 @@ class CameraFeed:
         self.fetch, self.clock = fetch, clock
         self.fetch_cover, self.picture = fetch_cover, picture
         self.watches, self.links = {}, {}
+        self.strips = {}
 
     # ----- fetching -----
     # A camera is fetched when a screen loads its picture: serving one starts fetching the next, so each load gets a
@@ -199,6 +287,7 @@ class CameraFeed:
     def refresh(self, entity, watch):
         """Start fetching the next snapshot, unless one is on its way or the camera failed a moment ago."""
         if (watch.task is None or watch.task.done()) and self.clock() >= watch.retry_at:
+            watch.fetched_at = self.clock()
             watch.task = asyncio.ensure_future(self.fetch_one(entity, watch))
 
     async def fetch_one(self, entity, watch):
@@ -258,6 +347,44 @@ class CameraFeed:
             cached = watch.frames[box] = (digest, image)
         return f'"{digest[:16]}-{box[0]}x{box[1]}"', cached[1]
 
+    # ----- live tiles -----
+    # A page's strip is made when the screen loads its link, out of the last snapshot of every tile's camera, and
+    # serving it starts the next fetch of the cameras whose pace has passed: a 15 s page fetches a 30 s camera every
+    # other load. The first load waits for the cameras that have no snapshot yet, all at once.
+    async def live_one(self, entity, pace, wait):
+        watch = self.watch(entity)
+        if watch.raw is None:
+            self.refresh(entity, watch)
+            if watch.task is not None and not watch.task.done():
+                try:
+                    await asyncio.wait_for(asyncio.shield(watch.task), wait)
+                except asyncio.TimeoutError:
+                    return None
+        elif self.clock() - watch.fetched_at >= pace:
+            self.refresh(entity, watch)
+        return watch.raw
+
+    async def live(self, entities, size, grounds, paces, wait=FIRST_FRAME_SECONDS):
+        """(etag, BMP, entities with '' where a camera has no snapshot) of the strip for a page's camera tiles at
+        `size` with `grounds` behind the corners and `paces` in seconds per tile, or None when no camera has one."""
+        raws = await asyncio.gather(*(self.live_one(entity, pace, wait) for entity, pace in zip(entities, paces)))
+        if all(raw is None for raw in raws):
+            return None
+        digests = [self.watches[entity].digest if raw is not None else '' for entity, raw in zip(entities, raws)]
+        key = ('live', size, tuple(grounds), tuple(digests))
+        tag = f'"{hashlib.sha1("|".join(digests).encode()).hexdigest()[:16]}-l{size}"'  # names the strip; not sent
+        cached = self.strips.get(key)
+        if cached is None:
+            try:
+                image = await asyncio.get_running_loop().run_in_executor(None, encode_live, raws, size, grounds)
+            except Exception as error:
+                LOG.info('The live pictures of %s cannot be read (%s)', ', '.join(entities), type(error).__name__)
+                return None
+            self.strips = {key: image}  # the last strip only: the next load makes another anyway
+        else:
+            image = cached
+        return tag, image, [entity if raw is not None else '' for entity, raw in zip(entities, raws)]
+
     # ----- covers -----
     # A media player's picture is fetched when a screen loads its cover link and Home Assistant's picture is another
     # one than the last fetched (or none was fetched yet): a cover never changes on its own between two loads, and the
@@ -321,12 +448,12 @@ class CameraFeed:
         while len(self.links) >= MAX_LINKS:
             del self.links[min(self.links, key=lambda token: self.links[token].used)]
 
-    def link(self, entity, box, still=None, cover=None):
+    def link(self, entity, box, still=None, cover=None, live=None):
         """A new random token for one camera at one size; `still` makes it one fixed image (an alert's), `cover`
-        (size, background) a media player's cover."""
+        (size, background) a media player's cover, `live` (entities, size, grounds, paces) a page's live tiles."""
         self.prune()
         token = secrets.token_urlsafe(18)
-        self.links[token] = Link(entity, box, self.clock(), still, cover)
+        self.links[token] = Link(entity, box, self.clock(), still, cover, live)
         return token
 
     async def serve(self, token, etag=None):
@@ -339,6 +466,13 @@ class CameraFeed:
         link.used = now
         if link.still:
             return (304, None, link.etag) if etag == link.etag else (200, link.still, link.etag)
+        if link.live:
+            # Always the whole strip, never a 304: ESPHome's http_request logs a 304 as a failed request and raises its
+            # error flag every time, and a page of slow cameras (a radar every five minutes) would do that every 15 s.
+            # The strip is small; the screen draws its squares again either way. The tag still goes out as an ETag:
+            # online_image warns at every load about a missing one.
+            found = await self.live(*link.live)
+            return (503, None, '') if found is None else (200, found[1], found[0])
         found = await self.cover(link.entity, *link.cover) if link.cover else await self.frame(link.entity, link.box)
         if found is None:
             return 503, None, ''
@@ -352,7 +486,9 @@ def web_app(feed):
 
     async def image(request):
         status, body, etag = await feed.serve(request.match_info['token'], request.headers.get('If-None-Match'))
-        headers = {'Cache-Control': 'no-cache', **({'ETag': etag} if etag else {})}
+        # Last-Modified with every picture: online_image warns at every load about a missing one (it never sends
+        # If-Modified-Since on its own; only the ETag decides a 304 here).
+        headers = {'Cache-Control': 'no-cache', 'Last-Modified': formatdate(usegmt=True), **({'ETag': etag} if etag else {})}
         if status == 200:
             return web.Response(body=body, content_type=CONTENT_TYPE, headers=headers)
         if status == 304:

screen_manager/app/claude_skill.py

@@ -13,13 +13,14 @@ import zipfile
 
 from i18n import t
 import tile_icons
-from core import (ALERT_CAMERA_FIELD, ALERT_ENDINGS, ALERT_EVENT, ALERT_FALLBACK_ICON, ALERT_FIELDS, ALERT_LIMITS, ALERT_MAX_TIMEOUT,
+from core import (ALERT_ACTION_FIELD, ALERT_CAMERA_FIELD, ALERT_ENDINGS, ALERT_EVENT, ALERT_FALLBACK_ICON, ALERT_FIELDS, ALERT_LIMITS, ALERT_MAX_TIMEOUT,
                   ALERT_MIN_FIRMWARE, ALERT_SUGGESTED_ICONS, AUTO_STANDBY_MIN_FIRMWARE, BROADCAST_DISMISS, BROADCAST_SHOW,
-                  CONTROLS, DISPLAYS, FULL_PAGE_MIN_FIRMWARE, MAX_PAGES, MAX_SLOTS, MAX_TILES, PAGE_TILE_REPEAT_MIN_FIRMWARE,
+                  CONTROLS, DISPLAYS, FULL_PAGE_MIN_FIRMWARE, LIVE_MIN_FIRMWARE, MAX_PAGES, MAX_SLOTS, MAX_TILES, PAGE_TILE_REPEAT_MIN_FIRMWARE,
                   SETTINGS_PAGE_MIN_FIRMWARE, SLOTS_PER_PAGE, TILE_BACKGROUNDS, TILE_EVENTS, TILE_RESULT_EVENT, WAKE_SLEEP_MIN_FIRMWARE,
                   SETTING_ENTITIES_MIN_FIRMWARE, DARK_MODE_MIN_FIRMWARE, PAGE_BUTTONS_MIN_FIRMWARE)
 # Full-page tiles, navigation tiles and forty-eight tiles per screen.
 FULL_PAGE_VERSION = '.'.join(str(part) for part in FULL_PAGE_MIN_FIRMWARE)
+LIVE_VERSION = '.'.join(str(part) for part in LIVE_MIN_FIRMWARE)
 # The same navigation tile on several pages (app 0.2.78).
 PAGE_TILE_REPEAT_VERSION = '.'.join(str(part) for part in PAGE_TILE_REPEAT_MIN_FIRMWARE)
 
@@ -99,7 +100,7 @@ actions:
 | `from_page`, `from_slot` | Only for `esp_screens_move_tile`: which copy of a navigation tile that is on several pages to move (see below). |
 | `size` | `single`, `wide` or `full` (the whole page; firmware {FULL_PAGE_VERSION} or newer). |
 | `controls` | What you can operate on the tile itself (see below). |
-| `display` | How the tile draws itself (see below). |
+| `display` | How the tile draws itself (see below). A camera or image tile on a Guition takes `live` (firmware {LIVE_VERSION} or newer): a small live picture in the icon's place, with `refresh` 15 or 30 (seconds). |
 | `icon`, `color` | An icon from the list further down, and one of the pastel colors. |
 | `tap` | What a tap does: `auto`, `detail` (open the card), `toggle`, `action` or `none`. `toggle` works for anything Home Assistant can toggle for that entity, such as a light, a cover (open, close, or stop while it moves) or a speaker that turns on and off. Holding the tile still opens its card. |
 | `action`, `data` | Perform action: an action Home Assistant offers for the tile's own entity, such as `cover.set_cover_position`, with `data` for its fields (`position: 50`). Giving `action` sets `tap` to `action`. The target is always the tile's entity. Only actions and fields Home Assistant lists for that entity are accepted; the answer says what is missing. |
@@ -197,6 +198,8 @@ actions:
 
 The event for every screen takes one more field, `{ALERT_CAMERA_FIELD[0]}`: {ALERT_CAMERA_FIELD[2]} Example: `{ALERT_CAMERA_FIELD[0]}: {ALERT_CAMERA_FIELD[3]}`. Use a real `camera.*` or `image.*` entity from this Home Assistant (a doorbell integration usually has one); the per-screen actions have no such field.
 
+It also takes `{ALERT_ACTION_FIELD[0]}` (app 0.2.91): {ALERT_ACTION_FIELD[2]} Example: `{ALERT_ACTION_FIELD[0]}: {ALERT_ACTION_FIELD[3]}`, or `action: light.turn_off` with `data: {{entity_id: light.hall}}`. Use an action Home Assistant lists; give `button_text` a word that says what the button does ("Open", "Turn off"). The per-screen actions have no such field either.
+
 Text limits are in bytes; an accented letter takes two. Keep the title short: a screen shows about twenty characters of it on one line and ends a longer title with an ellipsis. Put details in the subtitle.
 
 ## Colors

screen_manager/app/core.py

@@ -34,6 +34,9 @@ NEW_DOMAINS = frozenset('sun timer person screen'.split())
 # A camera or an image entity opens full screen on a Guition with firmware 0.2.57+ (camera_feed.py).
 CAMERA_DOMAINS = frozenset(('camera', 'image'))
 CAMERA_MIN_FIRMWARE = (0, 2, 57)
+# A live picture on a camera tile ("display": "live", app 0.2.91): firmware from here asks for the page's strip.
+LIVE_MIN_FIRMWARE = (0, 2, 77)
+LIVE_REFRESH = (15, 30)  # the paces a live tile may choose, in seconds; the first is the default
 # The media card with its cover (app 0.2.77): firmware from here draws it and asks for the cover.
 COVER_MIN_FIRMWARE = (0, 2, 64)
 # Forty-eight tiles (one per slot), a tile over the whole page and the screen.page tile (firmware 0.2.62+).
@@ -46,7 +49,7 @@ FIRST_MAX_TILES = 10
 REPO = 'https://github.com/MaxGramser/homeassistant_espscreen'
 REFS = {'cyd': 'main', 'guition': 'main'}
 # Firmware shipped with this app release; screens below it get an update offer.
-FIRMWARE_VERSION = '0.2.76'
+FIRMWARE_VERSION = '0.2.77'
 # The Auto standby switch a screen offers Home Assistant automations.
 AUTO_STANDBY_MIN_FIRMWARE = '0.2.41'
 # The settings page the screen opens itself, and the screen.settings tile that opens it.
@@ -85,7 +88,8 @@ def backgrounds():
     return {name: {**item, 'label': t(f'addon.labels.backgrounds.{name}')} for name, item in TILE_BACKGROUNDS.items()}
 
 # Display modes per domain; everything else offers standard and watch (large value).
-DISPLAYS = {'weather': ('standard', 'watch', 'forecast'), 'sensor': ('standard', 'watch', 'graph'), 'screen': ('digital', 'analog'), 'sun': ('standard', 'watch', 'sunpath')}
+DISPLAYS = {'weather': ('standard', 'watch', 'forecast'), 'sensor': ('standard', 'watch', 'graph'), 'screen': ('digital', 'analog'), 'sun': ('standard', 'watch', 'sunpath'),
+            'camera': ('standard', 'live'), 'image': ('standard', 'live')}
 # Displays that only work on a double-width card.
 WIDE_ONLY = ('forecast', 'sunpath')
 
@@ -482,6 +486,8 @@ def min_firmware(layout):
         return PAGE_TILE_REPEAT_MIN_FIRMWARE
     if len(layout['tiles']) > LEGACY_MAX_TILES or any(is_full(t) or page_target(t['entity']) for t in layout['tiles']):
         return FULL_PAGE_MIN_FIRMWARE
+    if any(t.get('options', {}).get('display') == 'live' for t in layout['tiles']):
+        return LIVE_MIN_FIRMWARE
     if any(t['entity'].split('.')[0] in CAMERA_DOMAINS for t in layout['tiles']):
         return CAMERA_MIN_FIRMWARE
     if any(t['entity'] == 'screen.settings' for t in layout['tiles']):
@@ -509,7 +515,7 @@ TILE_RESULT_EVENT = 'esp_screens_tile_result'
 # pastel background.
 TILE_EVENT_OPTIONS = {'size': 'size', 'controls': 'controls', 'display': 'display', 'icon': 'icon',
                       'color': 'background', 'background': 'background', 'tap': 'tap', 'inline': 'inline',
-                      'history_hours': 'history_hours'}
+                      'history_hours': 'history_hours', 'refresh': 'refresh'}
 TILE_SIZES = {'full': 'full', 'fullscreen': 'full', 'full screen': 'full', 'full-screen': 'full', 'page': 'full', 'whole page': 'full',
               'wide': 'wide', 'double': 'wide', 'large': 'wide', 'big': 'wide',
               'single': 'single', 'small': 'single', 'normal': 'single'}
@@ -590,7 +596,7 @@ def tile_options(data, current=None):
         if key not in data or data[key] in (None, ''):
             continue
         value = data[key]
-        if name == 'history_hours':
+        if name in ('history_hours', 'refresh'):
             options[name] = int(value) if str(value).isdigit() else value
         elif name == 'size':
             options[name] = TILE_SIZES.get(loose(value), str(value))
@@ -896,7 +902,7 @@ def validate_layout(data, stored=False):
                 continue
         if 'options' in tile:
             options = tile['options']
-            if not isinstance(options, dict) or set(options) - {'tap', 'display', 'inline', 'history_hours', 'background', 'size', 'icon', 'controls', 'action'}:
+            if not isinstance(options, dict) or set(options) - {'tap', 'display', 'inline', 'history_hours', 'background', 'size', 'icon', 'controls', 'action', 'refresh'}:
                 raise ValueError(t('addon.errors.layout.unknown_settings'))
             # A navigation tile (screen.page_<n>, firmware 0.2.62+) has a name, an icon, a colour and a width; never the page.
             if page_target(tile['entity']):
@@ -931,6 +937,12 @@ def validate_layout(data, stored=False):
                 raise ValueError(t('addon.errors.layout.no_mini_slider'))
             if 'history_hours' in options and (type(options['history_hours']) is not int or options['history_hours'] not in (1,6,24)):
                 raise ValueError(t('addon.errors.layout.history_hours'))
+            # A live picture's pace (app 0.2.91) belongs to the live display; another display leaves a stale one behind.
+            if options.get('display') == 'live':
+                if 'refresh' in options and (type(options['refresh']) is not int or options['refresh'] not in LIVE_REFRESH):
+                    raise ValueError(t('addon.errors.layout.refresh'))
+            elif 'refresh' in options:
+                options = {key: value for key, value in options.items() if key != 'refresh'}
             if options.get('display') == 'watch' and options.get('inline') == 'slider':
                 raise ValueError(t('addon.errors.layout.watch_or_slider'))
             if 'controls' in options:
@@ -1398,6 +1410,11 @@ ALERT_SUGGESTED_ICONS = ('doorbell', 'bell', 'bell-ring', 'alert-outline', 'alar
 # Not an argument of show_alert: the app sends the image itself to screens that can draw it (app 0.2.66, a Guition with
 # firmware 0.2.57+) and leaves it out for the others. (name, label, explanation, example) like ALERT_FIELDS.
 ALERT_CAMERA_FIELD = ('camera', 'Camera', 'A camera or image entity. A Guition with firmware 0.2.57+ shows its picture of that moment across the top of the card; a tap on it opens the camera full screen. Other screens show the alert without it. Only through the esp_screens_show_alert event.', 'camera.front_door')
+# An action behind the button (app 0.2.91): the event names a Home Assistant action, with data for its fields, that the
+# app performs when the button is pressed on any screen, once per alert. Not an argument of show_alert either: the
+# screen only reports the press (ALERT_EVENT, action "ok") and the app does the rest, so every firmware from 0.2.31 has it.
+ALERT_ACTION_FIELD = ('action', 'Action', 'A Home Assistant action, such as script.open_gate or light.turn_off, performed once when the button is pressed on any screen. `data` gives its fields (entity_id, brightness, ...). A timeout or a new alert leaves it unperformed. Only through the esp_screens_show_alert event.', 'script.open_gate')
+ALERT_ACTION_MAX_BYTES = 4096
 # The firmware's MAX_TIMEOUT_SECONDS.
 ALERT_MAX_TIMEOUT = 86400
 # One alert for every screen (app 0.2.45): an automation fires one of these Home Assistant events and the
@@ -1423,6 +1440,8 @@ def alert_reference():
                        for name, kind, _, _, value in ALERT_FIELDS],
             'camera': {'name': camera, 'label': t(f'addon.alerts.fields.{camera}.label'), 'help': t(f'addon.alerts.fields.{camera}.help'),
                        'example': ALERT_CAMERA_FIELD[3]},
+            'action': {'name': ALERT_ACTION_FIELD[0], 'label': t(f'addon.alerts.fields.{ALERT_ACTION_FIELD[0]}.label'),
+                       'help': t(f'addon.alerts.fields.{ALERT_ACTION_FIELD[0]}.help'), 'example': ALERT_ACTION_FIELD[3]},
             'limits': ALERT_LIMITS,
             'colors': [{'name': name, 'label': item['label'], 'color': item['color']} for name, item in backgrounds().items() if item['color']],
             'suggested_icons': [{'name': name, 'cp': tile_icons.GLYPHS[name]} for name in ALERT_SUGGESTED_ICONS],
@@ -1482,6 +1501,22 @@ def alert_camera(data):
     usable = isinstance(value, str) and re.fullmatch(r'[a-z0-9_]+\.[a-z0-9_]+', value.strip()) is not None and value.strip().split('.')[0] in CAMERA_DOMAINS
     return (value.strip(), True) if usable else ('', False)
 
+def alert_action(data):
+    """((action, data), usable): the alert's `action` and `data` when they name a Home Assistant action with a mapping of
+    fields; (None, True) without one, (None, False) when it is unusable."""
+    value = data.get(ALERT_ACTION_FIELD[0]) if isinstance(data, dict) else None
+    if value is None or value == '':
+        return None, True
+    fields = data.get('data')
+    if not isinstance(value, str) or not ACTION_NAME.fullmatch(value.strip()) or len(value) > 64 or (fields is not None and not isinstance(fields, dict)):
+        return None, False
+    try:
+        if len(json.dumps(fields or {}, ensure_ascii=False, allow_nan=False).encode()) > ALERT_ACTION_MAX_BYTES:
+            return None, False
+    except (TypeError, ValueError):
+        return None, False
+    return (value.strip(), dict(fields or {})), True
+
 def alert_targets(screens):
     """(ready, skipped): the paired screens that can show an alert now, and the others with the reason, in English for the
     log (the editor shows it in its own language, app 0.2.90).

screen_manager/app/ha_catalogue.py

@@ -170,6 +170,9 @@ def capabilities(entity_id, actions, state, services):
             displays.append('forecast')
     elif domain == 'sun':
         displays.append('sunpath')
+    elif domain in ('camera', 'image'):
+        # A live picture on the tile (app 0.2.91): the editor offers it on a Guition, the only board that draws images.
+        displays.append('live')
     return {
         'toggle': TOGGLE.format(domain=domain) in actions,
         'inline': domain in INLINE and _fits(INLINE[domain], actions, attributes, services),

screen_manager/app/server.py

@@ -19,7 +19,7 @@ import tile_icons
 from updates import Updater
 
 from aiohttp import ClientError, ClientSession, ClientTimeout, WSMsgType, web
-from core import BROADCAST_EVENTS, BROADCAST_SHOW, BUILTIN, CAMERA_DOMAINS, entity_id, SETTINGS_BESIDE_BLOCK, TILE_EVENTS, TILE_RESULT_EVENT, layout_snapshot, match_screen, HEADER_MIN_FIRMWARE, NAME_TILE_SETTINGS, TRANSPORT_MIN_FIRMWARE, alert_camera, alert_data, alert_reference, alert_service, alert_targets, backgrounds, builtin_name, controls_catalogue, device_prefixes, discover, discover_screens, encode, extras, forecast_kinds, header_items, inbox_prefix, message_action, min_firmware, packets, revision, screen_items, state_message, validate_header, validate_layout, validate_settings
+from core import ALERT_EVENT, BROADCAST_EVENTS, BROADCAST_SHOW, BUILTIN, CAMERA_DOMAINS, entity_id, SETTINGS_BESIDE_BLOCK, TILE_EVENTS, TILE_RESULT_EVENT, layout_snapshot, match_screen, HEADER_MIN_FIRMWARE, NAME_TILE_SETTINGS, TRANSPORT_MIN_FIRMWARE, alert_action, alert_camera, alert_data, alert_reference, alert_service, alert_targets, backgrounds, builtin_name, controls_catalogue, device_prefixes, discover, discover_screens, encode, extras, forecast_kinds, header_items, inbox_prefix, message_action, min_firmware, packets, revision, screen_items, state_message, validate_header, validate_layout, validate_settings
 from core import SETTING_ENTITIES, SETTING_RULES, setting_action, setting_entities, setting_from_state, state_word
 from core import PAGE_TILE_REPEAT_MIN_FIRMWARE, SLOTS_PER_PAGE, firmware_features, packed_slots, run_tile_event, screen_firmware, version_text
 import header_bar
@@ -215,8 +215,9 @@ class HomeAssistant:
                     # how the app talks to it.
                     self.esphome_services |= body.get('domain') == 'esphome'
                     self.services_changed.set()
-                elif event.get('event_type') in BROADCAST_EVENTS:
-                    # Queued, not handled here: the calls wait for results this reader has to deliver.
+                elif event.get('event_type') in BROADCAST_EVENTS or event.get('event_type') == ALERT_EVENT:
+                    # Queued, not handled here: the calls wait for results this reader has to deliver. A screen's own
+                    # report of an alert's end (app 0.2.91) goes the same way: the button may have an action behind it.
                     self.broadcasts.put_nowait((event['event_type'], body))
                 elif event.get('event_type') in TILE_EVENTS:
                     self.tile_events.put_nowait((event['event_type'], body))
@@ -382,7 +383,7 @@ class HomeAssistant:
                     await self.request('subscribe_events', event_type='esphome.screen_history')
                     await self.request('subscribe_events', event_type='esphome.screen_camera')
                     await self.request('subscribe_events', event_type=light_effects.OPTIONS_EVENT)
-                    for event_type in (*REGISTRY_EVENTS, *BROADCAST_EVENTS, *TILE_EVENTS, *SERVICE_EVENTS):
+                    for event_type in (*REGISTRY_EVENTS, *BROADCAST_EVENTS, ALERT_EVENT, *TILE_EVENTS, *SERVICE_EVENTS):
                         await self.request('subscribe_events', event_type=event_type)
                     self.states = {s['entity_id']: s for s in await self.request('get_states')}
                     await self.registries()
@@ -631,6 +632,8 @@ class Manager:
                                              fetch_cover=lambda entity: self.ha.media_image(entity),
                                              picture=lambda entity: self.ha.media_picture(entity))
         self.alert_cameras = {}
+        # The action behind an alert's button (app 0.2.91): node -> (key, action, data) of the alert on that screen.
+        self.alert_actions = {}
         if self.path.exists():
             raw = json.loads(self.path.read_text())
             # Versioned persistent data. Never silently overwrite an unknown schema.
@@ -1629,6 +1632,10 @@ class Manager:
         inbox = self.aliases.get(request.get('inbox'), request.get('inbox'))
         entity = request.get('entity')
         screen = self.screen(inbox) if isinstance(inbox, str) else None
+        # The live pictures of a page's camera tiles (app 0.2.91, firmware 0.2.77+): one strip for all of them.
+        if 'tiles' in request:
+            await self.answer_live(inbox, screen, request)
+            return
         # A media player's cover (app 0.2.77, firmware 0.2.64+) comes the same way, at the size the card asks for.
         cover = camera_feed.cover_request(request) if camera_feed.cover_supported(entity) else None
         if cover:
@@ -1643,6 +1650,34 @@ class Manager:
         await self.ha.send(inbox, message, action)
         LOG.info('%s %s on %s%s', 'Cover of' if cover else 'Camera', entity, screen['name'], '' if message['u'] else ': no image')
 
+    async def answer_live(self, inbox, screen, request):
+        """The strip for a page's live camera tiles: every tile the screen names must be a camera tile of its layout
+        set to a live picture; the pace of each comes from that tile's own setting."""
+        live = camera_feed.live_request(request)
+        if not live or not camera_feed.can_show_live(screen) or not screen.get('online'):
+            return
+        action = self.transport(inbox, screen)
+        if not action:
+            return
+        entities, size, grounds = live
+        tiles = {tile['entity']: tile.get('options') or {} for tile in self.layouts.get(inbox, {}).get('tiles', [])}
+        if any(entity not in tiles or tiles[entity].get('display') != 'live' for entity in entities):
+            LOG.info('Live pictures for %s: not the live camera tiles of %s', ', '.join(entities), screen['name'])
+            return
+        paces = [tiles[entity].get('refresh', camera_feed.LIVE_REFRESH[0]) for entity in entities]
+        url, listing = '', ','.join(entities)
+        base = await camera_feed.base_url(self.ha.request)
+        if base:
+            found = await self.camera.live(entities, size, grounds, paces)
+            if found:
+                listing = ','.join(found[2])
+                token = self.camera.link(listing, (size, size), live=(entities, size, grounds, paces))
+                url = f'{base}/camera/{token}.bmp'
+        else:
+            LOG.warning('Live pictures: no address for this app on the LAN; set SCREEN_CAMERA_URL')
+        await self.ha.send(inbox, {'v': 1, 'op': 'camera', 't': 'live', 'e': listing, 'u': url}, action)
+        LOG.info('Live pictures of %s on %s%s', ', '.join(entities), screen['name'], '' if url else ': no image')
+
     async def cover_message(self, entity, size, background):
         """The screen message with a link to a media player's cover at `size` with `background` behind the corners,
         or an empty link when the player shows no picture (the card keeps its placeholder)."""
@@ -1678,6 +1713,9 @@ class Manager:
         camera, usable = alert_camera(data) if event_type == BROADCAST_SHOW else ('', True)
         if not usable:
             unusable.append('camera')
+        button, usable = alert_action(data) if event_type == BROADCAST_SHOW else (None, True)
+        if not usable:
+            unusable.append('action')
         if unusable:
             LOG.warning('%s: unusable %s left empty', event_type, ', '.join(unusable))
         ready, skipped = alert_targets(self.screens())
@@ -1694,6 +1732,13 @@ class Manager:
         LOG.info('%s: %d of %d screens%s', event_type, len(ready) - len(failed), len(ready) + len(skipped),
                  f" (not: {'; '.join(notes)})" if notes else '')
         shown = [screen for screen, result in zip(ready, results) if not isinstance(result, BaseException)]
+        # The button's action waits on every screen that shows this alert; a dismissal forgets it everywhere.
+        for screen in ready:
+            self.alert_actions.pop(screen['node'], None)
+        if button and shown:
+            key = secrets.token_hex(4)
+            for screen in shown:
+                self.alert_actions[screen['node']] = (key, *button)
         if viewers and any(screen in shown for screen in viewers):
             await self.alert_images(camera, [screen for screen in viewers if screen in shown])
         return {'sent': len(ready) - len(failed), 'skipped': len(skipped), 'failed': len(failed)}
@@ -1758,10 +1803,31 @@ class Manager:
         while queue is not None:
             event_type, data = await queue.get()
             try:
-                await self.broadcast(event_type, data)
+                if event_type == ALERT_EVENT:
+                    await self.alert_ended(data)
+                else:
+                    await self.broadcast(event_type, data)
             except Exception as error:
                 LOG.warning('%s failed (%s)', event_type, type(error).__name__)
 
+    async def alert_ended(self, data):
+        """A screen reports how its alert ended (esphome.screen_alert). The button performs the action the alert
+        came with, once for the alert however many screens showed it; any other ending forgets it on that screen."""
+        node = data.get('screen') if isinstance(data, dict) else None
+        pending = self.alert_actions.pop(node, None) if isinstance(node, str) else None
+        if pending is None:
+            return
+        key, action, fields = pending
+        for other in [n for n, (k, _, _) in self.alert_actions.items() if k == key]:
+            del self.alert_actions[other]
+        if data.get('action') != 'ok':
+            return
+        try:
+            await self.ha.call(action, fields)
+            LOG.info('Alert button on %s: %s performed', node, action)
+        except Exception as error:
+            LOG.warning('Alert button on %s: %s failed (%s)', node, action, type(error).__name__)
+
 def create_app(manager, development=False):
     csrf = secrets.token_urlsafe(32)
     @web.middleware

screen_manager/app/static/index.html

@@ -5,8 +5,8 @@
     <meta name="viewport" content="width=device-width,initial-scale=1" />
     <meta name="color-scheme" content="light dark" />
     <title>ESP Screens</title>
-    <script type="module" crossorigin src="./assets/index-CLkdYPyY.js"></script>
-    <link rel="stylesheet" crossorigin href="./assets/index-B7AxjXTR.css">
+    <script type="module" crossorigin src="./assets/index-BwlmhXLI.js"></script>
+    <link rel="stylesheet" crossorigin href="./assets/index-DQxVdm_5.css">
   </head>
   <body>
     <div id="app"></div>