Closed bug report

Connections: Connect button refused during reconnect back-off, and container relay HTTP 500 shown raw with no hint

Colby Knox · 1mo ago ·closed by John Lauer

ab 2.1.170 · Windows (DESKTOP-BAR6S0F, Hydrogen embedded) · related: #233

What I did

In Connections on DESKTOP-BAR6S0F, I pressed Connect on a cloud container server (gallia) that was auto-discovered from my account.

Expected

Either it connects, or ab says why it can't, for example "the container is asleep, open it" or "adom-bridge serve isn't running in the container".

What happened

errorConnect failed: Server 'gallia' is already connected or connecting
gallia connecting [gallia] Connecting to wss://<container>/proxy/8765/...
gallia error      [gallia] Connection error: HTTP error: 500 Internal Server Error
gallia reconnecting [gallia] Reconnecting in 30s...

There are two problems here:

  1. The Connect button does nothing while ab waits to retry. The server sits in a 30 s back-off loop, so a manual Connect is refused as "already connected or connecting". The user can't make it retry now. Connect should cancel the back-off and dial right away.
  2. A raw HTTP 500 gives the user nothing to act on. The container's port proxy returns 500 when nothing is listening on 8765, which happens when the container is stopped or asleep or the relay isn't running. ab should map this to a hint such as "nothing is answering on the container's relay port: open the container, or run adom-bridge serve --supervise in it". The card should also show that state, not just keep retrying.

Also worth considering

The account has 8 containers named gallia, and the auto-discovered entry shows only the name. The card should show the slug or created date so the user can tell which container it points at.

6 Replies

John Lauer · 1mo ago

Shipped in 2.1.171 (insiders now).

  1. Connect during the back-off. A Connect click while ab waits to retry no longer answers "already connected or connecting". It wakes the waiting loop, which dials at once ("Retrying now (Connect pressed)" in the log). This also clears a held auth failure, so the 5-minute hold after a rejected token can be skipped by fixing the token and pressing Connect, as the log line always claimed. A live connection still answers "already connected".
  2. HTTP 500 explained. When a container's proxy answers 500/502/503/504, the log adds: "nothing is listening on its relay port. The container may be stopped or asleep (open it), or the relay is not running in it (run adom-bridge serve --supervise there)".
  3. Same-named containers. When several rows share a name, each row shows the container's host label next to the name, and hovering the name shows the full URL.

Checked by unit tests (the 5xx hint) and by building both changes. I have not reproduced the back-off click on a live box, so this stays open until you've confirmed it on DESKTOP-BAR6S0F with 2.1.171.

Colby Knox · 1mo ago

Sorry for the slow reply. Tested on DESKTOP-BAR6S0F, ab 2.1.172, driven remotely over the mesh, so I could not click the GUI button myself.

Setup: I added a temporary server zz-test-234 pointing at a dead container port (/proxy/8799/ returns 500 from the proxy) and let it fail into back-off. server_list showed it status: "disconnected", connected: false. I removed it afterwards.

What I saw from the verb side:

  • desktop_connect_all returned alreadyConnected: 3, spawned: 0, totalEnabled: 3. It counted the backed-off entry as "already connected" and did not wake its loop, which is the same problem as part 1 of this issue on the programmatic path. The GUI Connect fix may not have reached desktop_connect_all (or server_add on an existing unconnected entry, which also answered connected: true without a visible retry).
  • server_list doesn't expose the server's last error or retry state, so I couldn't check the new 5xx hint remotely. Adding lastError / nextRetryAt to server_list rows would make both fixes checkable without the GUI.

Not verified yet:

  • The GUI Connect click during back-off ("Retrying now (Connect pressed)").
  • The 5xx hint text in the log.
  • The host label on same-named rows.

Colby will check those in the GUI on DESKTOP-BAR6S0F and confirm here.

Colby Knox · 1mo ago

Follow-up: I pulled ab's own log from DESKTOP-BAR6S0F (2.1.172) for the zz-test-234 test above.

Fix 2 (5xx hint): confirmed. Every failed attempt now logs:

[WARN] [zz-test-234] The container answered HTTP 500: nothing is listening on its relay port. The container may be stopped or asleep (open it), or the relay is not running in it (run `adom-bridge serve --supervise` there). ab keeps retrying; press Connect to retry at once.

Fix 1 (Connect during back-off): the programmatic paths don't wake the loop.

20:59:28 [zz-test-234] Reconnecting in 16s...
20:59:43 server_add: zz-test-234 already connected; leaving live connection in place
20:59:47 [zz-test-234] Connecting to ...      <- the scheduled 16 s retry, not a wake
  • server_add on an entry that's in back-off logs "already connected" and leaves it waiting.
  • desktop_connect_all counts it in alreadyConnected and logs nothing.

The GUI Connect button itself is still untested: ab is hidden under Hydrogen on that box, and someone is using the laptop. Could server_add (existing unconnected entry) and desktop_connect_all use the same "wake the waiting loop" path as the button? Then an AI can unstick a backed-off server remotely, which is the case that matters when nobody is at the box.

Colby Knox · 1mo ago

I tested the GUI Connect button on DESKTOP-BAR6S0F (2.1.172) by driving the ab window through ab itself. The fix works:

21:09:36 [zz-test-234] Reconnecting in 30s...
21:09:52 connect_server: 'zz-test-234' is between reconnect attempts; retrying now
21:09:53 [zz-test-234] Connect pressed during back-off: dialing now
21:09:53 [zz-test-234] Connecting to ...

That retry came 16 s into a 30 s wait, with no "already connected or connecting".

Status of the three fixes:

  1. Connect during back-off: confirmed on the row's Connect button.
    • The row button changes with state. While the loop is mid-dial (yellow dot) it shows Disconnect, and only during the wait (red dot) does it show Connect. That's fine, but it explains how I hit "already connected or connecting" originally.
    • The programmatic paths still don't wake the loop: server_add and desktop_connect_all, as in my comment above.
    • Connect All (clicked via UIA at 21:09:24) gave no clear "retrying now" line. The next attempt came at 21:09:36, so I can't tell whether it woke the loop or not.
  2. 5xx hint: confirmed in the log.
  3. Host label on same-named rows: not tested. server_add upserts by name, so I couldn't create two rows with the same name. On a box with only one gallia row, the gallia row shows a Colby-ROG2 chip. Is that the connected-client chip, not the host label?

The test server is removed. Fine to close once the programmatic Connect path is either fixed or split into its own issue.

John Lauer · 1mo ago

Thanks for the thorough test. You were right: only the row's Connect button was fixed. 2.1.173 (insiders now) routes every "connect this server" path through one shared function, so the fix can't land on one door and miss the others:

  • desktop_connect_all wakes a server that is waiting out its back-off, and the reply says what it did: {spawned, woken:[names], dialing:[names], alreadyConnected}.
  • Connect All (GUI) and server_add on an existing entry use the same path.
  • The row button behaves as before: it wakes a waiting server and refuses only a live or mid-dial one.

server_list rows now carry lastError (with the plain-language hint for a container proxy 5xx) and nextRetryInSec, so all of this can be checked remotely.

Checked on arav-rog (2.1.173), the same shape as your test:

  • Added zz-test-234 pointing at a dead local port and let it back off. server_list showed the countdown and lastError: "no address of 127.0.0.1:9 accepted a TCP connection".
  • With 23 s left on its wait, desktop_connect_all answered woken: ["zz-test-234"], and the server was dialing again 0.9 s later.
  • The test server has been removed.

On your question about the chip next to the single gallia row: yes, that's the connected-client chip (the peers that relay sees). The host label only appears when two or more rows share a name, which server_add can't produce because it upserts by name. It mainly matters for auto-discovered containers.

All three parts are now confirmed: Connect during back-off on every path, the 5xx hint, and the duplicate-name label (by code). Closing; reopen if anything misbehaves.

John Lauer closed this · 1mo ago
Colby Knox · 1mo ago

Confirmed on DESKTOP-BAR6S0F with 2.1.173, remotely and using commands only. A test server backed off with nextRetryInSec: 24, and lastError carried the 5xx hint. desktop_connect_all answered woken: ["zz-test-234"]. 2 s later the wait had reset to nextRetryInSec: 33, which means it re-dialed straight away and started a fresh back-off. Test server removed. Thanks!

Log in to reply.