How I Made Code-OSS Support Remote AI Workspaces over SSH

GitHub repository: realreadpaper/coder

The Short Version

I was not building an SSH file editor. I wanted an AI development environment with correct remote semantics:

local desktop IDE
  -> SSH into a remote development machine
  -> open a remote directory as the workspace
  -> let Codex read, write, search, and execute there
  -> terminal, Git, Search, LSP, and Debug all run against the remote filesystem

SFTP and local mirrors are not enough for that. The final solution is a Code-OSS fork: reuse Workbench, Extension Host, Remote Agent, FileService, Terminal, Search, Git, LSP/DAP, then build the Remote-SSH product layer and AI remote execution layer ourselves.

One-line summary:

Reuse Code-OSS as the IDE platform.
Build Remote-SSH resolver, remote-ai-server, Codex remote bridge, approval, and sandbox.

Reading Path

This post follows the decision chain:

1. Why SFTP/local mirrors are not enough.
2. Why Code-OSS has to be the base.
3. How the Remote-SSH loop works.
4. How the remote server is packaged and verified.
5. The hardest Codex question: where does UI run, and where does execution run?
6. How ai-codex-remote-bridge moves execution to the remote host.
7. How approval, sandboxing, and verification close the loop.

1. The Goal Is Remote Semantics, Not SSH

The simplest idea is an SSH editor:

ssh list files
  -> read remote file
  -> edit locally
  -> save back over ssh
  -> run commands over ssh

That may be enough for a plain editor. It is not enough for Codex.

The Codex VS Code extension is not a simple CLI wrapper. It depends on VS Code platform capabilities:

• workspace.workspaceFolders
• workspace.fs
• commands.registerCommand
• Webview / sidebar / custom editor
• diagnostics / references / workspace symbols
• terminal
• Chat Sessions proposed API
• LSP MCP capabilities

With only SFTP, Codex may still see a local file workspace, a virtual URI scheme, or an incomplete VS Code API surface. A sidebar opening does not prove the agent is operating in the right directory.

The first key prompt was investigative:

Do not write code.
 
Analyze which VS Code APIs the OpenAI Codex extension depends on.
If we implemented a remote IDE from scratch, list the platform capabilities required.
Conclude:
- which parts must reuse Code-OSS
- which parts can be built ourselves
- which closed-source implementations must not be copied

That moved the plan from “write a remote editor” to “fork Code-OSS.”

2. Why Code-OSS

Code-OSS already provides the parts I should not rewrite:

• Workbench
• Extension Host
• FileService
• Webview
• Commands / Menus / Context keys
• Text model
• Terminal / PTY
• Search / ripgrep
• Git
• LSP / DAP
• Remote Agent protocol

The custom layers are:

our.remote-ssh
  -> SSH host selection
  -> remote server installation
  -> tunnel/local forwarding
  -> ResolvedAuthority
  -> diagnostics
 
remote-ai-server
  -> server tarball
  -> bootstrap
  -> manifest/releaseDoctor
  -> CentOS 7 ABI gate
 
ai-codex-remote-bridge
  -> remote Linux Codex CLI
  -> remote cwd / env
  -> approval
  -> workspace sandbox

The boundary matters: do not copy Microsoft ms-vscode-remote.remote-ssh closed-source code, and do not rebuild the remote stack Code-OSS already has.

3. The Remote-SSH Skeleton

The final path is:

Local Code-OSS fork
  -> our.remote-ssh resolver
  -> ssh-remote+dev
  -> SSH tunnel
  -> remote-ai-server
  -> Remote Agent
  -> Remote Extension Host
  -> remote FileService / Terminal / Search / Git / LSP

The remote workspace URI is not a local path:

vscode-remote://ssh-remote+dev/home/user/project

The local Workbench renders UI. File access, terminal, search, Git, and language servers sit behind the remote Extension Host and Remote Agent.

The implementation prompt was deliberately narrow:

Implement the minimal Code-OSS Remote-SSH fork loop.
 
Requirements:
- Do not copy the closed-source Remote-SSH extension
- Reuse Code-OSS Remote Authority Resolver
- Install remote-ai-server over SSH
- Open local port forwarding
- Return ResolvedAuthority
- Provide diagnostics
- Add tests or verification scripts for each step

This kept Codex on the product layer instead of rebuilding the IDE.

4. Remote Server Distribution

The remote installation looks like:

~/.remote-ai-server/bin/<commit>/
  bin/remote-ai-server
  node
  out/server-main.js
  product.json
  extensions/

The local resolver does four things:

1. Check the remote environment over SSH.
2. Install or reuse the server tarball for <commit>.
3. Start bin/remote-ai-server.
4. Open a local tunnel and return the authority to Code-OSS.

There is a real compatibility trap here: many enterprise Linux machines still run CentOS 7 with glibc 2.17. Some requirement scripts conservatively ask for glibc >= 2.28, while the actual binary ABI can be lower.

So I asked Codex for a release gate instead of blindly copying a requirement check:

Write remote server packaging and releaseDoctor.
 
Requirements:
- manifest records sha256, size, min glibc
- tarball must contain node, bin/remote-ai-server, out/server-main.js, product.json
- can overlay compatible Linux Node and native .node modules
- no macOS AppleDouble metadata in tarballs
- support validation on ssh dev with glibc 2.17

That turned “it launches” into a repeatable release condition.

5. The Hardest Codex Question: Where Does It Run?

In a full Remote-SSH workspace, Codex has three kinds of behavior:

UI:
  sidebar / webview / custom editor / commands
 
Workspace:
  workspaceFolders / file references / diagnostics / symbols
 
Agent execution:
  codex CLI / shell / git / rg / file patch

These cannot be placed blindly in one process.

The UI should stay local because the user needs the sidebar, webviews, and commands locally. Agent execution must be remote. Otherwise Codex edits local files, runs a local shell, and searches with local rg.

There is also a platform issue: the locally installed Codex extension may ship with a macOS CLI binary. That cannot run on a Linux remote host.

That is why the system needs ai-codex-remote-bridge.

6. What ai-codex-remote-bridge Does

ai-codex-remote-bridge is a workspace extension running in the remote Extension Host.

It is responsible for:

1. Detecting whether the current workspace is ssh-remote.
2. Detecting the openai.chatgpt extension.
3. Preparing a Linux Codex CLI on the remote host.
4. Pointing chatgpt.cliExecutable at a managed wrapper.
5. Syncing the necessary ~/.codex/auth.json and config.toml.
6. Setting the remote cwd, environment, and workspace root.
7. Forwarding approval requests back to local UI.

Platform selection must be a hard rule:

Remote Codex CLI must be Linux.
 
linux-x64   -> codex-linux-x64.tar.gz
linux-arm64 -> codex-linux-arm64.tar.gz
darwin-*    -> reject

Remote execution also needs explicit environment:

REMOTE_AI=1
REMOTE_AI_AUTHORITY=ssh-remote+dev
REMOTE_AI_WORKSPACE_ROOT=/home/user/project
REMOTE_AI_APPROVAL_CHANNEL=remoteai.approval

The first rule of remote AI is: never let a task that appears remote silently fall back to local execution.

7. Approval and Sandbox

Once AI can execute commands remotely, the safety model has to be part of the core design.

The default policy is:

read/list/search/git status: allow
write/applyPatch/delete: diff approval
exec: command approval
network command: elevated approval
sudo/root/system path: deny by default
outside workspace root: deny unless explicitly allowed

Local UI can show approval prompts, but path checks cannot live only in local UI. The remote bridge also needs a workspace sandbox:

workspace root realpath = /home/user/project
candidate path
  -> clean
  -> resolve symlink
  -> verify startsWith(root)

I had Codex start with tests like:

workspace sandbox:
- /home/user/project/src/a.ts allowed
- /home/user/project2/a.ts blocked
- /home/user/project/../.ssh/id_rsa blocked after normalization

Small tests like these prevent a dangerous illusion: a path string looking inside the workspace does not mean it is safe.

8. Native Chat UI

Code-OSS includes native Chat, Inline Chat, and Quick Chat entries. This fork is focused on RemoteAI + Codex, so I did not want another native chat surface.

The approach was “disable entry points, do not delete the world”:

• Hide Workbench Chat, Inline Chat, and Quick Chat.
• Remove native Chat picks from the command palette.
• Keep required chat-shaped types and DI services.
• Keep the openai.chatgpt proposed API entries in product.json.

That reduces product noise without breaking compatibility layers Codex or third-party extensions may still need.

9. Verification Evidence

I did not want the work to stop at “the architecture sounds right.” The project ended up with several verification layers:

• releaseDoctor checks tarballs and manifests.
• Gulp compiles our-remote-ssh, ai-codex-remote-bridge, and ai-approval-ui.
• Mocha covers packaging, the SSH resolver, the bridge, and workspace sandboxing.
• ssh dev validates remote install, startup, tunnels, and glibc 2.17.
• GUI validation covers the RemoteAI dashboard, remote folder picking, Explorer, terminal, and file saves.
• Codex binding validation ensures remote workspaces do not silently fall back to local codex.

One remote validation record looked like this:

server dir: ~/.remote-ai-server/bin/<commit>
bundled Node: v20.18.2
remote glibc: 2.17
bridge extension: installed
audit chain: bootstrap -> install -> launch -> tunnel

That evidence is more useful than “I think the architecture is right.”

10. How I Used Codex

In this project, Codex was most valuable in three roles:

1. Evidence gathering: find the real registration points for remote authority, server-main, extensionKind, and chat contributions.
2. Planning: split Remote-SSH, server packaging, Codex bridge, approval, and CentOS gates into verifiable tasks.
3. Convergent implementation: change one boundary at a time with a test or verification script.

The prompt pattern I kept using was:

First search and list evidence. Do not edit code.
 
Based on the evidence, give 2-3 options and recommend one.
Then implement only the smallest working slice of the recommended option.
Add a test or verification script.
 
Do not refactor unrelated modules.
Do not delete compatibility layers.
Do not copy closed-source implementations.

That pattern makes Codex act as an investigator before it acts as an engineer.

Takeaway

RemoteAI is not about SSH. It is about giving the IDE and AI correct remote semantics.

The judgments I kept are:

• File sync is not Remote-SSH.
• A virtual URI alone is not a full workspace.
• Codex UI running locally does not mean Codex execution should run locally.
• Remote AI needs a platform-correct CLI, remote cwd, remote toolchain, and approval path.
• Forking Code-OSS matters because it reuses the IDE platform instead of rebuilding the extension runtime.

The direction is simple: every action should happen on the machine where it belongs. UI is local. Workspace is remote. AI execution is remote too. The resolver, bridge, server packaging, and sandbox all exist to protect that line.