nixaws

proxies SSH TCP connections to ephemeral EC2 instances

nixaws creates one shared EC2 instance for each configured AWS region and instance type combination at startup, waits until each instance SSH port is reachable and its root EBS volume is fully initialized, then stops each warmed instance. When an SSH connection arrives, nixaws walks the warmed instance pool, prepares transient scratch storage on the first ready instance, starts it, and raw-proxies bytes between clients and the instance. Subsequent SSH connections reuse the running instance. After the last SSH session disconnects, nixaws stops the instance and removes transient scratch storage. When the application exits, it terminates prepared instances.

Because this is a raw TCP proxy, nixaws does not terminate SSH. The connecting SSH client authenticates directly to the EC2 instance. To avoid host-key churn when the same local endpoint points at different EC2 instances, nixaws signs each generated instance host key with a local nixaws SSH host CA and can add the scoped CA trust entry to ~/.ssh/known_hosts at startup.

requirements

• nix
• AWS credentials and region available through the standard AWS SDK environment/config chain

getting started

nix develop

run

nix run .#dev

All runtime configuration can be provided either as CLI flags or as NIXAWS_* environment variables. CLI flags take precedence over environment variables.

nix run .#dev -- \
  --instance-type c8id.large,c8a.medium \
  --security-group-ids sg-... \
  --subnet-id subnet-...

Normal nixaws runs require IAM user credentials. If --profile or NIXAWS_PROFILE is set, that profile must exist and authenticate as an IAM user. Otherwise nixaws first tries the generated nixaws-service profile, then falls back to the standard AWS SDK credential chain, including AWS_PROFILE, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. If no authentication exists, nixaws asks you to login with aws login or provide IAM user credentials. If only root/bootstrap authentication exists, nixaws asks you to create the IAM user with init-iam-user. Set --region or NIXAWS_REGION to one or more comma-separated AWS region candidates; AWS_REGION supplies the SDK default only when --region/NIXAWS_REGION is not set.

For long-running services, create a dedicated IAM user access key explicitly. This setup command always uses root/bootstrap AWS authentication, then writes a separate service profile for normal nixaws runs. If root/bootstrap authentication is unavailable, run aws login first.

If your bootstrap profile was created with aws login, authenticate it first:

aws login --profile bootstrap-admin

Then create the service profile:

nix run .#dev -- \
  --profile bootstrap-admin \
  --region us-west-2 \
  init-iam-user \
  --profile-name nixaws-service \
  --write-profile

If you omit --profile, the setup command searches the active AWS SDK credential chain and local AWS profiles for root/bootstrap authentication; it will not use the generated service IAM user to regenerate itself. The bootstrap identity must be allowed to create IAM users, attach inline user policies, and create/delete access keys. The generated user is granted the EC2 instance, scratch EBS volume, and temporary security group permissions nixaws needs at runtime.

Use --force when regenerating an existing service profile. It replaces the local credentials profile and rotates the previous access key recorded in that profile so repeated regeneration does not hit IAM's two-access-key limit.

If launched instances use an IAM instance profile, include the role ARN so the generated runtime user can pass that role:

nix run .#dev -- \
  --profile bootstrap-admin \
  --region us-west-2 \
  init-iam-user \
  --profile-name nixaws-service \
  --write-profile \
  --pass-role-arns arn:aws:iam::123456789012:role/nixaws-instance

Then run nixaws. The generated nixaws-service profile is selected by default:

export AWS_REGION=us-west-2
nix run .#dev -- --subnet-id subnet-...

For systemd, store the generated nixaws-service profile under the service account's home, set NIXAWS_PROFILE for a custom IAM user profile, or store AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in an EnvironmentFile readable only by the service account.

NixOS module

This flake exposes nixosModules.default and nixosModules.nixaws for running nixaws as a system service. The module runs without the TUI under systemd, stores its home and SSH host CA under /var/lib/nixaws, and sends SIGINT on stop so nixaws can run its EC2 cleanup path.

{
  inputs.nixaws.url = "github:spotdemo4/nixaws";

  outputs =
    { nixpkgs, nixaws, ... }:
    {
      nixosConfigurations.host = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          nixaws.nixosModules.default
          {
            services.nixaws = {
              enable = true;
              regions = [ "us-west-2" ];
              listenAddr = "0.0.0.0:2222";
              instanceTypes = [
                "c8id.large"
                "c8a.medium"
              ];
              subnetId = "subnet-...";
              publicKeyPath = "/etc/nixaws/id_ed25519.pub";
              environmentFiles = [ "/run/secrets/nixaws.env" ];
              openFirewall = true;
            };
          }
        ];
      };
    };
}

Use services.nixaws.environment for any NIXAWS_* or AWS_* variables that do not have first-class module options. Use services.nixaws.environmentFiles for secrets such as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY; plain Nix paths are copied into the world-readable Nix store.

AMI selection is resolved once at startup and defaults to the latest matching NixOS AMI owned by 427812963091:

nix run .#dev -- \
  --ami-owner 427812963091 \
  --ami-name-pattern 'nixos/*' \
  --ami-architecture x86_64 # defaults from the local client architecture

Or with environment variables:

export NIXAWS_AMI_OWNER=427812963091
export NIXAWS_AMI_NAME_PATTERN='nixos/*'
export NIXAWS_AMI_ARCHITECTURE=x86_64

Set --ami-id or NIXAWS_AMI_ID to skip lookup and use a specific AMI.

Set --ami-architecture arm64 or NIXAWS_AMI_ARCHITECTURE=arm64 when using ARM/Graviton instance types.

Determinate Systems' NixOS AMIs are also supported:

export NIXAWS_AMI_OWNER=535002876703
export NIXAWS_AMI_NAME_PATTERN='determinate/nixos/epoch-1/*'

Common options:

nix run .#dev -- \
  --region us-west-2,us-east-1 \
  --listen-addr 0.0.0.0:2222 \
  --instance-type c8id.large,m6id.large,c8a.medium \
  --scratch-storage auto \
  --region-warmup-retry-interval-secs 600 \
  --scratch-volume-size-gb 100 \
  --subnet-id subnet-...

export NIXAWS_REGION=us-west-2,us-east-1
export NIXAWS_LISTEN_ADDR=0.0.0.0:2222
export NIXAWS_INSTANCE_TYPE=c8id.large,m6id.large,c8a.medium
export NIXAWS_SCRATCH_STORAGE=auto
export NIXAWS_REGION_WARMUP_RETRY_INTERVAL_SECS=600
export NIXAWS_SCRATCH_VOLUME_SIZE_GB=100
export NIXAWS_SUBNET_ID=subnet-...

--instance-type and NIXAWS_INSTANCE_TYPE accept one or more comma-separated EC2 instance types in priority order. They are not launch fallbacks inside a single runtime. Instead, each instance type is combined with each configured region to form the instance pool.

--region and NIXAWS_REGION accept one or more comma-separated AWS regions in priority order. The instance pool is the Cartesian product of configured regions and instance types, ordered by region first and instance type second. nixaws initializes each region first by loading a region-specific EC2 client, resolving the AMI, and preparing the key pair and security group. It then warms pool entries concurrently, one shared instance per region/type entry. If startup for a pool entry fails, its warm instance resources are cleaned up and warmup is retried every --region-warmup-retry-interval-secs until it succeeds. The default retry interval is 600 seconds. The service starts once at least one pool entry warms successfully; later successful retries are added in configured pool order without restarting. New SSH connections try warmed pool entries in configured order, bridging to the first entry that can be acquired and started. --subnet-id and --security-group-ids are region-specific, so they must exist in every region you expect to use.

When --subnet-id is omitted, nixaws discovers all available subnets in the prepared security group's VPC and retries capacity failures across those subnets. With a temporary security group, this means the default VPC's available subnets. With existing --security-group-ids, all groups must belong to one VPC, and that VPC's available subnets are used.

The default instance type list is c8id.large,c8a.medium: c8id.large gives auto scratch storage local NVMe instance storage, while c8a.medium adds a cheaper EBS-scratch pool entry.

--scratch-storage and NIXAWS_SCRATCH_STORAGE accept auto, instance-store, memory, or ebs; the default is auto. ram is accepted as an alias for memory. In auto mode, nixaws uses local EC2 instance storage when the selected instance type has it, otherwise it falls back to a transient gp3 scratch EBS volume. Set instance-store to require local instance storage; pool entries whose instance type lacks instance storage fail warmup. Set memory to use a RAM-backed tmpfs on the EC2 instance. Set ebs to always use the EBS scratch volume path.

The launched instance uses the AMI default root EBS volume unless --root-volume-size-gb is set explicitly. Large build outputs go to transient scratch storage instead. The instance bootstrap configures Nix clients and nix-daemon to use an experimental local-overlay store whose upper layer, state database, logs, /build, and temp directory live on that scratch storage. SSH startup is gated on this scratch setup so builds do not fall back to root-backed /build. Root EBS volume initialization must complete before the pool entry is marked ready. Stopping the instance removes instance-store and memory scratch contents; deleting the scratch EBS volume removes EBS-backed transient store paths and their Nix database together.

--scratch-volume-size-gb defaults to 100 GiB and only applies when the EBS scratch backend might be used. memory uses the kernel's default tmpfs sizing. --scratch-device-name defaults to /dev/sdf; change it only if that EBS attachment name conflicts with your AMI or instance type.

nixaws uses EC2 user data to install the transient overlay-store bootstrap on the instance. If --user-data-base64 is also set, it must decode to a shell script so nixaws can append it after its own bootstrap.

Set --nix-extra-substituter and --nix-extra-trusted-public-key to append binary caches to the instance Nix configuration. Both flags can be repeated or comma-separated; the equivalent environment variables are NIXAWS_NIX_EXTRA_SUBSTITUTERS and NIXAWS_NIX_EXTRA_TRUSTED_PUBLIC_KEYS. nixaws writes these as extra-substituters and extra-trusted-public-keys, so they add to the AMI's existing cache settings instead of replacing them.

Set --niks3-server-url and --niks3-auth-token-file to enable niks3 automatic upload from EC2 builds. nixaws installs the prebuilt niks3-hook release binary, starts a niks3-auto-upload.socket on the instance, and configures Nix post-build-hook to enqueue every build output for background upload.

nix run .#dev -- \
  --niks3-server-url https://niks3.example.com \
  --niks3-auth-token-file ~/.config/niks3/token

export NIXAWS_NIKS3_SERVER_URL=https://niks3.example.com
export NIXAWS_NIKS3_AUTH_TOKEN_FILE=~/.config/niks3/token

The auth token file is read locally at startup and copied into EC2 user data so the instance can write /var/lib/nixaws/niks3-auth-token with root-only permissions. Treat the token as exposed to principals that can read EC2 user data for launched instances; use an upload-scoped token. --niks3-release-version defaults to v1.6.1 and selects the GitHub release asset for the instance architecture. Set --niks3-hook-package only if you want the instance to run nix build --out-link /var/lib/nixaws/niks3-hook for a cache-populated or pinned package instead of downloading a release binary. Additional tuning options are --niks3-socket-path, --niks3-batch-size, --niks3-idle-exit-timeout-secs, --niks3-max-concurrent-uploads, --niks3-verify-s3-integrity, and --niks3-debug.

When --security-group-ids is not set, nixaws creates a temporary security group in the target VPC, allows inbound TCP traffic on --ssh-port from 0.0.0.0/0, attaches it to launched instances, and deletes it on clean shutdown. Set --security-group-ids or NIXAWS_SECURITY_GROUP_IDS to use existing security groups instead; they must belong to one VPC and allow inbound TCP traffic on --ssh-port from the machine running nixaws.

nixaws connects over raw SSH TCP and does not use SSM Session Manager. SSM agent status messages in the AWS console do not affect whether nixaws can reach port 22.

The TUI CPU and network columns use native CloudWatch AWS/EC2 metrics. CPU shows CPUUtilization; network shows combined NetworkIn and NetworkOut throughput.

After the last SSH session disconnects, the shared instance stays running for --idle-timeout-secs before it is stopped. Quick reconnects during that window reuse the running instance and prepared scratch storage. The default is 10 seconds. Set --idle-timeout-secs 0 or NIXAWS_IDLE_TIMEOUT_SECS=0 to stop immediately after the last session.

The shared instance is stopped while idle instead of terminated, so reconnects avoid EC2 instance creation while idle compute charges stop. EBS scratch volumes are deleted on every idle stop, and memory or instance-store scratch contents are discarded, so only the AMI-sized root EBS volume remains while idle.

By default, the shared instance is launched as a persistent EC2 Spot Instance with stop interruption behavior so it can participate in the stop/start lifecycle. Spot interruptions can still drop proxied SSH connections.

Starting a stopped Spot Instance can take longer than On-Demand while AWS reactivates the persistent Spot request. nixaws retries transient Spot start errors until --startup-timeout-secs and logs the EC2 start error plus the current Spot request state/status; if MaxSpotInstanceCountExceeded persists, request a higher Spot Instance quota or use --spot false.

Set --spot false or NIXAWS_SPOT=false to launch on-demand instances instead.

Set --spot-max-price or NIXAWS_SPOT_MAX_PRICE only if you need a hard maximum hourly Spot price; AWS recommends leaving it unset for fewer interruptions.

SSH authentication is prepared once at startup. By default, nixaws imports the first public key found at ~/.ssh/id_ed25519.pub or ~/.ssh/id_rsa.pub as a temporary EC2 key pair. The temporary key pair is reused for launched instances and deleted when the application exits cleanly.

SSH host verification uses a local nixaws host CA at ${XDG_DATA_HOME:-~/.local/share}/nixaws/ssh_host_ca. On interactive startup, if the CA is not already trusted for the local SSH endpoint, nixaws prompts before appending a scoped @cert-authority line to ~/.ssh/known_hosts. The CA private key stays local; only per-instance host certificates are injected into EC2 user data.

Set --public-key-path or NIXAWS_PUBLIC_KEY_PATH to use a different public key file.

Set --key-name or NIXAWS_KEY_NAME to use an existing EC2 key pair and skip temporary public key import.

Optional settings:

nix run .#dev -- \
  --connect-address public \
  --spot false \
  --spot-max-price 0.01 \
  --ssh-port 22 \
  --root-volume-size-gb 100 \
  --scratch-device-name /dev/sdf \
  --startup-timeout-secs 600 \
  --region-warmup-retry-interval-secs 600 \
  --terminate-timeout-secs 300 \
  --idle-timeout-secs 10 \
  --instance-name nixaws-ssh-bridge \
  --tags Environment=dev,Owner=me

export NIXAWS_CONNECT_ADDRESS=public # or private
export NIXAWS_SPOT=false
export NIXAWS_SPOT_MAX_PRICE=0.01
export NIXAWS_SSH_PORT=22
export NIXAWS_ROOT_VOLUME_SIZE_GB=100
export NIXAWS_SCRATCH_DEVICE_NAME=/dev/sdf
export NIXAWS_STARTUP_TIMEOUT_SECS=600
export NIXAWS_REGION_WARMUP_RETRY_INTERVAL_SECS=600
export NIXAWS_TERMINATE_TIMEOUT_SECS=300
export NIXAWS_IDLE_TIMEOUT_SECS=10
export NIXAWS_INSTANCE_NAME=nixaws-ssh-bridge
export NIXAWS_TAGS=Environment=dev,Owner=me

format

nix fmt

check

nix flake check

build

nix build

release

bumper

releases are created automatically for significant changes

use

cargo

cargo install nixaws \
  --index sparse+https://trev.zip/api/packages/llc/cargo/

docker

docker run trev.zip/llc/nixaws:latest

nix

nix run git+https://trev.zip/llc/nixaws.git

download

https://trev.zip/llc/nixaws/releases