haze/README.md

# Haze

Hazy with a chance of clouds.

Easy setup and management of Nextcloud test instances using docker

## What

`haze` provides an easy way to set up Nextcloud test instances with a choice of
php version, database server, optional s3 or ldap setup and more.

## Setup

### Requirements

- Docker

### Installation

- Grab a binary from the
  [Codeberg releases](https://codeberg.org/icewind/haze/releases) and place it
  in your `$PATH`

### Config

Create a file `~/.config/haze/haze.toml` with the following options:

```toml
sources_root = "/path/to/nextcloud/sources"
```

See the [configuration section](#configuration) for more options.

### Quick examples

- Start a Nextcloud instance with `postgresql`, and `s3` primary storage:

  ```bash
  haze start pgsql s3
  ```

- Start a Nextcloud instance with `sqlite`, `php 8.3` and an `smb` external
  storage:

  ```bash
  haze start 8.3 smb
  ```

- Run specific units test against an `oracle` database
  ```bash
  haze test oracle apps/dav/tests/unit/Connector/Sabre
  ```

## Managing instances

#### Start an instance

```bash
haze start [--name <name>] [--detach] [database] [php-version] [services] [vX.Y.Z]
```

Where `database` is one of `sqlite`, `mysql`, `mariadb`, `pgsql` or `oracle`
with an optional version (e.g. `pgsql:12`), defaults to `sqlite`. And
`php-version` is one of `8.0`, `8.1`, `8.2`, `8.3`, `8.4` or `8.5`, defaults to
the maximum version support by the current Nextcloud version.

You can specify a version number (e.g. `v32.0.2`) to use the sources from a
release instead of using the local sources.

Use `--name <name>` to give the instance a specific name instead of a randomly
generated one. For example:

Use `--detach` to give the instance
[its own sources](#instances-with-their-own-sources) instead of sharing
`sources_root` with all other instances.

Additionally, you can use the following options when starting an instance:

- `s3`: set up an S3 server and configure to Nextcloud to use it as primary
  storage.
  - `s3s`: enable TLS for the S3 setup.
  - `s3mb`: enable multi-bucket S3 setup.
  - `s3m`: enable multi-instance S3 setup.
- `ldap`: set up an LDAP server.
- `office`: set up a Nextcloud Office server.
- `onlyoffice` setup an onlyoffice document server.
- `push` set up [client push](https://github.com/nextcloud/notify_push).
- `smb`: set up a samba server for external storage use.
- `dav`: set up a WebDAV server for external storage use.
- `sftp`: set up a SFTP server for external storage use.
- `sftp-key`: set up a SFTP server for external storage use with public key
  authentication.
- `kaspersky`: set up a kaspersky scan engine server in http mode. ( Requires
  [manually setting up the image](https://github.com/icewind1991/kaspersky-docker))
- `kaspersky-icap`: setup a kaspersky scan engine server in ICAP mode.
- `clamav`: set up a local clam av scanner in executable mode.
- `clamav-socket`: set up a clam av scanner in socket mode.
- `clamav-icap`: set up a clam av scanner in ICAP mode.
- `clamav-icap-tls`: set up a clam av scanner in ICAP mode with TLS encryption.
- `oc`: start an ownCloud instance in the same network.
- `imaginary`: start an Imaginary service and configure it for preview
  generation.
- `mail`: start a [smtp4dev](https://github.com/rnwood/smtp4dev) server and
  configure it the mail server.
- `webhook` start a
  [webhook tester](https://github.com/tarampampam/webhook-tester)
- `redis`: start a separate container for redis.
- `redis-tls`: connect to redis over TLS.
- `<path to app.tar.gz>`: by specifying the path to an app package this package
  will be extracted into the apps. directory of the new instance (overwriting
  any existing app code). This can be used to quickly test a packaged app.
- The name of any configured preset.

#### Run tests in a new instance

```bash
haze test [database] [php-version] [phpunit version] [path]
```

Where `path` is a file or folder to run PHPUnit in, relative to the sources
root.

### List running instances

```bash
haze
```

or

```bash
haze list
```

#### Remove all running instances

```bash
haze clean
```

### Instances with their own sources

By default every instance shares the sources configured as `sources_root`, so
all instances always run the same code. An instance started with `--detach` gets
its own `git worktree` of `sources_root` instead, created in `worktree_dir`,
which lets you run several instances on different branches at the same time.

```bash
haze start --name my-fix --detach
```

The worktree is checked out with a detached head. Every app in one of the
`app_directories` that gets enabled during setup receives its own worktree as
well, other apps keep running from the shared app directory.

The worktrees are removed together with the instance, by `haze stop` or
`haze clean`.

## Controlling running instances

The following commands run against the most recently started instance and allow
optionally providing a `match` to select a specific instance by its name.

#### Open an instance

```bash
haze [match] open
```

#### Open the database of an instance

```bash
haze [match] db
```

#### Execute a command on an instance

```bash
haze [match] [service] [cmd]
```

If no `cmd` is specified it will launch `bash`

If a service name or `db` is provided, the command will be in the container of
the service or database.

#### Create a new instance and run a command

```bash
haze [match] shell [cmd]
```

If no `cmd` is specified it will launch `bash`

#### Execute an occ command on an instance

```bash
haze [match] occ [cmd]
```

#### Connect to the database on an instance

```bash
haze [match] db
```

#### Show the logs of an instance

```bash
haze [match] logs
```

#### Stop an instance

```bash
haze [match] stop
```

#### Pin an instance

```bash
haze [match] pin
```

Pinned instances will not be removed by `haze clean`.

#### Unpin an instance

```bash
haze [match] unpin
```

#### Run a command with instance environment variables set

```bash
haze [match] env <cmd> [args]
```

Runs the provided command with `NEXTCLOUD_URL`, `DATABASE_URL` and `REDIS_URL`
environment variables set for the matched instance.

This is intended to run a local
[push daemon](https://github.com/nextcloud/notify_push) against an instance.

#### Update the container images

```bash
haze update
```

#### Edit a file in an instance with the local $EDITOR

```bash
haze [match] edit <path>
```

#### Reload the php config of an instance

```bash
haze [match] reload
```

The php configuration can edit changed with `haze edit  /config/php.ini`

#### Checkout a branch for all local apps

```bash
haze git checkout [branch]
```

Checks out the branch in all git repositories within the apps folder.

Defaults to the branch matching the current checked out server versions (e.g.
`master` or `stable33`).

`master` and `main` can be used interchangeably.

#### Pull remote changes for all local apps

```bash
haze git pull
```

Performs a pull in all git repositories within the apps folder.

## Federation

Multiple instances can reach each other by using their instance name as domain
name to allow for testing federation between instances. Alternatively, you can
set up the haze proxy and the proxied domains to get https support between
instances.

## Proxy

By default, instances can be accessed by their IP. In order to get more
memorable URLs and allow supporting https, haze comes with a builtin reverse
proxy to allow using a wildcard domain.

### Requirements

- A domain name you can set wildcard DNS records for
- A reverse proxy like Nginx or Apache
- (optionally) a wildcard ssl certificate (can be acquiring using letsencrypt
  and dns verification)

### DNS Setup

- Set a DNS record for `*.haze.example.com` and `haze.example.com` pointing to
  your development machine.
- Set the `proxy` configuration with your domain and desired listen endpoint.
- Set up a service to run `haze proxy` in the background as your own user. A
  systemd user service is recommended (see [haze.service](./haze.service) for an
  example).
- Configure your reverse proxy of choice to proxy `*.haze.example.com` and
  `haze.example.com` to the proxy's listen endpoint
- (optional) acquire a wildcard ssl certificate for your domain and set your
  reverse proxy to use it. This will be highly dependent on your DNS provider,
  [this](https://community.letsencrypt.org/t/dns-providers-who-easily-integrate-with-lets-encrypt-dns-validation/86438)
  lists some DNS providers and supported ACME clients.

### Local Setup

- Setup `dnsmasq` to resolve `*.haze.test` to your development machine.
- Generate a wildcard ssl certificate for `*.haze.test` using `mkcert`:

```bash
# Generate local wildcard certificate
mkcert -cert-file <path-to-your-certificats>haze.test.crt -key-file <path-to-your-certificats>haze.test.key '*.haze.test'
```

- Set up a service to run `haze proxy` in the background as your own user. A
  systemd user service is recommended (see [haze.service](./haze.service) for an
  example).
- Setup a reverse proxy to proxy `*.haze.test` and `haze.test` to the
  `haze proxy`'s socket. Example for Nginx:

```nginx
upstream haze-handler {
    server unix:/run/haze/haze.sock;
}

server {
    listen 80;
    listen 443 ssl;
    http2 on;
    server_name *.haze.test;

    ssl_certificate     <path-to-your-certificats>/haze.test.crt;
    ssl_certificate_key <path-to-your-certificats>/haze.test.key;

    location / {
            proxy_pass http://haze-handler;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

### Usage

When the proxy is configured, generated URLs for the instances will use a
subdomain of the configured domain, e.g. the `rolling-bees` instance will be
available at `rolling-bees.haze.example.com`. Additionally, `haze.example.com`
will automatically point to the last created instance.

Additionally, the proxy allows access to the server containers trough either
`<instance id>-<service id>.haze.example.com` for a specific instance, or
`<service-id>.haze.example.com` for the last created instance. For example
`rolling-bees-mail.haze.example.com` will give access to the smtp4dev web
interface of the `rolling-bees` instance.

## Configuration

Configuration is loaded from `~/.config/haze/haze.toml` and has the following
options

```toml
sources_root = "/path/to/sources" # path of the nextcloud sources. required
app_directories = ["/path/to/sources/more_app"] # paths to additional app directories.
work_dir = "/path/to/temp/dir" # path to temporary directory. optional, defaults to "/tmp/haze"
worktree_dir = "/path/to/worktrees" # where to create the worktrees of detached instances. optional, defaults to "<work_dir>/worktrees"

[auto_setup] # optional
enabled = false # whether or not to automatically install nextcloud on `haze start`. enabled by default
username = "foo" # username for admin user during auto setup. optional, defaults to "admin"
password = "bar" # password for admin user during auto setup. optional, defaults to "admin"
enable_apps = ["files_external"] # apps to enable after setup, defaults to []
disable_apps = ["contacts"] # apps to disable after setup, defaults to []
post_setup = [# commands to execute after setup, defaults to []
    "occ group:add test",
]
config = { "foo" = "bar" } # configuration options to set before install

[[volume]] # optional
source = "/tmp/haze-shared"
target = "/shared"
create = true

[[volume]]
source = "/home/me/Downloads"
target = "/Downloads"
read_only = true

[proxy] # optional
address = "haze.example.com" # base domain
https = true # Is the proxy behind a https terminating proxy
listen = "/run/haze/haze.sock" # either a unix socket path
#listen = "127.0.0.1:8080"     # or a socket address

# presets allow for easy usage of commonly used setups
[[preset]]
name = "groupfolders" # name of the preset
apps = ["groupfolders"] # app to enable
commands = ["occ groupfolders:create gf", "occ groupfolders:group 1 admin read write share delete"] # commands to run post-setup
```