# AOO Protocol Documentation for Interoperability (Complete)

## Overview

AOO (Audio over OSC) is a protocol for streaming audio over networks using OSC (Open Sound Control) messages. It operates over UDP for peer-to-peer audio data and control, and TCP (with SLIP framing) for server communication.

### Architecture

AOO uses a **source/sink model**:
- **Sources** send audio to sinks
- **Sinks** receive audio from sources and must **invite** sources to send to them
- Each peer typically has both a source (to send) and a sink (to receive)
- A source can send to multiple sinks; a sink can receive from multiple sources

---

## Protocol Constants

### Version
```
AOO_VERSION_MAJOR = 2
AOO_VERSION_MINOR = 0
AOO_VERSION_BUGFIX = 0
```

Version is encoded as a 32-bit integer:
```
version = (AOO_VERSION_MAJOR << 24) | (AOO_VERSION_MINOR << 16) | (AOO_VERSION_BUGFIX << 8) | (protocol_flags & 0xFF)
```

**Note:** The version check (`check_version()`) only validates the major version. Any minor/bugfix version is accepted.

### Protocol Flags
```
AOO_PROTOCOL_FLAG_COMPACT_DATA = 0x01  // Supports compact data message format (/d)
```

### Special IDs
```
AOO_ID_WILDCARD = -1
AOO_ID_NONE     = INT32_MIN
```

**Dummy Source ID 0**: Used for initial peer discovery handshake (see Connection Flow).

### ID Offsets (SonoBus-specific)
```
LATENCY_ID_OFFSET = 20000
ECHO_ID_OFFSET    = 40000
```

These offsets create separate source/sink pairs for latency measurement:
- Main audio: IDs 1, 2, 3, etc.
- Latency sources/sinks: main_id + 20000
- Echo sources/sinks: main_id + 40000

### Default Timing Values

**AOO Library Defaults (aoo.h):**
```
AOO_PACKETSIZE         = 512   // AOO library default
AOO_PING_INTERVAL      = 1000  // AOO library default (ms)
AOO_RESEND_BUFSIZE     = 1000  // Resend buffer size (ms)
AOO_RESEND_LIMIT       = 5     // Max resend attempts per packet
AOO_RESEND_INTERVAL    = 10    // Interval between resend attempts (ms)
AOO_RESEND_MAXNUMFRAMES = 16   // Max frames to request per resend call
AOO_SOURCE_BUFSIZE     = 10    // Source ring buffer size
AOO_SEND_REDUNDANCY    = 1     // Send redundancy count
AOO_TIMEFILTER_BANDWIDTH = 0.008 // Time DLL filter bandwidth
```

**SonoBus Runtime Overrides:**
```
packetsize    = 600   // Set via set_packetsize() per source
ping_interval = 2000  // Set via set_ping_interval() per source (ms)
```

---

## OSC Fundamentals

### Type Tags
```
'i' = int32    (32-bit big-endian signed integer)
'h' = int64    (64-bit big-endian signed integer)
'f' = float    (32-bit big-endian IEEE 754)
'd' = double   (64-bit big-endian IEEE 754)
't' = timetag  (64-bit NTP timestamp)
's' = string   (null-terminated, padded to 4-byte boundary)
'b' = blob     (32-bit size + data, padded to 4-byte boundary)
'T' = true     (no data bytes)
'F' = false    (no data bytes)
'N' = nil      (no data bytes)
```

### Address Pattern Prefixes
```
AOO_MSG_DOMAIN    = "/aoo"
AOONET_MSG_SERVER = "/server"
AOONET_MSG_CLIENT = "/client"
AOONET_MSG_PEER   = "/peer"
AOO_MSG_SOURCE    = "/src"
AOO_MSG_SINK      = "/sink"
```

---

## Data Structures

### Audio Format
```c
typedef struct aoo_format {
    const char *codec;     // Codec name (e.g., "opus", "pcm")
    int32_t nchannels;     // Number of audio channels (1-255)
    int32_t samplerate;    // Sample rate in Hz
    int32_t blocksize;     // Block size in samples
} aoo_format;
```

### Opus Codec Format
```c
typedef struct aoo_format_opus {
    aoo_format header;
    int32_t bitrate;           // Total bitrate for all channels (0 = OPUS_AUTO)
    int32_t complexity;        // 0-10 (SonoBus default: 10)
    int32_t signal_type;       // OPUS_SIGNAL_MUSIC (3002), OPUS_SIGNAL_VOICE (3001), or OPUS_AUTO (-1000)
    int32_t application_type;  // OPUS_APPLICATION_RESTRICTED_LOWDELAY (2051) - SonoBus default
} aoo_format_opus;
```

**Valid Opus Sample Rates:** 8000, 12000, 16000, 24000, 48000 Hz

**Valid Block Sizes at 48000 Hz:** 120, 240, 480, 960, 1920, 2880 samples

**Note:** The `validate_format()` function clamps block sizes to valid ranges:
- Minimum: `samplerate / 400` (120 samples at 48kHz)
- Maximum: `minblocksize * 24` (2880 samples at 48kHz)
- Non-standard values are rounded down to the nearest valid size (powers of 2 × minblocksize)

**Opus Frame Duration Calculation:**
```
frame_duration_ms = blocksize / (samplerate / 1000)
```

At 48kHz:
| Block Size | Duration |
|------------|----------|
| 120 | 2.5 ms |
| 240 | 5 ms |
| 480 | 10 ms |
| 960 | 20 ms |
| 1920 | 40 ms |
| 2880 | 60 ms |

**SonoBus Opus Presets (per channel bitrate):**
| Index | Bitrate | Complexity | Signal Type | Min Blocksize |
|-------|---------|------------|-------------|---------------|
| 0 | 16 kbps | 10 | MUSIC | 960 |
| 1 | 24 kbps | 10 | MUSIC | 480 |
| 2 | 48 kbps | 10 | MUSIC | 240 |
| 3 | 64 kbps | 10 | MUSIC | 240 |
| 4 | 96 kbps | 10 | MUSIC | 120 |
| 5 | 128 kbps | 10 | MUSIC | 120 |
| 6 | 160 kbps | 10 | MUSIC | 120 |
| 7 | 256 kbps | 10 | MUSIC | 120 |

### PCM Codec Format
```c
typedef struct aoo_format_pcm {
    aoo_format header;
    int32_t bitdepth;   // AOO_PCM_INT16 (2), AOO_PCM_INT24 (3), AOO_PCM_FLOAT32 (4), AOO_PCM_FLOAT64 (8)
} aoo_format_pcm;
```

**PCM Codec Options Blob:** Contains a single int32 for bitdepth indicator:
- 2 = 16-bit signed integer
- 3 = 24-bit signed integer  
- 4 = 32-bit float
- 8 = 64-bit float

---

## Opus Codec Details

### Opus Application Types
```c
#define OPUS_APPLICATION_VOIP                2048  // Optimized for voice
#define OPUS_APPLICATION_AUDIO               2049  // Optimized for music/audio
#define OPUS_APPLICATION_RESTRICTED_LOWDELAY 2051  // Lowest latency (SonoBus default)
```

### Opus Signal Types
```c
#define OPUS_AUTO         -1000  // Auto-detect signal type
#define OPUS_SIGNAL_VOICE  3001  // Bias toward voice
#define OPUS_SIGNAL_MUSIC  3002  // Bias toward music (SonoBus default)
```

### Opus Bandwidth Constants
```c
#define OPUS_BANDWIDTH_NARROWBAND     1101  // 4 kHz passband
#define OPUS_BANDWIDTH_MEDIUMBAND     1102  // 6 kHz passband
#define OPUS_BANDWIDTH_WIDEBAND       1103  // 8 kHz passband
#define OPUS_BANDWIDTH_SUPERWIDEBAND  1104  // 12 kHz passband
#define OPUS_BANDWIDTH_FULLBAND       1105  // 20 kHz passband
```

### Opus Bitrate Limits
- Minimum: 500 bps
- Maximum: 512000 bps (or OPUS_BITRATE_MAX = -1 for maximum possible)
- Typical range for music: 64-256 kbps per channel

### Opus Error Codes
```c
#define OPUS_OK                0   // No error
#define OPUS_BAD_ARG          -1   // Invalid argument
#define OPUS_BUFFER_TOO_SMALL -2   // Buffer too small
#define OPUS_INTERNAL_ERROR   -3   // Internal codec error
#define OPUS_INVALID_PACKET   -4   // Corrupted or invalid packet
#define OPUS_UNIMPLEMENTED    -5   // Unsupported request
#define OPUS_INVALID_STATE    -6   // Invalid encoder/decoder state
#define OPUS_ALLOC_FAIL       -7   // Memory allocation failed
```

### Multi-channel Opus Encoding

SonoBus uses **all decoupled streams** for any channel count (no coupled streams):

```c
// Channel mapping for multistream (SonoBus approach)
// streams = nchannels (one independent mono stream per channel)
// coupled_streams = 0 (no stereo coupling)
// Total coded channels = streams + coupled_streams = nchannels

int streams = nchannels;
int coupled_streams = 0;
unsigned char mapping[255];
for (int i = 0; i < nchannels; i++) {
    mapping[i] = i;  // Sequential mapping
}
memset(mapping + nchannels, 255, 256 - nchannels);  // Fill unused with 255

OpusMSEncoder *encoder = opus_multistream_encoder_create(
    samplerate,      // Must be 8000, 12000, 16000, 24000, or 48000
    nchannels,       // Up to 255 channels
    streams,         // Number of streams (equals nchannels)
    coupled_streams, // Number of coupled (stereo) streams (always 0)
    mapping,         // Channel mapping array
    OPUS_APPLICATION_RESTRICTED_LOWDELAY,
    &error
);

// Set bitrate (total for all channels)
opus_multistream_encoder_ctl(encoder, OPUS_SET_BITRATE(bitrate * nchannels));
opus_multistream_encoder_ctl(encoder, OPUS_SET_COMPLEXITY(10));
opus_multistream_encoder_ctl(encoder, OPUS_SET_SIGNAL(OPUS_SIGNAL_MUSIC));
```

**Critical:** The bitrate in the format message is **total for all channels**, not per-channel:
```cpp
// SonobusPluginProcessor.cpp:6331
fmt->bitrate = info.bitrate * fmt->header.nchannels;
```

So for stereo at 96kbps/channel, the total bitrate should be 192000.

### Opus Encoding Process
```c
// Input: interleaved float samples in range [-1.0, 1.0]
// frame_size: samples per channel (must match blocksize)
int encoded_bytes = opus_multistream_encode_float(
    encoder,
    input_pcm,        // Interleaved float samples
    frame_size,       // Samples per channel
    output_buffer,    // Compressed data output
    max_output_bytes  // Maximum output size (recommend 4000)
);
```

### Opus Decoding Process
```c
OpusMSDecoder *decoder = opus_multistream_decoder_create(
    samplerate,
    nchannels,
    streams,
    coupled_streams,
    mapping,
    &error
);

int decoded_samples = opus_multistream_decode_float(
    decoder,
    compressed_data,  // NULL for packet loss concealment
    data_length,      // 0 for packet loss concealment
    output_pcm,       // Interleaved float output
    max_frame_size,   // Maximum samples per channel
    0                 // decode_fec flag
);
```

---

## SLIP Framing (TCP Only)

All TCP messages use SLIP (Serial Line Internet Protocol) framing:

```
SLIP_END     = 192 (0xC0)  // Frame delimiter
SLIP_ESC     = 219 (0xDB)  // Escape character
SLIP_ESC_END = 220 (0xDC)  // Escaped END
SLIP_ESC_ESC = 221 (0xDD)  // Escaped ESC
```

**Frame structure:** `SLIP_END | escaped_data | SLIP_END`

**Escape rules:**
- Byte 192 in data ? 219, 220
- Byte 219 in data ? 219, 221

**SLIP Encoding Algorithm:**
```c
void slip_encode(const uint8_t *input, size_t in_len, uint8_t *output, size_t *out_len) {
    size_t j = 0;
    output[j++] = SLIP_END;
    for (size_t i = 0; i < in_len; i++) {
        switch (input[i]) {
            case SLIP_END:
                output[j++] = SLIP_ESC;
                output[j++] = SLIP_ESC_END;
                break;
            case SLIP_ESC:
                output[j++] = SLIP_ESC;
                output[j++] = SLIP_ESC_ESC;
                break;
            default:
                output[j++] = input[i];
        }
    }
    output[j++] = SLIP_END;
    *out_len = j;
}
```

**SLIP Decoding Algorithm:**
```c
bool slip_decode(const uint8_t *input, size_t in_len, uint8_t *output, size_t *out_len) {
    size_t j = 0;
    bool in_escape = false;
    for (size_t i = 0; i < in_len; i++) {
        if (in_escape) {
            switch (input[i]) {
                case SLIP_ESC_END: output[j++] = SLIP_END; break;
                case SLIP_ESC_ESC: output[j++] = SLIP_ESC; break;
                default: return false;  // Invalid escape sequence
            }
            in_escape = false;
        } else {
            switch (input[i]) {
                case SLIP_END: break;  // Frame delimiter, ignore
                case SLIP_ESC: in_escape = true; break;
                default: output[j++] = input[i];
            }
        }
    }
    *out_len = j;
    return true;
}
```

---

## Server/Client Protocol (TCP)

### UDP Port Discovery

Before TCP login, client discovers its public IP:port via UDP:

**Request (Client ? Server, UDP):**
```
Address: /aoo/server/request
Type tags: ,
Arguments: (none)
```

**Reply (Server ? Client, UDP):**
```
Address: /aoo/client/reply
Type tags: ,si
Arguments:
    1. string - Client's public IP address
    2. int32  - Client's public port
```

### Login

**Request (Client ? Server, TCP):**
```
Address: /aoo/server/login
Type tags: ,sssisih
Arguments:
    1. string - Username
    2. string - Password hash (MD5, uppercase hex)
    3. string - Public IP address
    4. int32  - Public port
    5. string - Local IP address
    6. int32  - Local port
    7. int64  - Client token (random identifier)
```

**Response (Server ? Client, TCP):**
```
Address: /aoo/client/login
Type tags: ,i (success) or ,is (failure)
Arguments:
    1. int32  - Status (> 0 = success)
    2. string - Error message (on failure only)
```

### Group Join

**Request (Client ? Server, TCP):**
```
Address: /aoo/server/group/join
Type tags: ,ssF or ,ssT
Arguments:
    1. string  - Group name
    2. string  - Password hash (MD5, uppercase hex)
    3. boolean - F = private, T = public (type tag only, no data bytes)
```

**Response (Server ? Client, TCP):**
```
Address: /aoo/client/group/join
Type tags: ,si
Arguments:
    1. string - Group name
    2. int32  - Status (> 0 = success)
```

### Group Leave

**Request (Client ? Server, TCP):**
```
Address: /aoo/server/group/leave
Type tags: ,s
Arguments:
    1. string - Group name
```

### Peer Notifications

**Peer Join (Server ? Client, TCP):**
```
Address: /aoo/client/peer/join
Type tags: ,sssisih
Arguments:
    1. string - Group name
    2. string - Username
    3. string - Public IP
    4. int32  - Public port
    5. string - Local IP
    6. int32  - Local port
    7. int64  - Peer's token
```

**Note:** Peer join notifications may arrive BEFORE the group join response. Store them and process after group join succeeds.

**Peer Leave (Server ? Client, TCP):**
```
Address: /aoo/client/peer/leave
Type tags: ,ss
Arguments:
    1. string - Group name
    2. string - Username
```

### Server Ping (Keepalive)

```
Address: /aoo/server/ping
Type tags: ,
Arguments: (none)
```

Send periodically over both TCP and UDP to maintain connection.

---

## Peer-to-Peer Protocol (UDP)

### Peer Ping (NAT Traversal & Keepalive)

```
Address: /aoo/peer/ping
Type tags: ,h (with token) or , (without)
Arguments:
    1. int64 - YOUR client token (not the received one)
```

**Critical:** When receiving a peer ping, you MUST respond with your own peer ping containing YOUR token. This is required for NAT traversal.

**NAT Port Updates:** When matching incoming packets to peers, if the IP matches but port differs, UPDATE the stored port to the new value.

---

## Audio Streaming Protocol (UDP)

### Invite (Sink ? Source)

A sink must invite a source before it will receive audio.

```
Address: /aoo/src/<source_id>/invite
Type tags: ,i or ,ii
Arguments:
    1. int32 - Sink ID (the inviting sink's ID)
    2. int32 - Protocol flags (optional)
```

### Uninvite (Sink ? Source)

```
Address: /aoo/src/<source_id>/uninvite
Type tags: ,i
Arguments:
    1. int32 - Sink ID
```

### Format Request (Sink ? Source)

Request retransmission of format message:

```
Address: /aoo/src/<source_id>/format
Type tags: ,ii
Arguments:
    1. int32 - Sink ID
    2. int32 - Version (with protocol flags)
```

### Format (Source ? Sink)

**Must be sent immediately upon receiving an invite, and periodically (every 3-5 seconds) for reliability.**

```
Address: /aoo/sink/<sink_id>/format
Type tags: ,iiiiiisb [,b]
Arguments:
    1. int32  - Source ID
    2. int32  - Version (with protocol flags)
    3. int32  - Salt (session identifier, regenerate on reconnection)
    4. int32  - Number of channels
    5. int32  - Sample rate
    6. int32  - Block size
    7. string - Codec name ("opus" or "pcm")
    8. blob   - Codec-specific options
    9. blob   - (Optional) User format layout (JUCE ValueTree binary)
```

**Note:** The 9th argument (userformat) is optional. SonoBus checks `msg.ArgumentCount() > 8` before reading it.

**Opus Options Blob (16 bytes, big-endian int32s):**
```
Bytes 0-3:   Bitrate (total for all channels)
Bytes 4-7:   Complexity (0-10)
Bytes 8-11:  Signal type (OPUS_SIGNAL_MUSIC=3002, OPUS_SIGNAL_VOICE=3001, OPUS_AUTO=-1000)
Bytes 12-15: Application type (OPUS_APPLICATION_RESTRICTED_LOWDELAY=2051)
```

**PCM Options Blob (4 bytes, big-endian int32):**
```
Bytes 0-3:   Bitdepth indicator (AOO_PCM_INT16=2, AOO_PCM_INT24=3, AOO_PCM_FLOAT32=4, AOO_PCM_FLOAT64=8)
```

### Data (Source ? Sink)

```
Address: /aoo/sink/<sink_id>/data
Type tags: ,iiidiiiib
Arguments:
    1. int32  - Source ID
    2. int32  - Salt
    3. int32  - Sequence number (per-peer, starts at 0)
    4. double - Sample rate
    5. int32  - Channel onset (which sink channel to start at, usually 0)
    6. int32  - Total size of encoded data
    7. int32  - Number of frames (usually 1)
    8. int32  - Frame number (0-based, usually 0)
    9. blob   - Encoded audio data
```

### Compact Data (Source ? Sink)

Only use if `AOO_PROTOCOL_FLAG_COMPACT_DATA` was negotiated.

```
Address: /d
Type tags: ,iib or ,iidb
Arguments (3-arg form):
    1. int32 - Salt
    2. int32 - Sequence number
    3. blob  - Encoded audio data

Arguments (4-arg form, when sample rate changed):
    1. int32  - Salt
    2. int32  - Sequence number
    3. double - Sample rate
    4. blob   - Encoded audio data
```

**Note:** Use full data format for maximum compatibility.

### Data Request / Resend (Sink ? Source)

Request retransmission of lost packets:

```
Address: /aoo/src/<source_id>/data
Type tags: ,ii[ii]*
Arguments:
    1. int32 - Sink ID
    2. int32 - Salt
    3+ Pairs of (int32 sequence, int32 frame) - frame = -1 for whole block
```

### Ping (Source ? Sink)

For timing and keepalive (send every 1-2 seconds):

```
Address: /aoo/sink/<sink_id>/ping
Type tags: ,it
Arguments:
    1. int32   - Source ID
    2. timetag - NTP timestamp
```

### Ping Response (Sink ? Source)

```
Address: /aoo/src/<source_id>/ping
Type tags: ,itti
Arguments:
    1. int32   - Sink ID
    2. timetag - Original timestamp (tt1)
    3. timetag - Reply timestamp (tt2)
    4. int32   - Lost blocks count
```

---

## NTP Timestamp Format

AOO uses NTP timestamps (64-bit) for timing:

```c
// NTP timestamp structure
typedef struct {
    uint32_t seconds;      // Seconds since Jan 1, 1900
    uint32_t fraction;     // Fractional seconds (2^32 = 1 second)
} ntp_timestamp;

// Convert system time to NTP timestamp
uint64_t time_to_ntp(double seconds_since_epoch) {
    // NTP epoch is Jan 1, 1900; Unix epoch is Jan 1, 1970
    // Difference: 2208988800 seconds
    const uint64_t NTP_UNIX_OFFSET = 2208988800ULL;
    
    double ntp_seconds = seconds_since_epoch + NTP_UNIX_OFFSET;
    uint32_t sec = (uint32_t)ntp_seconds;
    uint32_t frac = (uint32_t)((ntp_seconds - sec) * 4294967296.0);
    
    return ((uint64_t)sec << 32) | frac;
}

// Convert NTP timestamp to seconds
double ntp_to_seconds(uint64_t ntp) {
    uint32_t sec = (uint32_t)(ntp >> 32);
    uint32_t frac = (uint32_t)(ntp & 0xFFFFFFFF);
    return (double)sec + (double)frac / 4294967296.0;
}
```

---

## Connection Flow

### 1. Server Connection
1. Client connects TCP to server
2. Client sends `/aoo/server/request` via UDP
3. Server responds with `/aoo/client/reply` containing public IP:port
4. Client sends `/aoo/server/login` via TCP
5. Server responds with `/aoo/client/login`

### 2. Group Join
1. Client sends `/aoo/server/group/join` via TCP
2. Server sends `/aoo/client/peer/join` for each existing peer (may arrive before join response)
3. Server responds with `/aoo/client/group/join`

### 3. Peer Audio Connection (Dummy Source Handshake)

When peers are notified of each other:

```
Peer A                                         Peer B
   |                                              |
   |<-- /aoo/client/peer/join (from server) -----|
   |                                              |
   |-- /aoo/src/0/invite (A's sink_id) --------->|  (invite dummy source)
   |                                              |
   |        B receives invite to source 0         |
   |        B creates real source/sink pair       |
   |        B sets: remote_sink_id = A's sink_id  |
   |        B sets: remote_source_id = A's sink_id (mirrored)
   |                                              |
   |<-- /aoo/sink/<A's sink>/format -------------|  (B sends format)
   |<-- /aoo/src/<A's source>/invite ------------|  (B invites A back)
   |                                              |
   |        A receives invite                     |
   |        A sets: remote_sink_id = B's sink_id  |
   |        A sets: remote_source_id = B's sink_id (mirrored)
   |                                              |
   |-- /aoo/sink/<B's sink>/format ------------->|  (A sends format)
   |                                              |
   |<========= audio data flows both ways ======>|
```

**Key Points:**
- The invite to source 0 triggers the handshake
- The sink ID from the invite becomes BOTH the remote_sink_id AND the source_id used in outgoing messages
- Each peer connection requires unique sink IDs
- Salt must be regenerated on new connections (reset to 0 forces regeneration)
- Sequence numbers are per-peer and reset with new salt

### 4. Ongoing Maintenance
- Send `/aoo/peer/ping` periodically for NAT keepalive
- Send `/aoo/sink/<id>/ping` from sources for timing
- Resend format messages periodically (every 3-5 seconds)
- Send `/aoo/server/ping` to maintain server connection

---

## SonoBus Custom Messages (UDP)

SonoBus extends AOO with custom messages under the `/sb` domain:

### Peer Info (`/sb/pinfo`)
```
Address: /sb/pinfo
Type tags: ,b
Arguments:
    1. blob - JSON containing peer info
```

**JSON Fields:**
```json
{
    "jitbuf": <float>,    // Jitter buffer time in ms
    "inlat": <float>,     // Input latency in ms
    "outlat": <float>,    // Output latency in ms
    "nettype": <int>,     // Network type (0=unknown, 1=ethernet, 2=wifi, 3=mobile)
    "rec": <bool>         // Is recording
}
```

### Channel Layout Info (`/sb/clayinfo`)
```
Address: /sb/clayinfo
Type tags: ,ib
Arguments:
    1. int32 - Source ID
    2. blob  - ValueTree binary format containing channel layout
```

### Chat Message (`/sb/chat`)
```
Address: /sb/chat
Type tags: ,sssss
Arguments:
    1. string - Group name
    2. string - From username
    3. string - Targets (pipe-separated usernames, empty = all)
    4. string - Tags
    5. string - Message text
```

### Ping (`/sb/ping`)
```
Address: /sb/ping
Type tags: ,t
Arguments:
    1. timetag - Timestamp
```

### Ping Acknowledgment (`/sb/pngack`)
```
Address: /sb/pngack
Type tags: ,tt
Arguments:
    1. timetag - Original timestamp (tt1)
    2. timetag - Reply timestamp (tt2)
```

### Request Latency Info (`/sb/reqlatinfo`)
```
Address: /sb/reqlatinfo
Type tags: ,
Arguments: (none)
```

### Latency Info (`/sb/latinfo`)
```
Address: /sb/latinfo
Type tags: ,b
Arguments:
    1. blob - JSON array of latency info objects
```

**JSON Format:**
```json
[
    {
        "srcname": "<username>",
        "destname": "<username>",
        "latms": <float>
    },
    ...
]
```

### Suggest Latency Match (`/sb/suggestlat`)
```
Address: /sb/suggestlat
Type tags: ,sf
Arguments:
    1. string - Username making suggestion
    2. float  - Suggested latency in ms
```

### Blocked Info (`/sb/blockedinfo`)
```
Address: /sb/blockedinfo
Type tags: ,sT or ,sF
Arguments:
    1. string  - Username
    2. boolean - Blocked status (T=blocked, F=unblocked)
```

### Suggest Group (`/sb/suggestgroup`)
```
Address: /sb/suggestgroup
Type tags: ,b
Arguments:
    1. blob - JSON with group suggestion
```

**JSON Format:**
```json
{
    "user": "<username>",
    "group": "<group_name>",
    "group_pass": "<password>",
    "public": <bool>,
    "others": ["<username1>", "<username2>", ...]
}
```

---

## Jitter Buffer Auto-Sizing

SonoBus implements automatic jitter buffer sizing with several modes:

### AutoNetBufferMode Values
```
AutoNetBufferModeOff = 0           // Manual only
AutoNetBufferModeAutoIncreaseOnly = 1  // Only increase, never decrease
AutoNetBufferModeAutoFull = 2      // Increase and decrease automatically
AutoNetBufferModeInitAuto = 3      // Start at 0, auto-increase until stable, then lock
```

### Auto-Sizing Algorithm

**Increase Trigger:**
- When drop rate exceeds threshold
  - InitAuto mode: 1.0 drops/sec
  - Other auto modes: configurable via `mAutoresizeDropRateThresh`
- Increase by one audio block's worth of time: `1000.0 * currSamplesPerBlock / sampleRate` ms
- Minimum time between increases: 0.5 seconds (`adjustlimit = 0.5f`)

**Decrease Trigger (AutoFull mode only):**
- When no drops for 10 seconds (`nodropsthresh = 10.0`)
- Decrease by one audio block's worth of time
- Won't decrease below `netBufAutoBaseline`
- Minimum time between decreases: 10 seconds (`adjustlimit = 10`)

**InitAuto Completion:**
- Mode transitions to "completed" when no drops for 7 seconds (`nodropsthresh = 7.0`)
- Once completed, stops auto-adjusting
- Safety mute is cleared when init completes

### Safety Muting

New connections start "safety muted" (`resetSafetyMuted = true`) until one of these conditions is met:
- Buffer time is > 15ms (`jitterbufthresh = 15.0f`), OR
- Time since reset > 0.5s AND drop rate is 0 AND time since last drop > 0.75s, OR
- Time since reset > 0.5s AND drop rate > 0 AND drop rate < 2.0/sec

**Constants:**
```cpp
safetyunmutethreshrate = 2.0f;    // drops/sec threshold
safetyunmutethreshmintime = 0.5f;  // minimum time since reset (seconds)
safetyunmutethreshtime = 0.75f;    // time since last drop (seconds)
jitterbufthresh = 15.0f;          // buffer time threshold (ms)
```

---

## Latency/Echo Testing System

SonoBus uses dedicated source/sink pairs for latency measurement:

### ID Allocation
```
Main audio ID: peer->ourId (e.g., 1)
Latency test ID: peer->ourId + LATENCY_ID_OFFSET (e.g., 20001)
Echo ID: peer->ourId + ECHO_ID_OFFSET (e.g., 40001)
```

### Latency Test Flow
1. Local latencysink invites remote echosource (ID + ECHO_ID_OFFSET)
2. Local latencysource adds remote echosink as destination
3. Test signal is sent through latencysource
4. Remote echosink receives and echosource re-sends immediately
5. Local latencysink receives echo
6. Round-trip time calculated from timestamps

### Echo Sources
Echo sources simply loop back received audio:
- echosink receives audio
- echosource immediately transmits it back
- No processing or buffering applied

---

## Codec Change Request

When a sink wants the source to change codec:

```
Address: /aoo/src/<source_id>/codec
Type tags: ,i...  (format-specific)
Arguments:
    1. int32 - Sink ID
    (remaining args define the requested format)
```

The source may honor this request by calling `set_format()` and re-sending format messages.

SonoBus sources have `set_respect_codec_change_requests(1)` enabled.

---

## Implementation Notes

### Salt Management
```c
// Reset on new connection
peer->salt = 0;
peer->sequence = 0;
peer->format_sent = false;

// Generate before sending format
if (peer->salt == 0) {
    peer->salt = generate_random_salt();
    peer->sequence = 0;
}
```

### Sequence Numbers
- Per-peer, not global
- Reset to 0 when salt changes
- Increment after each data message sent

### Audio Processing
```c
// Sending (in audio callback)
uint64_t t = aoo_osctime_get();  // NTP timestamp
source->process(input_buffers, num_samples, t);

// In send thread
source->send();

// Receiving (handle incoming UDP)
sink->handle_message(buf, nbytes, endpoint);

// In audio callback
sink->process(output_buffers, num_samples, t);

// In send thread (for control messages)
sink->send();
```

### Buffer Sizes
```c
// Send buffer
float sendbufsize = jmax(10.0, SENDBUFSIZE_SCALAR * 1000.0f * currSamplesPerBlock / getSampleRate());
// where SENDBUFSIZE_SCALAR = 2.0f

// Receive buffer (jitter buffer)
peer->oursink->set_buffersize(peer->buffertimeMs);
```

### Dynamic Resampling
When enabled, sinks perform resampling to match source sample rate variations:
```c
peer->oursink->set_dynamic_resampling(mDynamicResampling.get() ? 1 : 0);
```

---

## Key Areas for Stereo/Multi-channel Support

### 1. Channel Count in Format Setup

In `setupSourceFormat()`, the channel count is determined by:

```cpp
int channels = latencymode ? 1 : peer ? peer->sendChannels : getMainBusNumInputChannels();
```

The `peer->sendChannels` is calculated in `updateRemotePeerSendChannels()`:

```cpp
newchancnt = isAnythingRoutedToPeer(index) ? getMainBusNumOutputChannels() : 
             remote->nominalSendChannels <= 0 ? totinchans : remote->nominalSendChannels;
```

Where `totinchans` sums up all input channel groups.

### 2. Sink Setup with Correct Channel Count

When receiving format, the sink is reconfigured:

```cpp
if (peer->recvChannels != f.header.nchannels) {
    const ScopedWriteLock sl (peer->sinkLock);
    peer->recvChannels = std::min(MAX_PANNERS, f.header.nchannels);
    int sinkchan = std::max(getMainBusNumOutputChannels(), peer->recvChannels);
    peer->oursink->setup(getSampleRate(), currSamplesPerBlock, sinkchan);
}
```

### 3. Critical: Source Setup Must Match

In `setupSourceFormatsForAll()`:

```cpp
s->oursource->setup(sampleRate, currSamplesPerBlock, s->sendChannels);
```

**The `sendChannels` must match what's encoded in the format.**

### 4. Opus Multi-channel Encoding

For Opus with multiple channels, the bitrate is **total for all channels**:

```cpp
fmt->bitrate = info.bitrate * fmt->header.nchannels;  // Multiply by channel count!
```

So for stereo at 96kbps/channel, the total bitrate should be 192000.

---

## OSC Message Construction Reference

### OSC Packet Structure
```
+------------------+
| Address Pattern  |  Null-terminated string, padded to 4 bytes
+------------------+
| Type Tag String  |  Comma + type chars + null, padded to 4 bytes
+------------------+
| Arguments        |  Each argument padded to 4 bytes
+------------------+
```

### Padding Calculation
```c
size_t padded_length(size_t len) {
    return (len + 4) & ~3;  // Round up to next multiple of 4
}
```

### Example: Format Message Construction
```c
// /aoo/sink/1/format ,iiiiiisb
// Arguments: source_id, version, salt, channels, samplerate, blocksize, codec, options

void build_format_message(uint8_t *buffer, size_t *len,
                         int32_t sink_id, int32_t source_id, int32_t version,
                         int32_t salt, int32_t channels, int32_t samplerate,
                         int32_t blocksize, const char *codec,
                         const uint8_t *options, size_t options_len) {
    uint8_t *p = buffer;
    
    // Address pattern
    char addr[64];
    snprintf(addr, sizeof(addr), "/aoo/sink/%d/format", sink_id);
    size_t addr_len = strlen(addr) + 1;
    memcpy(p, addr, addr_len);
    p += padded_length(addr_len);
    
    // Type tag string
    const char *types = ",iiiiiisb";
    size_t types_len = strlen(types) + 1;
    memcpy(p, types, types_len);
    p += padded_length(types_len);
    
    // Arguments (all big-endian)
    write_int32_be(p, source_id); p += 4;
    write_int32_be(p, version); p += 4;
    write_int32_be(p, salt); p += 4;
    write_int32_be(p, channels); p += 4;
    write_int32_be(p, samplerate); p += 4;
    write_int32_be(p, blocksize); p += 4;
    
    // Codec string
    size_t codec_len = strlen(codec) + 1;
    memcpy(p, codec, codec_len);
    p += padded_length(codec_len);
    
    // Options blob (size + data)
    write_int32_be(p, options_len); p += 4;
    memcpy(p, options, options_len);
    p += padded_length(options_len);
    
    *len = p - buffer;
}
```

---

## Complete Message Reference Table

| Direction | Address | Type Tags | Description |
|-----------|---------|-----------|-------------|
| C?S (TCP) | /aoo/server/login | sssisih | Login request |
| S?C (TCP) | /aoo/client/login | i or is | Login response |
| C?S (TCP) | /aoo/server/group/join | ssT/ssF | Join group |
| S?C (TCP) | /aoo/client/group/join | si | Join response |
| C?S (TCP) | /aoo/server/group/leave | s | Leave group |
| S?C (TCP) | /aoo/client/peer/join | sssisih | Peer joined |
| S?C (TCP) | /aoo/client/peer/leave | ss | Peer left |
| C?S (UDP) | /aoo/server/request | (none) | Request public IP |
| S?C (UDP) | /aoo/client/reply | si | Public IP response |
| C?S (both) | /aoo/server/ping | (none) | Keepalive |
| P?P (UDP) | /aoo/peer/ping | h or (none) | NAT traversal |
| Sink?Src | /aoo/src/N/invite | i or ii | Invite source |
| Sink?Src | /aoo/src/N/uninvite | i | Uninvite source |
| Sink?Src | /aoo/src/N/format | ii | Request format |
| Sink?Src | /aoo/src/N/data | ii[ii]* | Request resend |
| Sink?Src | /aoo/src/N/ping | itti | Ping response |
| Sink?Src | /aoo/src/N/codec | i... | Codec change request |
| Src?Sink | /aoo/sink/N/format | iiiiiisb [,b] | Audio format (9th arg: optional userformat blob) |
| Src?Sink | /aoo/sink/N/data | iiidiiiib | Audio data |
| Src?Sink | /aoo/sink/N/ping | it | Source ping |
| Src?Sink | /d | iib or iidb | Compact data |

---

## Glossary

| Term | Definition |
|------|------------|
| AOO | Audio over OSC - the underlying protocol |
| OSC | Open Sound Control - message format used |
| Source | Entity that sends audio |
| Sink | Entity that receives audio |
| Salt | Session identifier, changes on reconnection |
| Sequence | Per-peer packet counter |
| Timetag | 64-bit NTP timestamp |
| SLIP | Serial Line Internet Protocol - TCP framing |
| Jitter Buffer | Buffer to handle network timing variations |
| NAT | Network Address Translation |
| FEC | Forward Error Correction |
| PLC | Packet Loss Concealment |
