Bernd Steckmeister
25ed765a03
6 Wochen uncommittete Arbeit (Voice-Channels, LiveKit-Manager, Settings-Modal u.v.m.) als ein WIP-Commit gesichert, damit nichts verloren geht und der Pi den aktuellen Stand klonen kann. Thematische Aufarbeitung: siehe ROADMAP M0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CPrAGBxBT6GfPXzeWQ4AXb
1488 lines
44 KiB
Markdown
1488 lines
44 KiB
Markdown
# 08b — LiveKit Streaming: Vollständige Implementierungsanleitung
|
|
|
|
> **Ziel:** Echtzeit Audio/Video-Calls und Screen-Sharing in Pyramid via LiveKit SFU,
|
|
> authentifiziert über MatrixRTC (Matrix-Standard), selbst gehostet auf dem Raspberry Pi.
|
|
|
|
---
|
|
|
|
## Inhaltsverzeichnis
|
|
- [Architektur-Überblick](#architektur-überblick)
|
|
- [Selbst-gehostete Infrastruktur](#selbst-gehostete-infrastruktur)
|
|
- [LiveKit-Server Setup (Raspberry Pi)](#livekit-server-setup-raspberry-pi)
|
|
- [LiveKit JWT Service (lk-jwt-service)](#livekit-jwt-service-lk-jwt-service)
|
|
- [Synapse MatrixRTC Konfiguration](#synapse-matrixrtc-konfiguration)
|
|
- [TURN-Server Setup (coturn)](#turn-server-setup-coturn)
|
|
- [Nginx Reverse-Proxy](#nginx-reverse-proxy)
|
|
- [Flutter: LiveKit-Paket & Abhängigkeiten](#flutter-livekit-paket--abhängigkeiten)
|
|
- [MatrixRTC Membership Flow (Signaling)](#matrixrtc-membership-flow-signaling)
|
|
- [LiveKit-Token holen (OpenID-Austausch)](#livekit-token-holen-openid-austausch)
|
|
- [LiveKit Room verbinden](#livekit-room-verbinden)
|
|
- [Audio-Call implementieren](#audio-call-implementieren)
|
|
- [Video-Call implementieren](#video-call-implementieren)
|
|
- [Screen-Sharing implementieren](#screen-sharing-implementieren)
|
|
- [Klingel-System (Ringing)](#klingel-system-ringing)
|
|
- [Call-State-Management (Riverpod)](#call-state-management-riverpod)
|
|
- [Call-UI Komponenten](#call-ui-komponenten)
|
|
- [Hintergrund-Service (Android)](#hintergrund-service-android)
|
|
- [Komplettes Dart-Implementierungs-Template](#komplettes-dart-implementierungs-template)
|
|
- [Troubleshooting & Diagnose](#troubleshooting--diagnose)
|
|
|
|
---
|
|
|
|
## Architektur-Überblick
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Pyramid App (Flutter) │
|
|
│ │
|
|
│ MatrixRTC Signaling LiveKit Media │
|
|
│ ┌──────────────────┐ ┌─────────────────────┐ │
|
|
│ │ Matrix SDK │ │ livekit_client pkg │ │
|
|
│ │ - call.member │ │ - Audio/Video tracks │ │
|
|
│ │ state events │ │ - Screen share │ │
|
|
│ │ - to-device msgs │ │ - Data channels │ │
|
|
│ └──────┬───────────┘ └──────────┬──────────┘ │
|
|
│ │ │ │
|
|
└─────────┼────────────────────────────────┼──────────────────────┘
|
|
│ HTTPS/WSS │ WSS (WebSocket Secure)
|
|
▼ ▼
|
|
┌─────────────────────┐ ┌──────────────────────────────┐
|
|
│ Synapse (Matrix HS)│ │ LiveKit Server (SFU) │
|
|
│ matrix.steggi- │ │ livekit.steggi-matrix.work │
|
|
│ matrix.work │ │ Port: 7880 (HTTPS/WS) │
|
|
│ │ │ Port: 7881 (TURN/TLS) │
|
|
│ OpenID Token API │ │ Port: 50100-50200 (UDP RTP) │
|
|
└────────┬────────────┘ └───────────────┬──────────────┘
|
|
│ OpenID Token │
|
|
▼ │
|
|
┌────────────────────┐ │
|
|
│ lk-jwt-service │───────── JWT Token ───────┘
|
|
│ (Token Exchange) │
|
|
│ Port: 8080 │
|
|
└────────────────────┘
|
|
│
|
|
▼
|
|
┌────────────────────┐
|
|
│ coturn (TURN) │
|
|
│ Port: 3478 UDP │
|
|
│ Port: 5349 TLS │
|
|
└────────────────────┘
|
|
|
|
Nginx Reverse Proxy (Port 443):
|
|
/livekit/ → LiveKit :7880
|
|
/lk-jwt/ → lk-jwt-service :8080
|
|
(coturn läuft direkt auf 3478/5349)
|
|
```
|
|
|
|
---
|
|
|
|
## Selbst-gehostete Infrastruktur
|
|
|
|
### Benötigte Komponenten
|
|
|
|
| Dienst | Zweck | Port | Software |
|
|
|--------|-------|------|----------|
|
|
| **LiveKit Server** | SFU — verteilt Media-Streams | 7880 (WS/HTTP), 7881 (TURN/TLS), 50100-50200 UDP | `livekit/livekit-server` |
|
|
| **lk-jwt-service** | Token-Austausch OpenID → LiveKit JWT | 8080 | `element-hq/lk-jwt-service` |
|
|
| **coturn** | TURN-Server für NAT-Traversal | 3478 UDP, 5349 TLS | `coturn` |
|
|
| **Nginx** | Reverse Proxy + SSL-Termination | 443 | `nginx` |
|
|
| **Synapse** | Matrix Homeserver (bereits vorhanden) | 8448 | ✓ läuft |
|
|
|
|
### DNS-Einträge (Beispiel für steggi-matrix.work)
|
|
```
|
|
livekit.steggi-matrix.work A <Pi-IP>
|
|
turn.steggi-matrix.work A <Pi-IP>
|
|
```
|
|
|
|
---
|
|
|
|
## LiveKit-Server Setup (Raspberry Pi)
|
|
|
|
### Installation
|
|
|
|
```bash
|
|
# LiveKit Binary herunterladen (ARM64 für Raspberry Pi 4/5)
|
|
curl -sSL https://get.livekit.io | bash
|
|
# oder manuell:
|
|
wget https://github.com/livekit/livekit/releases/latest/download/livekit_linux_arm64.tar.gz
|
|
tar xzf livekit_linux_arm64.tar.gz
|
|
sudo mv livekit-server /usr/local/bin/
|
|
|
|
# API-Key und Secret generieren
|
|
livekit-server generate-keys
|
|
# Ausgabe:
|
|
# API Key: APIxxxxxxxxxxxxxxxxxx
|
|
# API Secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
# → SICHER AUFBEWAHREN!
|
|
```
|
|
|
|
### Konfiguration: `/etc/livekit/livekit.yaml`
|
|
|
|
```yaml
|
|
port: 7880
|
|
bind_addresses:
|
|
- "" # Alle Interfaces
|
|
|
|
# API-Schlüssel (aus generate-keys)
|
|
keys:
|
|
APIxxxxxxxxxxxxxxxxxx: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
|
|
# WebRTC-Einstellungen
|
|
rtc:
|
|
tcp_port: 7881 # TURN over TLS
|
|
port_range_start: 50100
|
|
port_range_end: 50200
|
|
# Öffentliche IP des Raspberry Pi (wichtig für NAT!)
|
|
use_external_ip: true
|
|
# STUN-Server für ICE (externe Erreichbarkeit prüfen)
|
|
stun_servers:
|
|
- stun:stun.l.google.com:19302
|
|
|
|
# TURN eingebaut (LiveKit kann selbst TURN machen)
|
|
turn:
|
|
enabled: true
|
|
domain: livekit.steggi-matrix.work
|
|
tls_port: 5349 # TURN über TLS
|
|
udp_port: 3478
|
|
|
|
# Logging
|
|
logging:
|
|
level: info
|
|
pion_level: error # WebRTC-interne Logs reduzieren
|
|
|
|
# Redis (optional, für Cluster — beim Pi nicht nötig)
|
|
# redis:
|
|
# address: localhost:6379
|
|
```
|
|
|
|
### Systemd Service: `/etc/systemd/system/livekit.service`
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=LiveKit SFU Server
|
|
After=network.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=livekit
|
|
Group=livekit
|
|
ExecStart=/usr/local/bin/livekit-server --config /etc/livekit/livekit.yaml
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
LimitNOFILE=65536
|
|
|
|
# Logs
|
|
StandardOutput=journal
|
|
StandardError=journal
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
```bash
|
|
# User anlegen und Service aktivieren
|
|
sudo useradd -r -s /bin/false livekit
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable livekit
|
|
sudo systemctl start livekit
|
|
sudo systemctl status livekit
|
|
|
|
# Logs verfolgen
|
|
sudo journalctl -u livekit -f
|
|
```
|
|
|
|
---
|
|
|
|
## LiveKit JWT Service (lk-jwt-service)
|
|
|
|
Dieser Service tauscht Matrix OpenID-Tokens gegen LiveKit JWTs aus.
|
|
Das ist die **Brücke zwischen Matrix-Authentifizierung und LiveKit**.
|
|
|
|
### Installation
|
|
|
|
```bash
|
|
# Go installieren (falls nicht vorhanden)
|
|
# Für Raspberry Pi (ARM64):
|
|
wget https://go.dev/dl/go1.22.0.linux-arm64.tar.gz
|
|
sudo tar -C /usr/local -xzf go1.22.0.linux-arm64.tar.gz
|
|
export PATH=$PATH:/usr/local/go/bin
|
|
|
|
# lk-jwt-service bauen
|
|
git clone https://github.com/element-hq/lk-jwt-service
|
|
cd lk-jwt-service
|
|
go build -o lk-jwt-service .
|
|
sudo mv lk-jwt-service /usr/local/bin/
|
|
```
|
|
|
|
### Konfiguration: `/etc/lk-jwt-service/config.yaml`
|
|
|
|
```yaml
|
|
# LiveKit-Verbindung
|
|
livekit_url: wss://livekit.steggi-matrix.work
|
|
livekit_key: APIxxxxxxxxxxxxxxxxxx # selber Key wie in livekit.yaml
|
|
livekit_secret: xxxxxxxxxxxxxxxxxxxxx # selbes Secret
|
|
|
|
# Matrix-Homeserver für OpenID-Validierung
|
|
matrix_server: https://matrix.steggi-matrix.work
|
|
|
|
# Welche Matrix-Homeserver dürfen den Service nutzen
|
|
allowed_matrix_servers:
|
|
- steggi-matrix.work
|
|
|
|
# Server-Port
|
|
port: 8080
|
|
```
|
|
|
|
### Systemd Service: `/etc/systemd/system/lk-jwt-service.service`
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=LiveKit JWT Token Service for MatrixRTC
|
|
After=network.target livekit.service
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=livekit
|
|
ExecStart=/usr/local/bin/lk-jwt-service --config /etc/lk-jwt-service/config.yaml
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
```bash
|
|
sudo systemctl enable lk-jwt-service
|
|
sudo systemctl start lk-jwt-service
|
|
```
|
|
|
|
### API-Endpoint des lk-jwt-service
|
|
|
|
```
|
|
POST /sfu/get
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"room": "!roomId:server",
|
|
"openid_token": {
|
|
"access_token": "...", ← von client.requestOpenIdToken()
|
|
"token_type": "Bearer",
|
|
"matrix_server_name": "steggi-matrix.work"
|
|
},
|
|
"device_id": "PYRAMID_DEVICE_ID"
|
|
}
|
|
|
|
Antwort:
|
|
{
|
|
"url": "wss://livekit.steggi-matrix.work",
|
|
"jwt": "eyJ..." ← LiveKit JWT Token
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Synapse MatrixRTC Konfiguration
|
|
|
|
In der Synapse-Konfiguration (`homeserver.yaml`):
|
|
|
|
```yaml
|
|
# MatrixRTC / LiveKit Integration
|
|
experimental_features:
|
|
msc3401_enabled: true # MatrixRTC Group Calls
|
|
msc4140_enabled: true # Delayed events (für Membership-Cleanup)
|
|
|
|
# Erlaubt das Senden von com.famedly.call.member State-Events
|
|
rc_message:
|
|
per_second: 100
|
|
burst_count: 200
|
|
|
|
# LiveKit-Service URL im .well-known bekannt machen (MSC4143)
|
|
# → client kann automatisch den richtigen JWT-Service finden
|
|
```
|
|
|
|
### `.well-known/matrix/client` erweitern
|
|
|
|
Nginx liefert diese JSON-Datei. Ergänze LiveKit-Info:
|
|
|
|
```json
|
|
{
|
|
"m.homeserver": {
|
|
"base_url": "https://matrix.steggi-matrix.work"
|
|
},
|
|
"org.matrix.msc4143.rtc_foci": [
|
|
{
|
|
"type": "livekit",
|
|
"livekit_service_url": "https://livekit.steggi-matrix.work/lk-jwt"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## TURN-Server Setup (coturn)
|
|
|
|
LiveKit hat einen eingebauten TURN-Server, aber coturn als Fallback ist empfohlen.
|
|
|
|
```bash
|
|
sudo apt install coturn -y
|
|
```
|
|
|
|
### `/etc/turnserver.conf`
|
|
|
|
```conf
|
|
# Listening
|
|
listening-port=3478
|
|
tls-listening-port=5349
|
|
listening-ip=0.0.0.0
|
|
|
|
# Externe IP (Raspberry Pi öffentliche IP)
|
|
external-ip=<DEINE_ÖFFENTLICHE_IP>
|
|
|
|
# Realm
|
|
realm=turn.steggi-matrix.work
|
|
server-name=turn.steggi-matrix.work
|
|
|
|
# TLS-Zertifikate (Let's Encrypt)
|
|
cert=/etc/letsencrypt/live/turn.steggi-matrix.work/fullchain.pem
|
|
pkey=/etc/letsencrypt/live/turn.steggi-matrix.work/privkey.pem
|
|
|
|
# Auth (shared secret für Matrix)
|
|
use-auth-secret
|
|
static-auth-secret=ZUFAELLIGES_LANGES_GEHEIMNIS_HIER
|
|
|
|
# Relay-Ports
|
|
min-port=49152
|
|
max-port=65535
|
|
|
|
# Sicherheit
|
|
no-multicast-peers
|
|
no-loopback-peers
|
|
denied-peer-ip=10.0.0.0-10.255.255.255
|
|
denied-peer-ip=192.168.0.0-192.168.255.255
|
|
denied-peer-ip=172.16.0.0-172.31.255.255
|
|
|
|
# Logging
|
|
log-file=/var/log/turnserver.log
|
|
```
|
|
|
|
```bash
|
|
# TURN-Service aktivieren
|
|
sudo systemctl enable coturn
|
|
sudo systemctl start coturn
|
|
|
|
# Synapse: TURN-Credentials bekannt machen (in homeserver.yaml)
|
|
turn_uris:
|
|
- "turn:turn.steggi-matrix.work:3478?transport=udp"
|
|
- "turn:turn.steggi-matrix.work:3478?transport=tcp"
|
|
- "turns:turn.steggi-matrix.work:5349?transport=tcp"
|
|
turn_shared_secret: "ZUFAELLIGES_LANGES_GEHEIMNIS_HIER"
|
|
turn_user_lifetime: 86400000
|
|
turn_allow_guests: false
|
|
```
|
|
|
|
---
|
|
|
|
## Nginx Reverse-Proxy
|
|
|
|
```nginx
|
|
# /etc/nginx/sites-available/livekit.steggi-matrix.work
|
|
|
|
server {
|
|
listen 443 ssl http2;
|
|
server_name livekit.steggi-matrix.work;
|
|
|
|
ssl_certificate /etc/letsencrypt/live/livekit.steggi-matrix.work/fullchain.pem;
|
|
ssl_certificate_key /etc/letsencrypt/live/livekit.steggi-matrix.work/privkey.pem;
|
|
|
|
# LiveKit WebSocket + HTTP API
|
|
location / {
|
|
proxy_pass http://localhost:7880;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
proxy_read_timeout 3600s;
|
|
proxy_send_timeout 3600s;
|
|
proxy_buffering off;
|
|
client_max_body_size 0;
|
|
}
|
|
|
|
# lk-jwt-service (Token-Austausch)
|
|
location /lk-jwt/ {
|
|
rewrite ^/lk-jwt/(.*) /$1 break;
|
|
proxy_pass http://localhost:8080;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
add_header 'Access-Control-Allow-Origin' '*' always;
|
|
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
|
|
add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type' always;
|
|
}
|
|
}
|
|
|
|
# HTTP → HTTPS Redirect
|
|
server {
|
|
listen 80;
|
|
server_name livekit.steggi-matrix.work;
|
|
return 301 https://$host$request_uri;
|
|
}
|
|
```
|
|
|
|
```bash
|
|
sudo nginx -t && sudo nginx -s reload
|
|
```
|
|
|
|
### Firewall-Regeln
|
|
|
|
```bash
|
|
# UFW
|
|
sudo ufw allow 443/tcp # HTTPS (LiveKit + lk-jwt)
|
|
sudo ufw allow 3478/udp # TURN UDP
|
|
sudo ufw allow 3478/tcp # TURN TCP
|
|
sudo ufw allow 5349/tcp # TURNS TLS
|
|
sudo ufw allow 7881/tcp # LiveKit TURN/TLS (eingebaut)
|
|
sudo ufw allow 50100:50200/udp # RTP Media Ports
|
|
|
|
# Für Raspberry Pi hinter Router: Port-Forwarding nötig!
|
|
# Im Router: alle oben genannten Ports → Pi-interne IP weiterleiten
|
|
```
|
|
|
|
---
|
|
|
|
## Flutter: LiveKit-Paket & Abhängigkeiten
|
|
|
|
### `pubspec.yaml`
|
|
|
|
```yaml
|
|
dependencies:
|
|
livekit_client: ^2.4.0 # LiveKit Flutter SDK
|
|
flutter_webrtc: ^0.10.0 # WebRTC (wird von livekit_client benötigt)
|
|
permission_handler: ^11.0.0 # Mikrofon/Kamera-Berechtigung
|
|
wakelock_plus: ^1.0.0 # Screen während Call aktiv halten
|
|
|
|
# Bereits vorhanden:
|
|
matrix: ^6.x.x
|
|
flutter_riverpod: ^2.x.x
|
|
```
|
|
|
|
### Android: `android/app/src/main/AndroidManifest.xml`
|
|
|
|
```xml
|
|
<!-- Kamera und Mikrofon -->
|
|
<uses-permission android:name="android.permission.CAMERA"/>
|
|
<uses-permission android:name="android.permission.RECORD_AUDIO"/>
|
|
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>
|
|
<uses-permission android:name="android.permission.BLUETOOTH" android:maxSdkVersion="30"/>
|
|
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT"/>
|
|
|
|
<!-- Foreground-Service für Call im Hintergrund -->
|
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
|
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE"/>
|
|
|
|
<!-- Screen-Sharing -->
|
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION"/>
|
|
|
|
<!-- Wakelock (Screen während Call) -->
|
|
<uses-permission android:name="android.permission.WAKE_LOCK"/>
|
|
|
|
<!-- Foreground-Service registrieren -->
|
|
<service
|
|
android:name=".CallForegroundService"
|
|
android:foregroundServiceType="microphone|mediaProjection"
|
|
android:exported="false"/>
|
|
```
|
|
|
|
### iOS: `ios/Runner/Info.plist`
|
|
|
|
```xml
|
|
<key>NSCameraUsageDescription</key>
|
|
<string>Pyramid benötigt die Kamera für Video-Calls</string>
|
|
<key>NSMicrophoneUsageDescription</key>
|
|
<string>Pyramid benötigt das Mikrofon für Audio-Calls</string>
|
|
```
|
|
|
|
---
|
|
|
|
## MatrixRTC Membership Flow (Signaling)
|
|
|
|
MatrixRTC verwendet State-Events im Raum um anzuzeigen, wer im Call ist.
|
|
|
|
```dart
|
|
// lib/features/call/matrixrtc_service.dart
|
|
|
|
const kCallMemberEventType = 'org.matrix.msc3401.call.member';
|
|
|
|
class MatrixRtcMembership {
|
|
final String roomId;
|
|
final String deviceId;
|
|
final String callId;
|
|
final String livekitServiceUrl;
|
|
|
|
MatrixRtcMembership({
|
|
required this.roomId,
|
|
required this.deviceId,
|
|
required this.callId,
|
|
required this.livekitServiceUrl,
|
|
});
|
|
|
|
// ── Call beitreten: Membership-Event senden ────────────────────────────
|
|
|
|
Future<void> join(Client matrixClient) async {
|
|
final expiresTs = DateTime.now()
|
|
.add(const Duration(hours: 4))
|
|
.millisecondsSinceEpoch;
|
|
|
|
await matrixClient.sendState(
|
|
roomId,
|
|
kCallMemberEventType,
|
|
{
|
|
'application': 'm.call',
|
|
'call_id': callId,
|
|
'device_id': deviceId,
|
|
'expires_ts': expiresTs,
|
|
'membershipID': '${deviceId}_${DateTime.now().millisecondsSinceEpoch}',
|
|
'backend': {
|
|
'type': 'livekit',
|
|
'livekit_service_url': livekitServiceUrl,
|
|
},
|
|
},
|
|
stateKey: '$deviceId:$callId',
|
|
);
|
|
}
|
|
|
|
// ── Call verlassen: Membership entfernen ───────────────────────────────
|
|
|
|
Future<void> leave(Client matrixClient) async {
|
|
await matrixClient.sendState(
|
|
roomId,
|
|
kCallMemberEventType,
|
|
{}, // Leer = Austritt
|
|
stateKey: '$deviceId:$callId',
|
|
);
|
|
}
|
|
|
|
// ── Aktive Mitglieder auflisten ───────────────────────────────────────
|
|
|
|
static List<Map<String, dynamic>> getActiveMembers(Room room, String callId) {
|
|
final states = room.states[kCallMemberEventType] ?? {};
|
|
final now = DateTime.now().millisecondsSinceEpoch;
|
|
return states.values
|
|
.where((e) {
|
|
final content = e.content;
|
|
return content['call_id'] == callId &&
|
|
(content['expires_ts'] as int? ?? 0) > now &&
|
|
content.isNotEmpty;
|
|
})
|
|
.map((e) => Map<String, dynamic>.from(e.content))
|
|
.toList();
|
|
}
|
|
|
|
// ── Auf Membership-Änderungen hören ──────────────────────────────────
|
|
|
|
static Stream<void> watchCallMembers(Client client, String roomId) {
|
|
return client.onSync.stream
|
|
.where((update) {
|
|
final rooms = update.rooms?.join;
|
|
if (rooms == null) return false;
|
|
final roomUpdate = rooms[roomId];
|
|
return roomUpdate?.state?.events?.any(
|
|
(e) => e.type == kCallMemberEventType) ?? false;
|
|
})
|
|
.map((_) {});
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## LiveKit-Token holen (OpenID-Austausch)
|
|
|
|
```dart
|
|
// lib/features/call/livekit_service.dart
|
|
|
|
import 'dart:convert';
|
|
import 'package:http/http.dart' as http;
|
|
import 'package:matrix/matrix.dart';
|
|
|
|
class LiveKitService {
|
|
static const String _jwtServicePath = '/lk-jwt/sfu/get';
|
|
|
|
/// Tauscht ein Matrix OpenID-Token gegen einen LiveKit JWT.
|
|
/// [livekitServiceUrl] kommt aus dem MatrixRTC-Membership-Event des Raums.
|
|
static Future<LiveKitCredentials> getToken({
|
|
required Client matrixClient,
|
|
required String roomId,
|
|
required String livekitServiceUrl,
|
|
}) async {
|
|
// 1. OpenID-Token vom Homeserver holen
|
|
final openIdToken = await matrixClient.requestOpenIdToken(
|
|
matrixClient.userID!,
|
|
);
|
|
|
|
// 2. JWT beim lk-jwt-service tauschen
|
|
final serviceUrl = Uri.parse(livekitServiceUrl).resolve(_jwtServicePath);
|
|
final response = await http.post(
|
|
serviceUrl,
|
|
headers: {'Content-Type': 'application/json'},
|
|
body: jsonEncode({
|
|
'room': roomId,
|
|
'openid_token': {
|
|
'access_token': openIdToken.accessToken,
|
|
'token_type': 'Bearer',
|
|
'matrix_server_name': openIdToken.matrixServerName,
|
|
},
|
|
'device_id': matrixClient.deviceID,
|
|
}),
|
|
).timeout(const Duration(seconds: 10));
|
|
|
|
if (response.statusCode != 200) {
|
|
throw Exception(
|
|
'lk-jwt-service Fehler ${response.statusCode}: ${response.body}');
|
|
}
|
|
|
|
final data = jsonDecode(response.body) as Map<String, dynamic>;
|
|
return LiveKitCredentials(
|
|
url: data['url'] as String,
|
|
jwt: data['jwt'] as String,
|
|
);
|
|
}
|
|
}
|
|
|
|
class LiveKitCredentials {
|
|
final String url; // wss://livekit.steggi-matrix.work
|
|
final String jwt; // LiveKit JWT Token
|
|
|
|
const LiveKitCredentials({required this.url, required this.jwt});
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## LiveKit Room verbinden
|
|
|
|
```dart
|
|
// lib/features/call/livekit_room_manager.dart
|
|
|
|
import 'package:livekit_client/livekit_client.dart';
|
|
|
|
class LiveKitRoomManager {
|
|
Room? _room;
|
|
EventsListener<RoomEvent>? _listener;
|
|
|
|
Room get room => _room!;
|
|
bool get isConnected => _room?.connectionState == ConnectionState.connected;
|
|
|
|
Future<void> connect({
|
|
required String url,
|
|
required String token,
|
|
bool audioEnabled = true,
|
|
bool videoEnabled = false,
|
|
}) async {
|
|
_room = Room(
|
|
roomOptions: RoomOptions(
|
|
adaptiveStream: true, // Qualität automatisch anpassen
|
|
dynacast: true, // Inaktive Streams pausieren
|
|
defaultAudioPublishOptions: const AudioPublishOptions(
|
|
name: 'microphone',
|
|
audioBitrate: 24000, // 24 kbps — gut für Sprache
|
|
),
|
|
defaultVideoPublishOptions: VideoPublishOptions(
|
|
simulcast: true, // Mehrere Qualitätsstufen senden
|
|
videoEncoding: VideoEncoding(
|
|
maxBitrate: 1200 * 1000, // 1.2 Mbps max
|
|
maxFramerate: 30,
|
|
),
|
|
),
|
|
defaultScreenShareCaptureOptions: const ScreenShareCaptureOptions(
|
|
useiOSBroadcastExtension: true,
|
|
params: VideoParameters(
|
|
dimensions: VideoDimensions(1920, 1080),
|
|
encoding: VideoEncoding(maxBitrate: 3 * 1000 * 1000, maxFramerate: 15),
|
|
),
|
|
),
|
|
defaultCameraCaptureOptions: CameraCaptureOptions(
|
|
cameraPosition: CameraPosition.front,
|
|
params: VideoParameters(
|
|
dimensions: VideoDimensions(1280, 720),
|
|
encoding: VideoEncoding(maxBitrate: 1200 * 1000, maxFramerate: 30),
|
|
),
|
|
),
|
|
),
|
|
);
|
|
|
|
_listener = _room!.createListener();
|
|
|
|
try {
|
|
await _room!.connect(url, token,
|
|
connectOptions: const ConnectOptions(
|
|
autoSubscribe: true, // Alle Tracks automatisch empfangen
|
|
rtcConfig: RTCConfiguration(
|
|
iceTransportPolicy: RTCIceTransportPolicy.all,
|
|
),
|
|
),
|
|
);
|
|
|
|
// Mikrofon aktivieren
|
|
if (audioEnabled) {
|
|
await _room!.localParticipant?.setMicrophoneEnabled(true);
|
|
}
|
|
// Kamera aktivieren
|
|
if (videoEnabled) {
|
|
await _room!.localParticipant?.setCameraEnabled(true);
|
|
}
|
|
} catch (e) {
|
|
await _room?.dispose();
|
|
_room = null;
|
|
rethrow;
|
|
}
|
|
}
|
|
|
|
Future<void> disconnect() async {
|
|
await _room?.localParticipant?.setMicrophoneEnabled(false);
|
|
await _room?.localParticipant?.setCameraEnabled(false);
|
|
await _room?.disconnect();
|
|
await _room?.dispose();
|
|
_room = null;
|
|
}
|
|
|
|
// Event-Listener
|
|
void onParticipantConnected(void Function(RemoteParticipant) callback) {
|
|
_listener?.on<ParticipantConnectedEvent>(
|
|
(event) => callback(event.participant));
|
|
}
|
|
|
|
void onParticipantDisconnected(void Function(RemoteParticipant) callback) {
|
|
_listener?.on<ParticipantDisconnectedEvent>(
|
|
(event) => callback(event.participant));
|
|
}
|
|
|
|
void onTrackPublished(
|
|
void Function(RemoteParticipant, RemoteTrackPublication) callback) {
|
|
_listener?.on<TrackPublishedEvent>((event) =>
|
|
callback(event.participant, event.publication));
|
|
}
|
|
|
|
void onTrackSubscribed(
|
|
void Function(Track, TrackPublication, RemoteParticipant) callback) {
|
|
_listener?.on<TrackSubscribedEvent>(
|
|
(event) => callback(event.track, event.publication, event.participant));
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Audio-Call implementieren
|
|
|
|
```dart
|
|
// lib/features/call/call_notifier.dart (Audio-Call)
|
|
|
|
import 'package:permission_handler/permission_handler.dart';
|
|
|
|
Future<void> startAudioCall({
|
|
required Client matrixClient,
|
|
required String roomId,
|
|
}) async {
|
|
// 1. Berechtigung anfordern
|
|
final status = await Permission.microphone.request();
|
|
if (!status.isGranted) throw Exception('Mikrofon-Berechtigung verweigert');
|
|
|
|
// 2. LiveKit-Service-URL aus Raum-Metadaten holen (oder Default)
|
|
const livekitServiceUrl = 'https://livekit.steggi-matrix.work';
|
|
|
|
// 3. MatrixRTC Membership senden
|
|
final callId = roomId; // Convention: callId = roomId
|
|
final membership = MatrixRtcMembership(
|
|
roomId: roomId,
|
|
deviceId: matrixClient.deviceID!,
|
|
callId: callId,
|
|
livekitServiceUrl: livekitServiceUrl,
|
|
);
|
|
await membership.join(matrixClient);
|
|
|
|
// 4. LiveKit-Token holen
|
|
final credentials = await LiveKitService.getToken(
|
|
matrixClient: matrixClient,
|
|
roomId: roomId,
|
|
livekitServiceUrl: livekitServiceUrl,
|
|
);
|
|
|
|
// 5. Mit LiveKit verbinden (nur Audio)
|
|
final manager = LiveKitRoomManager();
|
|
await manager.connect(
|
|
url: credentials.url,
|
|
token: credentials.jwt,
|
|
audioEnabled: true,
|
|
videoEnabled: false,
|
|
);
|
|
|
|
// 6. Screen aktiv halten
|
|
WakelockPlus.enable();
|
|
}
|
|
|
|
// Stummschalten
|
|
Future<void> setMuted(LiveKitRoomManager manager, bool muted) async {
|
|
await manager.room.localParticipant?.setMicrophoneEnabled(!muted);
|
|
}
|
|
|
|
// Lautsprecher umschalten (Android)
|
|
Future<void> setSpeakerOn(bool speakerOn) async {
|
|
await Helper.setSpeakerphoneOn(speakerOn);
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Video-Call implementieren
|
|
|
|
```dart
|
|
// Widget: Lokale Kamera-Vorschau
|
|
class LocalVideoView extends StatelessWidget {
|
|
final LocalParticipant participant;
|
|
const LocalVideoView({required this.participant, super.key});
|
|
|
|
@override
|
|
Widget build(BuildContext context) {
|
|
return StreamBuilder(
|
|
stream: participant.trackPublicationsNotifier,
|
|
builder: (ctx, _) {
|
|
final videoPub = participant.videoTrackPublications
|
|
.where((p) => p.source == TrackSource.camera)
|
|
.firstOrNull;
|
|
if (videoPub?.track case final VideoTrack track) {
|
|
return VideoTrackRenderer(track);
|
|
}
|
|
return const SizedBox();
|
|
},
|
|
);
|
|
}
|
|
}
|
|
|
|
// Widget: Remote-Teilnehmer Video
|
|
class RemoteVideoView extends StatelessWidget {
|
|
final RemoteParticipant participant;
|
|
const RemoteVideoView({required this.participant, super.key});
|
|
|
|
@override
|
|
Widget build(BuildContext context) {
|
|
return StreamBuilder(
|
|
stream: participant.trackPublicationsNotifier,
|
|
builder: (ctx, _) {
|
|
final videoPub = participant.videoTrackPublications
|
|
.where((p) =>
|
|
p.source == TrackSource.camera ||
|
|
p.source == TrackSource.screenShareVideo)
|
|
.firstOrNull;
|
|
if (videoPub?.track case final VideoTrack track) {
|
|
return VideoTrackRenderer(track, fit: RTCVideoViewObjectFit.contain);
|
|
}
|
|
return Container(
|
|
color: Colors.black87,
|
|
child: const Center(
|
|
child: Icon(Icons.person, size: 80, color: Colors.white38),
|
|
),
|
|
);
|
|
},
|
|
);
|
|
}
|
|
}
|
|
|
|
// Kamera ein-/ausschalten
|
|
Future<void> toggleCamera(Room room) async {
|
|
final enabled = room.localParticipant?.isCameraEnabled() ?? false;
|
|
await room.localParticipant?.setCameraEnabled(!enabled);
|
|
}
|
|
|
|
// Kamera wechseln (Vorder-/Rückkamera)
|
|
Future<void> switchCamera(Room room) async {
|
|
final cameraPub = room.localParticipant?.videoTrackPublications
|
|
.where((p) => p.source == TrackSource.camera)
|
|
.firstOrNull;
|
|
if (cameraPub?.track case final LocalVideoTrack track) {
|
|
await track.switchCamera();
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Screen-Sharing implementieren
|
|
|
|
```dart
|
|
// Android: benötigt MediaProjection-Foreground-Service
|
|
|
|
// Screen-Sharing starten
|
|
Future<void> startScreenShare(Room room) async {
|
|
// Android: MediaProjection-Berechtigung anfordern (System-Dialog)
|
|
await room.localParticipant?.setScreenShareEnabled(
|
|
true,
|
|
captureScreenAudio: true, // Audio mitschneiden
|
|
);
|
|
}
|
|
|
|
// Screen-Sharing beenden
|
|
Future<void> stopScreenShare(Room room) async {
|
|
await room.localParticipant?.setScreenShareEnabled(false);
|
|
}
|
|
|
|
// Android Foreground Service für Screen-Sharing
|
|
// android/app/src/main/kotlin/chat/pyramid/pyramid/ScreenShareService.kt:
|
|
/*
|
|
class ScreenShareService : Service() {
|
|
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
|
|
val notification = NotificationCompat.Builder(this, "call_channel")
|
|
.setSmallIcon(R.drawable.ic_notification)
|
|
.setContentTitle("Bildschirm wird geteilt")
|
|
.setContentText("Pyramid teilt deinen Bildschirm")
|
|
.build()
|
|
startForeground(NOTIF_ID, notification,
|
|
ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION)
|
|
return START_STICKY
|
|
}
|
|
companion object { const val NOTIF_ID = 2001 }
|
|
}
|
|
*/
|
|
```
|
|
|
|
---
|
|
|
|
## Klingel-System (Ringing)
|
|
|
|
```dart
|
|
// lib/features/call/ringing_service.dart
|
|
|
|
// ── Anruf ankündigen (to-device Event) ─────────────────────────────────
|
|
|
|
Future<void> ringUser({
|
|
required Client matrixClient,
|
|
required String targetUserId,
|
|
required String roomId,
|
|
required String callId,
|
|
}) async {
|
|
await matrixClient.sendToDevice(
|
|
'com.famedly.call', // Event-Typ (wie element/fluffychat)
|
|
{
|
|
targetUserId: {
|
|
'*': { // Alle Geräte des Users
|
|
'call_id': callId,
|
|
'room_id': roomId,
|
|
'type': 'invite',
|
|
'sender_session_id': matrixClient.deviceID,
|
|
'sender_name': matrixClient.displayName,
|
|
'version': 1,
|
|
},
|
|
},
|
|
},
|
|
);
|
|
}
|
|
|
|
// ── Klingeln abbrechen ────────────────────────────────────────────────
|
|
|
|
Future<void> cancelRing({
|
|
required Client matrixClient,
|
|
required String targetUserId,
|
|
required String callId,
|
|
}) async {
|
|
await matrixClient.sendToDevice(
|
|
'com.famedly.call',
|
|
{
|
|
targetUserId: {
|
|
'*': {'call_id': callId, 'type': 'hangup', 'reason': 'user_hangup'},
|
|
},
|
|
},
|
|
);
|
|
}
|
|
|
|
// ── Eingehende to-device Events abhören ──────────────────────────────
|
|
|
|
StreamSubscription<ToDeviceEvent> listenForIncomingCalls(
|
|
Client matrixClient, void Function(IncomingCall) onCall) {
|
|
return matrixClient.onToDeviceEvent.stream
|
|
.where((e) => e.type == 'com.famedly.call')
|
|
.listen((event) {
|
|
final type = event.content['type'] as String?;
|
|
final callId = event.content['call_id'] as String?;
|
|
final roomId = event.content['room_id'] as String?;
|
|
if (type == 'invite' && callId != null && roomId != null) {
|
|
onCall(IncomingCall(
|
|
callId: callId,
|
|
roomId: roomId,
|
|
callerId: event.senderId,
|
|
senderName: event.content['sender_name'] as String?,
|
|
));
|
|
}
|
|
if (type == 'hangup') {
|
|
// Klingeln beenden (Anruf abgebrochen)
|
|
}
|
|
});
|
|
}
|
|
|
|
class IncomingCall {
|
|
final String callId;
|
|
final String roomId;
|
|
final String callerId;
|
|
final String? senderName;
|
|
const IncomingCall({
|
|
required this.callId,
|
|
required this.roomId,
|
|
required this.callerId,
|
|
this.senderName,
|
|
});
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Call-State-Management (Riverpod)
|
|
|
|
```dart
|
|
// lib/features/call/call_provider.dart
|
|
|
|
enum CallState { idle, calling, ringing, connecting, connected, ended, failed }
|
|
enum CallType { audio, video }
|
|
|
|
@immutable
|
|
class CallStateData {
|
|
final CallState state;
|
|
final CallType type;
|
|
final String? roomId;
|
|
final String? callId;
|
|
final bool isMuted;
|
|
final bool isCameraOff;
|
|
final bool isSpeakerOn;
|
|
final bool isScreenSharing;
|
|
final List<RemoteParticipant> participants;
|
|
|
|
const CallStateData({
|
|
this.state = CallState.idle,
|
|
this.type = CallType.audio,
|
|
this.roomId,
|
|
this.callId,
|
|
this.isMuted = false,
|
|
this.isCameraOff = false,
|
|
this.isSpeakerOn = false,
|
|
this.isScreenSharing = false,
|
|
this.participants = const [],
|
|
});
|
|
|
|
CallStateData copyWith({
|
|
CallState? state,
|
|
CallType? type,
|
|
String? roomId,
|
|
String? callId,
|
|
bool? isMuted,
|
|
bool? isCameraOff,
|
|
bool? isSpeakerOn,
|
|
bool? isScreenSharing,
|
|
List<RemoteParticipant>? participants,
|
|
}) => CallStateData(
|
|
state: state ?? this.state,
|
|
type: type ?? this.type,
|
|
roomId: roomId ?? this.roomId,
|
|
callId: callId ?? this.callId,
|
|
isMuted: isMuted ?? this.isMuted,
|
|
isCameraOff: isCameraOff ?? this.isCameraOff,
|
|
isSpeakerOn: isSpeakerOn ?? this.isSpeakerOn,
|
|
isScreenSharing: isScreenSharing ?? this.isScreenSharing,
|
|
participants: participants ?? this.participants,
|
|
);
|
|
}
|
|
|
|
class CallNotifier extends StateNotifier<CallStateData> {
|
|
final Ref _ref;
|
|
LiveKitRoomManager? _manager;
|
|
MatrixRtcMembership? _membership;
|
|
|
|
CallNotifier(this._ref) : super(const CallStateData());
|
|
|
|
Future<void> startCall(String roomId, CallType type) async {
|
|
state = state.copyWith(state: CallState.calling, type: type, roomId: roomId);
|
|
try {
|
|
final client = await _ref.read(matrixClientProvider.future);
|
|
final callId = roomId;
|
|
const livekitServiceUrl = 'https://livekit.steggi-matrix.work';
|
|
|
|
_membership = MatrixRtcMembership(
|
|
roomId: roomId, deviceId: client.deviceID!,
|
|
callId: callId, livekitServiceUrl: livekitServiceUrl,
|
|
);
|
|
await _membership!.join(client);
|
|
|
|
final credentials = await LiveKitService.getToken(
|
|
matrixClient: client, roomId: roomId, livekitServiceUrl: livekitServiceUrl,
|
|
);
|
|
|
|
_manager = LiveKitRoomManager();
|
|
state = state.copyWith(state: CallState.connecting, callId: callId);
|
|
|
|
await _manager!.connect(
|
|
url: credentials.url,
|
|
token: credentials.jwt,
|
|
audioEnabled: true,
|
|
videoEnabled: type == CallType.video,
|
|
);
|
|
|
|
_manager!.onParticipantConnected((p) {
|
|
state = state.copyWith(participants: [...state.participants, p]);
|
|
});
|
|
_manager!.onParticipantDisconnected((p) {
|
|
state = state.copyWith(
|
|
participants: state.participants.where((x) => x.sid != p.sid).toList(),
|
|
);
|
|
});
|
|
|
|
WakelockPlus.enable();
|
|
state = state.copyWith(state: CallState.connected);
|
|
} catch (e) {
|
|
await _cleanup();
|
|
state = state.copyWith(state: CallState.failed);
|
|
}
|
|
}
|
|
|
|
Future<void> hangUp() async {
|
|
final client = await _ref.read(matrixClientProvider.future);
|
|
await _membership?.leave(client);
|
|
await _cleanup();
|
|
state = const CallStateData(state: CallState.ended);
|
|
await Future.delayed(const Duration(seconds: 1));
|
|
state = const CallStateData();
|
|
}
|
|
|
|
Future<void> toggleMute() async {
|
|
final muted = !state.isMuted;
|
|
await _manager?.room.localParticipant?.setMicrophoneEnabled(!muted);
|
|
state = state.copyWith(isMuted: muted);
|
|
}
|
|
|
|
Future<void> toggleCamera() async {
|
|
final off = !state.isCameraOff;
|
|
await _manager?.room.localParticipant?.setCameraEnabled(!off);
|
|
state = state.copyWith(isCameraOff: off);
|
|
}
|
|
|
|
Future<void> toggleSpeaker() async {
|
|
final on = !state.isSpeakerOn;
|
|
await Helper.setSpeakerphoneOn(on);
|
|
state = state.copyWith(isSpeakerOn: on);
|
|
}
|
|
|
|
Future<void> toggleScreenShare() async {
|
|
final sharing = !state.isScreenSharing;
|
|
if (_manager != null) {
|
|
await _manager!.room.localParticipant?.setScreenShareEnabled(sharing);
|
|
}
|
|
state = state.copyWith(isScreenSharing: sharing);
|
|
}
|
|
|
|
Future<void> _cleanup() async {
|
|
await _manager?.disconnect();
|
|
_manager = null;
|
|
WakelockPlus.disable();
|
|
}
|
|
}
|
|
|
|
final callProvider = StateNotifierProvider<CallNotifier, CallStateData>(
|
|
(ref) => CallNotifier(ref),
|
|
);
|
|
```
|
|
|
|
---
|
|
|
|
## Call-UI Komponenten
|
|
|
|
```dart
|
|
// lib/features/call/call_page.dart
|
|
|
|
class CallPage extends ConsumerWidget {
|
|
const CallPage({super.key});
|
|
|
|
@override
|
|
Widget build(BuildContext context, WidgetRef ref) {
|
|
final call = ref.watch(callProvider);
|
|
final notifier = ref.read(callProvider.notifier);
|
|
|
|
return Scaffold(
|
|
backgroundColor: Colors.black,
|
|
body: SafeArea(
|
|
child: Stack(
|
|
children: [
|
|
// ── Haupt-Video (Remote Participant) ──────────────────────
|
|
if (call.participants.isNotEmpty)
|
|
Positioned.fill(
|
|
child: RemoteVideoView(participant: call.participants.first),
|
|
)
|
|
else
|
|
const Center(
|
|
child: Icon(Icons.person, size: 120, color: Colors.white24),
|
|
),
|
|
|
|
// ── Lokale Kamera (klein, Overlay) ────────────────────────
|
|
if (call.type == CallType.video)
|
|
Positioned(
|
|
right: 16, top: 16,
|
|
width: 120, height: 180,
|
|
child: ClipRRect(
|
|
borderRadius: BorderRadius.circular(12),
|
|
child: _LocalPreview(),
|
|
),
|
|
),
|
|
|
|
// ── Status-Text ───────────────────────────────────────────
|
|
Positioned(
|
|
top: 24, left: 0, right: 0,
|
|
child: Text(
|
|
_statusText(call.state),
|
|
textAlign: TextAlign.center,
|
|
style: const TextStyle(color: Colors.white, fontSize: 18),
|
|
),
|
|
),
|
|
|
|
// ── Call-Controls ─────────────────────────────────────────
|
|
Positioned(
|
|
bottom: 40, left: 0, right: 0,
|
|
child: Row(
|
|
mainAxisAlignment: MainAxisAlignment.spaceEvenly,
|
|
children: [
|
|
_CallButton(
|
|
icon: call.isMuted ? Icons.mic_off : Icons.mic,
|
|
label: call.isMuted ? 'Ton an' : 'Stumm',
|
|
color: call.isMuted ? Colors.red : Colors.white24,
|
|
onTap: notifier.toggleMute,
|
|
),
|
|
_CallButton(
|
|
icon: Icons.call_end,
|
|
label: 'Auflegen',
|
|
color: Colors.red,
|
|
size: 72,
|
|
onTap: notifier.hangUp,
|
|
),
|
|
if (call.type == CallType.video)
|
|
_CallButton(
|
|
icon: call.isCameraOff
|
|
? Icons.videocam_off
|
|
: Icons.videocam,
|
|
label: 'Kamera',
|
|
color: call.isCameraOff ? Colors.red : Colors.white24,
|
|
onTap: notifier.toggleCamera,
|
|
)
|
|
else
|
|
_CallButton(
|
|
icon: call.isSpeakerOn
|
|
? Icons.volume_up
|
|
: Icons.volume_down,
|
|
label: 'Lautsprecher',
|
|
color: Colors.white24,
|
|
onTap: notifier.toggleSpeaker,
|
|
),
|
|
],
|
|
),
|
|
),
|
|
],
|
|
),
|
|
),
|
|
);
|
|
}
|
|
|
|
String _statusText(CallState state) => switch (state) {
|
|
CallState.calling => 'Verbinde…',
|
|
CallState.connecting => 'Verbinde…',
|
|
CallState.connected => 'Verbunden',
|
|
CallState.ended => 'Aufgelegt',
|
|
CallState.failed => 'Fehler',
|
|
_ => '',
|
|
};
|
|
}
|
|
|
|
class _CallButton extends StatelessWidget {
|
|
final IconData icon;
|
|
final String label;
|
|
final Color color;
|
|
final double size;
|
|
final VoidCallback onTap;
|
|
|
|
const _CallButton({
|
|
required this.icon, required this.label,
|
|
required this.color, required this.onTap,
|
|
this.size = 56,
|
|
});
|
|
|
|
@override
|
|
Widget build(BuildContext context) => Column(
|
|
mainAxisSize: MainAxisSize.min,
|
|
children: [
|
|
GestureDetector(
|
|
onTap: onTap,
|
|
child: CircleAvatar(
|
|
radius: size / 2,
|
|
backgroundColor: color,
|
|
child: Icon(icon, color: Colors.white, size: size * 0.5),
|
|
),
|
|
),
|
|
const SizedBox(height: 6),
|
|
Text(label, style: const TextStyle(color: Colors.white70, fontSize: 12)),
|
|
],
|
|
);
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Hintergrund-Service (Android)
|
|
|
|
```kotlin
|
|
// android/app/src/main/kotlin/chat/pyramid/pyramid/CallForegroundService.kt
|
|
|
|
package chat.pyramid.pyramid
|
|
|
|
import android.app.Notification
|
|
import android.app.NotificationChannel
|
|
import android.app.NotificationManager
|
|
import android.app.Service
|
|
import android.content.Intent
|
|
import android.os.Build
|
|
import android.os.IBinder
|
|
import androidx.core.app.NotificationCompat
|
|
|
|
class CallForegroundService : Service() {
|
|
|
|
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
|
|
when (intent?.action) {
|
|
ACTION_START -> startCallNotification()
|
|
ACTION_STOP -> stopSelf()
|
|
}
|
|
return START_STICKY
|
|
}
|
|
|
|
private fun startCallNotification() {
|
|
ensureChannel()
|
|
val notification = NotificationCompat.Builder(this, CHANNEL_ID)
|
|
.setSmallIcon(R.drawable.ic_notification)
|
|
.setContentTitle("Pyramid — Aktiver Anruf")
|
|
.setContentText("Tippe um zum Anruf zurückzukehren")
|
|
.setOngoing(true)
|
|
.setPriority(NotificationCompat.PRIORITY_HIGH)
|
|
.build()
|
|
|
|
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
|
|
startForeground(NOTIF_ID, notification,
|
|
android.content.pm.ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE)
|
|
} else {
|
|
startForeground(NOTIF_ID, notification)
|
|
}
|
|
}
|
|
|
|
private fun ensureChannel() {
|
|
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
|
|
val channel = NotificationChannel(
|
|
CHANNEL_ID, "Anrufe", NotificationManager.IMPORTANCE_HIGH)
|
|
(getSystemService(NOTIFICATION_SERVICE) as NotificationManager)
|
|
.createNotificationChannel(channel)
|
|
}
|
|
}
|
|
|
|
override fun onBind(intent: Intent?): IBinder? = null
|
|
|
|
companion object {
|
|
const val ACTION_START = "START_CALL"
|
|
const val ACTION_STOP = "STOP_CALL"
|
|
const val CHANNEL_ID = "pyramid_calls"
|
|
const val NOTIF_ID = 1001
|
|
}
|
|
}
|
|
```
|
|
|
|
```dart
|
|
// Dart: Service starten/stoppen
|
|
import 'package:flutter/services.dart';
|
|
|
|
const _callChannel = MethodChannel('chat.pyramid.pyramid/call');
|
|
|
|
Future<void> startCallService() async {
|
|
if (!Platform.isAndroid) return;
|
|
await _callChannel.invokeMethod('startCallService');
|
|
}
|
|
|
|
Future<void> stopCallService() async {
|
|
if (!Platform.isAndroid) return;
|
|
await _callChannel.invokeMethod('stopCallService');
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting & Diagnose
|
|
|
|
### Server-Tests
|
|
|
|
```bash
|
|
# LiveKit-Server erreichbar?
|
|
curl https://livekit.steggi-matrix.work/
|
|
|
|
# lk-jwt-service erreichbar?
|
|
curl -X POST https://livekit.steggi-matrix.work/lk-jwt/sfu/get \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"room":"!test:server","openid_token":{"access_token":"test","token_type":"Bearer","matrix_server_name":"steggi-matrix.work"},"device_id":"TEST"}'
|
|
# Erwartete Antwort: {"url":"wss://...","jwt":"eyJ..."}
|
|
|
|
# TURN-Server Test (stuncheck)
|
|
pip install stun-client
|
|
python -c "import stun; print(stun.get_ip_info())"
|
|
|
|
# LiveKit Logs
|
|
sudo journalctl -u livekit -f
|
|
# lk-jwt-service Logs
|
|
sudo journalctl -u lk-jwt-service -f
|
|
|
|
# UDP-Port-Reichbarkeit (von außen)
|
|
nmap -sU -p 50100-50110 <PI_IP>
|
|
```
|
|
|
|
### Häufige Probleme
|
|
|
|
| Problem | Ursache | Lösung |
|
|
|---------|---------|--------|
|
|
| `ICE failed` | UDP-Ports gesperrt | TURN aktivieren, Firewall prüfen |
|
|
| Token-Fehler 401 | OpenID-Validierung schlägt fehl | Matrix-Server-URL in lk-jwt-service prüfen |
|
|
| `WebSocket closed` | Nginx-Proxy-Timeout | `proxy_read_timeout 3600s` in Nginx |
|
|
| Kein Audio | Mikrofon-Berechtigung | `Permission.microphone.request()` |
|
|
| `Room not found` | LiveKit-Key falsch | API-Key in livekit.yaml = lk-jwt-service Config |
|
|
| Kein Video | Kamera gesperrt | `Permission.camera.request()` |
|
|
| Call fällt nach 4h ab | `expires_ts` zu niedrig | Bei 30min vor Ablauf Membership erneuern |
|
|
|
|
### Dart-Diagnose-Tool
|
|
|
|
```dart
|
|
Future<Map<String, bool>> diagnoseLiveKit(Client client, String roomId) async {
|
|
final results = <String, bool>{};
|
|
|
|
// 1. OpenID-Token
|
|
try {
|
|
await client.requestOpenIdToken(client.userID!);
|
|
results['openid_token'] = true;
|
|
} catch (_) { results['openid_token'] = false; }
|
|
|
|
// 2. JWT-Service erreichbar
|
|
try {
|
|
final r = await http.get(
|
|
Uri.parse('https://livekit.steggi-matrix.work/'));
|
|
results['livekit_reachable'] = r.statusCode < 500;
|
|
} catch (_) { results['livekit_reachable'] = false; }
|
|
|
|
// 3. Token-Austausch
|
|
try {
|
|
await LiveKitService.getToken(
|
|
matrixClient: client,
|
|
roomId: roomId,
|
|
livekitServiceUrl: 'https://livekit.steggi-matrix.work',
|
|
);
|
|
results['token_exchange'] = true;
|
|
} catch (_) { results['token_exchange'] = false; }
|
|
|
|
return results;
|
|
}
|
|
```
|