QUICKSTART.md

#!/usr/bin/env bash
chmod +x /Users/pgedge/pge/pgControlPlane/scripts/deploy-full-cluster.sh

# pgControlPlane Quick Start Guide

Get a production-ready PostgreSQL cluster with all pgElephant components running in minutes!

## 🚀 One-Command Deployment

```bash
# Deploy a complete 3-node cluster with all components
cd pgControlPlane
./scripts/deploy-full-cluster.sh --name mycluster --nodes 3
```

That's it! You'll get:
- ✅ 3-node PostgreSQL 16.1 cluster
- ✅ Automated failover & healing
- ✅ Connection pooling (pgbalancer)
- ✅ Automated backups (pgBackRest)
- ✅ Real-time monitoring (pgSentinel)
- ✅ Testing framework (FauxDB)

## 📋 Prerequisites

- Docker & Docker Compose (for local deployment)
- OR Kubernetes cluster (for production)
- OR bare VMs with SSH access

## 🎯 Deployment Options

### Option 1: Local Docker (Development)

```bash
# Clone the repository
git clone https://github.com/pgElephant/pgControlPlane
cd pgControlPlane

# Start the complete stack
docker-compose -f deployments/complete-stack.yaml up -d

# Wait for services to be ready
docker-compose ps

# Access the cluster
psql postgresql://postgres:postgres@localhost:5435/production
```

**Services Available:**
- pgControlPlane API: http://localhost:8080
- pgbalancer: localhost:5435
- pgSentinel Dashboard: http://localhost:3000
- Grafana: http://localhost:3001 (admin/admin)
- Prometheus: http://localhost:9091
- FauxDB (MongoDB compat): mongodb://localhost:27017

### Option 2: Kubernetes (Production)

```bash
# Install via Helm
helm repo add pgelephant https://pgelephant.github.io/charts
helm repo update

# Create values file
cat > values.yaml <<EOF
cluster:
  name: production
  nodeCount: 3
  postgresVersion: "16.1"
  
components:
  pgbalancer: true
  pgbackrest: true
  pgraft: false
  pgsentinel: true
  fauxdb: true

storage:
  size: 100Gi
  storageClass: fast-ssd

resources:
  postgres:
    requests:
      cpu: 2
      memory: 4Gi
    limits:
      cpu: 4
      memory: 8Gi
EOF

# Install
helm install mycluster pgelephant/pgcontrolplane -f values.yaml

# Check status
kubectl get pgclusters
kubectl get pods -l app=mycluster
```

### Option 3: Automated Script (Any Infrastructure)

```bash
# Full customization
export CLUSTER_NAME=production
export NODE_COUNT=5
export POSTGRES_VERSION=16.1
export REPLICATION_MODE=sync
export ENABLE_PGRAFT=true  # Use Raft consensus
export REGION=us-west-2

./scripts/deploy-full-cluster.sh
```

## 🔧 Configuration Examples

### 1. High Availability Cluster (Synchronous Replication)

```bash
./scripts/deploy-full-cluster.sh \
  --name ha-cluster \
  --nodes 3 \
  --replication sync
```

### 2. Raft Consensus Cluster (Strong Consistency)

```bash
./scripts/deploy-full-cluster.sh \
  --name raft-cluster \
  --nodes 5 \
  --with-raft
```

### 3. Development Cluster (Minimal Resources)

```bash
export INSTANCE_TYPE=t3.small
export STORAGE_GB=20
export ENABLE_PGSENTINEL=false

./scripts/deploy-full-cluster.sh \
  --name dev-cluster \
  --nodes 1
```

### 4. Multi-Region Cluster

```bash
# Primary region
./scripts/deploy-full-cluster.sh \
  --name primary \
  --nodes 3 \
  --region us-east-1

# Standby region
./scripts/deploy-full-cluster.sh \
  --name standby \
  --nodes 3 \
  --region us-west-2
```

## 📊 Component Overview

### pgControlPlane
**The Orchestrator** - Manages the entire cluster lifecycle

```bash
# Check cluster status
curl http://localhost:8080/api/v1/clusters/production/status

# List all nodes
curl http://localhost:8080/api/v1/clusters/production/nodes

# Trigger manual promotion
curl -X POST http://localhost:8080/api/v1/clusters/production/nodes/node-2/promote
```

### pgbalancer
**Connection Pooler & Load Balancer**

```bash
# Connect through pgbalancer (recommended)
psql postgresql://postgres:postgres@localhost:5435/production

# Check backend status via REST API
curl http://localhost:8081/api/status

# Reload configuration
curl -X POST http://localhost:8081/api/reload
```

### pgBackRest
**Enterprise Backup Solution**

```bash
# Manual backup
curl -X POST http://localhost:8080/api/v1/clusters/production/backups

# List backups
curl http://localhost:8080/api/v1/clusters/production/backups

# Point-in-time recovery
curl -X POST http://localhost:8080/api/v1/clusters/production/restore \
  -d '{"point_in_time": "2024-01-15T10:30:00Z"}'
```

### pgSentinel
**Real-time Monitoring Dashboard**

Open http://localhost:3000 to access:
- Live cluster topology
- Query performance analytics
- Replication lag monitoring
- Automated alerts
- Historical trends

### FauxDB
**MongoDB Compatibility Layer**

```bash
# Connect with MongoDB client
mongo mongodb://localhost:27017/production

# Use MongoDB commands with PostgreSQL backend
db.users.insertOne({name: "John", email: "john@example.com"})
db.users.find({name: "John"})

# Check FauxDB statistics
curl http://localhost:8082/api/stats
```

### pgraft (Optional)
**Raft Consensus for Strong Consistency**

When enabled, provides:
- CP guarantee (Consistency + Partition tolerance)
- Leader election
- Log replication
- etcd-compatible API

## 🎭 Common Operations

### Scale the Cluster

```bash
# Add more nodes
./scripts/scale-cluster.sh --cluster-id production --nodes 5

# Remove nodes
./scripts/scale-cluster.sh --cluster-id production --nodes 2
```

### Upgrade PostgreSQL Version

```bash
# Rolling upgrade
./scripts/upgrade-cluster.sh \
  --cluster-id production \
  --version 17.0 \
  --strategy rolling
```

### Failover Testing

```bash
# Simulate primary failure
docker stop postgres-node1

# Watch automatic failover
curl http://localhost:8080/api/v1/clusters/production/status

# Check logs
docker logs controlplane -f
```

### Backup & Restore

```bash
# Create full backup
curl -X POST http://localhost:8080/api/v1/clusters/production/backups \
  -d '{"type": "full"}'

# Restore from backup
curl -X POST http://localhost:8080/api/v1/clusters/production/restore \
  -d '{"backup_id": "backup-123"}'
```

## 📈 Monitoring & Observability

### Prometheus Metrics

```bash
# Cluster health
curl http://localhost:2112/metrics | grep pgcp_cluster_health_score

# Replication lag
curl http://localhost:2112/metrics | grep pgcp_cluster_replication_lag

# Failover count
curl http://localhost:2112/metrics | grep pgcp_failovers_total
```

### Grafana Dashboards

1. Open http://localhost:3001
2. Login: admin/admin
3. Navigate to Dashboards > pgElephant

Available dashboards:
- Cluster Overview
- Node Health
- Replication Status
- Backup History
- Query Performance

### Real-time Events (WebSocket)

```javascript
const ws = new WebSocket('ws://localhost:8081/api/v1/ws');

ws.send(JSON.stringify({
  action: 'subscribe',
  cluster_id: 'production'
}));

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log('Event:', data);
};
```

## 🧪 Testing

### Run All Tests

```bash
# Unit tests
make test

# Integration tests
make test-integration

# E2E tests
make test-e2e

# Load tests
./scripts/load-test.sh --users 1000 --duration 300
```

### FauxDB Test Scenarios

```bash
# Load testing
curl -X POST http://localhost:5000/api/tests/load \
  -d '{"connections": 100, "duration": 60}'

# Failover simulation
curl -X POST http://localhost:5000/api/tests/failover

# Replication lag test
curl -X POST http://localhost:5000/api/tests/replication-lag

# Backup/restore test
curl -X POST http://localhost:5000/api/tests/backup-restore
```

## 🔒 Security

### Enable TLS

```yaml
# config.yaml
security:
  tls:
    enabled: true
    cert_file: /etc/pgcp/tls/cert.pem
    key_file: /etc/pgcp/tls/key.pem
    ca_file: /etc/pgcp/tls/ca.pem
```

### Enable Authentication

```yaml
# config.yaml
security:
  jwt:
    secret: your-secret-key
    expiry: 24h
  rbac:
    enabled: true
```

### mTLS for Agents

```bash
# Generate certificates
./scripts/generate-certs.sh --cluster production

# Configure agents
export PGCP_AGENT_TLS_CERT=/etc/pgcp/agent-cert.pem
export PGCP_AGENT_TLS_KEY=/etc/pgcp/agent-key.pem
```

## 🐛 Troubleshooting

### Cluster Won't Start

```bash
# Check control plane logs
docker logs controlplane

# Check database connectivity
docker exec -it pgcp-database psql -U pgcp -c "SELECT 1"

# Verify network
docker network inspect pgelephant_default
```

### High Replication Lag

```bash
# Check lag on all nodes
curl http://localhost:8080/api/v1/clusters/production/status | jq '.max_replication_lag_ms'

# Check network latency
docker exec postgres-node2 ping postgres-node1

# Adjust replication settings
# Edit postgresql.conf: wal_sender_timeout, max_wal_senders
```

### Failover Not Working

```bash
# Check reconciler logs
docker logs controlplane | grep reconciler

# Verify auto-failover is enabled
curl http://localhost:8080/api/v1/clusters/production | jq '.auto_failover'

# Check agent connectivity
curl http://localhost:8080/api/v1/agents
```

### Performance Issues

```bash
# Check resource usage
docker stats

# Analyze slow queries (pgSentinel)
open http://localhost:3000/queries

# Check connection pool
curl http://localhost:8081/api/pool-status
```

## 📚 Additional Resources

- **Full Documentation**: https://docs.pgelephant.com/pgcontrolplane
- **API Reference**: http://localhost:8080/api/docs
- **GitHub**: https://github.com/pgElephant/pgControlPlane
- **Community**: https://discord.gg/pgelephant
- **Blog**: https://www.pgelephant.com/blog

## 💡 Best Practices

1. **Always use pgbalancer** for client connections
2. **Enable automated backups** with appropriate retention
3. **Monitor replication lag** - alert if > 10s
4. **Test failover regularly** in staging
5. **Keep PostgreSQL updated** - use rolling upgrades
6. **Use pgSentinel** for proactive monitoring
7. **Run FauxDB tests** before production deployment

## 🎓 Next Steps

1. **Tutorial**: [Building Your First HA Cluster](docs/tutorials/first-cluster.md)
2. **Guide**: [Production Checklist](docs/guides/production-checklist.md)
3. **Deep Dive**: [Architecture Overview](ARCHITECTURE.md)
4. **Advanced**: [Multi-Region Setup](docs/guides/multi-region.md)

## 🆘 Getting Help

- **Issues**: https://github.com/pgElephant/pgControlPlane/issues
- **Discussions**: https://github.com/pgElephant/pgControlPlane/discussions
- **Discord**: https://discord.gg/pgelephant
- **Email**: support@pgelephant.com

---

**Happy Clustering! 🐘**