Vector Logging Setup

This guide covers setting up Vector logging with cloud storage sinks that require IAM configuration (AWS S3, Google Cloud Storage, Azure Blob Storage).

Overview

Vector collects logs from all Rulebricks components and forwards them to your chosen destination. For cloud storage sinks (S3, GCS, Azure Blob), you need to configure IAM permissions so Vector can write logs.

The CLI provides automatic IAM setup commands that handle the complexity of cloud provider authentication.

Prerequisites

Before setting up cloud storage logging:

Deploy your cluster - IAM setup requires an existing deployment
Configure logging in config - Set up the sink in rulebricks.yaml
Install cloud CLI tools - Required for IAM setup:
- AWS: aws CLI and eksctl
- GCP: gcloud CLI
- Azure: az CLI

AWS S3 Setup

Configuration

Configure S3 logging in your rulebricks.yaml:

logging:
  enabled: true
  vector:
    sink:
      type: aws_s3
      config:
        bucket: "my-logs-bucket"
        region: "us-east-1"
        setup_iam: true  # Enable automatic IAM setup prompt

Automatic Setup

After deployment, run the automatic setup:

rulebricks vector setup-s3

This command:

Creates OIDC provider for your EKS cluster (if needed)
Creates IAM policy with S3 permissions
Creates IRSA (IAM Roles for Service Accounts) service account
Updates Vector deployment to use the service account
Verifies S3 access

Manual Setup

If you prefer manual setup or need custom configuration:

rulebricks vector generate-iam-config --sink aws_s3 --bucket my-logs-bucket

This generates:

IAM policy document
Step-by-step setup instructions
CLI commands to execute

Command Options

rulebricks vector setup-s3 [flags]
 
Flags:
  --bucket string    S3 bucket name (uses config value if not specified)
  --region string    AWS region (uses config value if not specified)
  --cluster string   EKS cluster name (uses config value if not specified)

What Gets Created

OIDC Provider: For EKS cluster (if not exists)
IAM Policy: Grants S3 write permissions
IAM Role: Assumed by Vector pods
Service Account: Annotated with IAM role ARN
Vector Deployment: Updated to use service account

Verification

After setup, verify logs are being written:

# Check Vector logs
rulebricks logs vector
 
# Check S3 bucket
aws s3 ls s3://my-logs-bucket/

Google Cloud Storage Setup

Configuration

Configure GCS logging in your rulebricks.yaml:

logging:
  enabled: true
  vector:
    sink:
      type: gcp_cloud_storage
      config:
        bucket: "my-gcs-bucket"
        use_workload_identity: true
        setup_iam: true

Automatic Setup

After deployment, run the automatic setup:

rulebricks vector setup-gcs

This command:

Enables Workload Identity on your GKE cluster (if needed)
Creates GCP service account
Grants storage permissions to service account
Binds Workload Identity
Updates Vector deployment
Verifies GCS access

Manual Setup

Generate manual setup instructions:

rulebricks vector generate-iam-config --sink gcp_cloud_storage --bucket my-gcs-bucket

Command Options

rulebricks vector setup-gcs [flags]
 
Flags:
  --bucket string    GCS bucket name (uses config value if not specified)
  --project string   GCP project ID (uses config value if not specified)
  --cluster string   GKE cluster name (uses config value if not specified)

What Gets Created

Workload Identity: Enabled on GKE cluster
GCP Service Account: For Vector logging
IAM Binding: Links Kubernetes service account to GCP service account
Storage Permissions: Grants object write permissions
Vector Deployment: Updated to use Workload Identity

Verification

After setup, verify logs are being written:

# Check Vector logs
rulebricks logs vector
 
# Check GCS bucket
gsutil ls gs://my-gcs-bucket/

Azure Blob Storage Setup

Configuration

Configure Azure Blob logging in your rulebricks.yaml:

logging:
  enabled: true
  vector:
    sink:
      type: azure_blob
      config:
        container_name: "logs"
        storage_account: "mylogs"
        use_managed_identity: true
        setup_iam: true

Automatic Setup

After deployment, run the automatic setup:

rulebricks vector setup-azure

This command:

Creates managed identity
Assigns storage permissions to managed identity
Configures pod identity
Updates Vector deployment
Verifies Azure access

Manual Setup

Generate manual setup instructions:

rulebricks vector generate-iam-config --sink azure_blob --bucket my-container

Command Options

rulebricks vector setup-azure [flags]
 
Flags:
  --storage-account string   Azure storage account name
  --container string         Blob container name (uses config value if not specified)
  --resource-group string    Azure resource group (uses config value if not specified)
  --cluster string           AKS cluster name (uses config value if not specified)

What Gets Created

Managed Identity: For Vector logging
Role Assignment: Grants storage permissions
Pod Identity: Links pods to managed identity
Vector Deployment: Updated to use managed identity

Verification

After setup, verify logs are being written:

# Check Vector logs
rulebricks logs vector
 
# Check Azure container
az storage blob list --account-name mylogs --container-name logs

Generate IAM Configuration

Generate IAM configuration and setup instructions for any sink:

rulebricks vector generate-iam-config [flags]
 
Flags:
  --sink string    Sink type (aws_s3, gcp_cloud_storage, azure_blob)
  --bucket string  Bucket/container name

Output includes:

IAM policy documents
Step-by-step manual setup instructions
CLI commands to execute
Verification steps

Troubleshooting

Setup Command Fails

Common issues:

Missing cloud CLI tools:
- Install required CLI tools
- Verify they're in your PATH
- Authenticate with cloud provider
Insufficient permissions:
- Verify your cloud account has admin/IAM permissions
- Check service account permissions (GCP)
- Verify subscription access (Azure)
Cluster not found:
- Verify cluster name is correct
- Check kubectl context is set
- Ensure cluster is accessible

Logs Not Appearing in Storage

Check Vector status:

# Check Vector pods
kubectl get pods -n <logging-namespace>
 
# Check Vector logs
rulebricks logs vector
 
# Check for errors
kubectl logs <vector-pod> -n <logging-namespace>

Verify IAM configuration:

AWS: Check service account annotation:
```
kubectl get sa vector -n <logging-namespace> -o yaml
```
Should have eks.amazonaws.com/role-arn annotation.
GCP: Check Workload Identity binding:
```
kubectl get sa vector -n <logging-namespace> -o yaml
```
Should have iam.gke.io/gcp-service-account annotation.

Azure: Check pod identity:

kubectl get podidentity <name> -n <logging-namespace>

Test access manually:

AWS: Test S3 access from a pod
GCP: Test GCS access from a pod
Azure: Test blob access from a pod

Permission Errors

If you see permission errors:

Verify IAM policy - Check policy grants correct permissions
Check role binding - Verify service account is bound correctly
Review cloud provider logs - Check cloud provider audit logs
Re-run setup - Try running setup command again

Bucket/Container Not Found

Ensure:

Bucket/container exists
Bucket/container name is correct
You have access to the bucket/container
Region/location matches configuration

Best Practices

Use automatic setup - Simplifies IAM configuration
Separate buckets - Use different buckets for different environments
Set up lifecycle policies - Automatically archive or delete old logs
Monitor log volume - Track storage costs
Secure access - Use least-privilege IAM policies
Test after setup - Verify logs are being written

Security Considerations

IAM Policies: Use least-privilege principles
Bucket Policies: Restrict access to necessary services only
Encryption: Enable encryption at rest
Access Logging: Enable access logging for audit
Network Policies: Restrict network access if possible

Next Steps

Monitor log volume: Track storage usage and costs
Set up log analysis: Configure log analysis tools
Configure retention: Set up lifecycle policies
Review logs: Regularly review logs for issues

Observability Intalling Upgrades