Docker Setup - LiteAgent Documentation

Overview

Docker is the recommended way to run LiteAgent as it provides:

Isolated environments for each agent
Consistent dependencies across platforms
Easy parallel execution
Simple deployment and scaling

Docker Architecture

Quick Start

Clone Repository

git clone https://github.com/devinat1/agent-collector.git
cd agent-collector
git submodule update --init --recursive

Configure Environment

cp collector/.env.example collector/.env
# Edit collector/.env with your API keys

Build and Run

# Build all images
docker compose build

# Run specific agent
docker compose up browseruse

# Run in background
docker compose up -d browseruse

Docker Compose Configuration

Main Configuration File

The docker-compose.yml file defines all agent services:

services:
  browseruse:
    build:
      context: .
      dockerfile: Dockerfile.browseruse
    image: browseruse-runner
    volumes:
      - ./data/db:/app/data/db
      - ./data/prompts:/app/data/prompts
      - ./collector/logs:/app/collector/logs
    working_dir: /app
    command: ["bash", "-c", "./run.sh browseruse --category test --virtual --timeout 180"]
    env_file:
      - collector/.env
    environment:
      - PYTHONUNBUFFERED=1

Agent Services

BrowserUse
Agent E
Skyvern

browseruse:
  build:
    context: .
    dockerfile: Dockerfile.browseruse
  image: browseruse-runner
  volumes:
    - ./data/db:/app/data/db
    - ./data/prompts:/app/data/prompts
    - ./collector/logs:/app/collector/logs
  working_dir: /app
  command: ["bash", "-c", "./run.sh browseruse --category test --timeout 180"]
  env_file:
    - collector/.env
  environment:
    - PYTHONUNBUFFERED=1
    - OPENAI_API_KEY=${OPENAI_API_KEY}

Dockerfile Examples

Base Dockerfile Structure

FROM python:3.11-slim

# Install system dependencies
RUN apt-get update && apt-get install -y \
    wget \
    gnupg \
    unzip \
    curl \
    git \
    && rm -rf /var/lib/apt/lists/*

# Install Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list \
    && apt-get update \
    && apt-get install -y google-chrome-stable \
    && rm -rf /var/lib/apt/lists/*

# Set working directory
WORKDIR /app

# Copy requirements and install
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Install Playwright
RUN pip install playwright
RUN playwright install chromium
RUN playwright install-deps

# Copy application code
COPY . .

# Set execute permissions
RUN chmod +x run.sh

# Create data directories
RUN mkdir -p data/db data/prompts collector/logs

ENTRYPOINT ["./run.sh"]

Agent-Specific Dockerfiles

Dockerfile.browseruse

FROM python:3.11-slim

# Install Chrome and dependencies
RUN apt-get update && apt-get install -y \
    wget gnupg unzip curl git \
    chromium-browser chromium-chromedriver \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Copy and install requirements
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Install BrowserUse specific dependencies
RUN pip install browser-use playwright
RUN playwright install chromium
RUN playwright install-deps

# Copy application
COPY . .
RUN chmod +x run.sh

# Create directories
RUN mkdir -p data/db data/prompts collector/logs

ENTRYPOINT ["./run.sh"]

Dockerfile.skyvern

FROM python:3.11-slim

# Install PostgreSQL client and other dependencies
RUN apt-get update && apt-get install -y \
    postgresql-client \
    wget gnupg unzip curl git \
    chromium-browser \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Install Skyvern dependencies
RUN pip install skyvern playwright
RUN playwright install chromium
RUN playwright install-deps

# Copy application
COPY . .
RUN chmod +x run.sh
RUN chmod +x collector/agents/skyvern/entrypoint.sh

# Create directories
RUN mkdir -p data/db data/prompts collector/logs

ENTRYPOINT ["./collector/agents/skyvern/entrypoint.sh"]

Volume Management

Persistent Data

LiteAgent uses volumes to persist data across container restarts:

volumes:
  # Test data output
  - ./data/db:/app/data/db

  # Test prompts input
  - ./data/prompts:/app/data/prompts

  # Application logs
  - ./collector/logs:/app/collector/logs

  # Browser profiles (for DoBrowser)
  - ./data/browser_data:/app/data/browser_data

Browser Data Persistence

For agents that require browser profiles:

dobrowser:
  volumes:
    - ./data/browser_data/dobrowser:/app/data/browser_data/dobrowser
    - ./data/browser_data/browseruse:/app/data/browser_data/browseruse

Parallel Execution

Using Docker Compose Replicas

services:
  browseruse:
    # ... configuration ...
    deploy:
      replicas: 5  # Run 5 parallel instances
    environment:
      - WORKER_ID=${WORKER_ID:-1}

Run with scaling:

# Scale to 5 instances
docker compose up --scale browseruse=5

# Or using deploy.replicas
docker compose up -d browseruse

Multiple Agent Types

# Run multiple different agents
docker compose up -d browseruse agente multion

# Scale specific agents
docker compose up --scale browseruse=3 --scale agente=2

Environment Configuration

Environment Files

Create separate environment files for different configurations:

# Development
collector/.env.dev

# Production
collector/.env.prod

# Testing
collector/.env.test

Use with Docker Compose:

services:
  browseruse:
    env_file:
      - collector/.env
      - collector/.env.${ENV:-dev}

Runtime Environment Variables

Override environment variables at runtime:

# Override category
CATEGORY=dark_patterns docker compose up browseruse

# Override timeout
TIMEOUT=600 docker compose up agente

# Multiple overrides
CATEGORY=benchmark TIMEOUT=300 REPLICAS=3 docker compose up browseruse

Networking

Internal Communication

Services can communicate using service names:

services:
  skyvern:
    depends_on:
      - postgres
    environment:
      - DATABASE_URL=postgresql://user:pass@postgres:5432/skyvern

  postgres:
    image: postgres:13
    environment:
      - POSTGRES_DB=skyvern
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=pass

Port Exposure

Expose services for external access:

services:
  webhook-server:
    ports:
      - "8080:8080"  # External:Internal
    environment:
      - WEBHOOK_PORT=8080

Monitoring and Logging

Log Configuration

Configure logging for better debugging:

services:
  browseruse:
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

Health Checks

Add health checks to monitor container status:

services:
  browseruse:
    healthcheck:
      test: ["CMD", "python", "-c", "import requests; requests.get('http://localhost:8000/health')"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Monitoring Commands

# View logs
docker compose logs -f browseruse

# Monitor resource usage
docker stats

# Check container status
docker compose ps

# View specific service logs
docker compose logs --tail=100 browseruse

Optimization

Build Optimization

Use multi-stage builds to reduce image size:

# Build stage
FROM python:3.11-slim as builder
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Runtime stage
FROM python:3.11-slim
COPY --from=builder /root/.local /root/.local
ENV PATH=/root/.local/bin:$PATH

# Copy application
COPY . /app
WORKDIR /app

Resource Limits

Set resource limits to prevent container overconsumption:

services:
  browseruse:
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 4G
        reservations:
          cpus: '1.0'
          memory: 2G

Caching

Use Docker layer caching for faster builds:

# Enable BuildKit
export DOCKER_BUILDKIT=1

# Build with cache
docker compose build --parallel

# Use build cache from registry
docker compose build --cache-from browseruse-runner:latest

Production Deployment

Production Compose File

Create docker-compose.prod.yml:

version: '3.8'

services:
  browseruse:
    image: liteagent/browseruse:latest
    restart: unless-stopped
    deploy:
      replicas: 10
      resources:
        limits:
          memory: 4G
    environment:
      - ENV=production
    volumes:
      - /data/liteagent/db:/app/data/db
      - /data/liteagent/logs:/app/collector/logs

Deploy with:

docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Troubleshooting

Build failures

# Clear build cache
docker builder prune -a

# Rebuild without cache
docker compose build --no-cache

# Check build logs
docker compose build 2>&1 | tee build.log

Permission issues

# Fix file permissions
sudo chown -R $USER:$USER data/

# Add user to docker group
sudo usermod -aG docker $USER
newgrp docker

Out of space

# Clean up unused resources
docker system prune -a

# Remove old volumes
docker volume prune

# Check disk usage
docker system df

Next Steps

Local Setup

Alternative local development setup

Environment Variables

Configure API keys and settings

Running Tests

Start running tests with Docker

Getting Started

Core Concepts

Setup & Configuration

Running Tests

Output & Analysis

​Overview

​Docker Architecture

​Quick Start

​Docker Compose Configuration

​Main Configuration File

​Agent Services

​Dockerfile Examples

​Base Dockerfile Structure

​Agent-Specific Dockerfiles

​Volume Management

​Persistent Data

​Browser Data Persistence

​Parallel Execution

​Using Docker Compose Replicas

​Multiple Agent Types

​Environment Configuration

​Environment Files

​Runtime Environment Variables

​Networking

​Internal Communication

​Port Exposure

​Monitoring and Logging

​Log Configuration

​Health Checks

​Monitoring Commands

​Optimization

​Build Optimization

​Resource Limits

​Caching

​Production Deployment

​Production Compose File

​Troubleshooting

​Next Steps

Local Setup

Environment Variables

Running Tests

Overview

Docker Architecture

Quick Start

Docker Compose Configuration

Main Configuration File

Agent Services

Dockerfile Examples

Base Dockerfile Structure

Agent-Specific Dockerfiles

Volume Management

Persistent Data

Browser Data Persistence

Parallel Execution

Using Docker Compose Replicas

Multiple Agent Types

Environment Configuration

Environment Files

Runtime Environment Variables

Networking

Internal Communication

Port Exposure

Monitoring and Logging

Log Configuration

Health Checks

Monitoring Commands

Optimization

Build Optimization

Resource Limits

Caching

Production Deployment

Production Compose File

Troubleshooting

Next Steps