Today's Agenda

• Part 1: Your Technology Stack - What You Now Know
• Part 2: Functions - The Universal Unit of Work
• Part 3: The Four Pillars of Backend Development
• Part 4: What Can You Build? (Spoiler: Almost Anything)
• Part 5: What's Missing? Preview of System Design
• Demo: End-to-End System Build
• Exploration: System Analysis & Modification

The Journey So Far

Six weeks of technologies. Today we see how they combine.

Your Complete Stack

This stack can build most backend systems.

How They Fit Together

• PostgreSQL - persistent data storage
• Redis - job queue broker
• rq Worker - background task processing

All running in Docker containers

Each Technology Has a Job

Everything is Functions

# Flask route - a function
@app.route('/users', methods=['POST'])
def create_user():
    ...

# Background task - a function
@job('emails', connection=redis)
def send_welcome_email(user_id):
    ...

# Validation - a function
def validate_email(value):
    ...

# Database query - functions on objects
user = User.query.filter_by(email=email).first()

• Routes, tasks, validators, queries - all functions
• Functions are the universal unit of work

Functions Encapsulate Work

def process_order(order_id, user_email):
    """
    A function encapsulates:
    1. WHAT to do (the code inside)
    2. WHAT it needs (the parameters)
    3. WHAT it returns (the output)
    """
    order = get_order(order_id)       # Get data
    validate_order(order)              # Check it
    charge_payment(order)              # Do work
    send_confirmation(user_email)      # Side effect
    return {"status": "completed"}     # Return result

• Functions package up work with a name
• Call them, pass them around, schedule them
• This is true in every programming context

Decorators Change Execution Context

# The SAME work, different EXECUTION CONTEXTS:

# Runs NOW when HTTP request arrives
@app.route('/email', methods=['POST'])
def send_email_api():
    send_email(data['to'], data['subject'], data['body'])
    return {"sent": True}

# Runs LATER when worker picks it up
@job('emails', connection=redis)
def send_email_task(to, subject, body):
    send_email(to, subject, body)
    return {"sent": True}

• @app.route → "Run when HTTP request arrives"
• @job → "Run when worker picks up from queue"
• The function is the same - the trigger changes

Sync vs Async: Same Function, Different Timing

Choose based on: Does the user need the result right now?

Four Principles, Every Week

Looking across all our lectures, four principles kept appearing:

Pillar 1: Validate Everything

"Never trust data crossing a boundary"

# Week 6: Marshmallow at the API boundary
data = UserSchema().load(request.get_json())  # Validates!

# Week 7: Database constraints as backup
class User(db.Model):
    email = db.Column(db.String(255), unique=True, nullable=False)

# Week 8: Validate before queuing
if not is_valid_email(email):
    return {"error": "Invalid email"}, 400
send_email.delay(email, message)  # Only queue valid data

• HTTP input, database writes, queue messages - all boundaries
• Validate at every boundary

Pillar 2: Separate Concerns

"Each function/component does one thing"

# Bad: One function does everything
def handle_order(data):
    validate(data); charge_card(data); update_inventory(data)
    send_email(data); log_analytics(data)

# Good: Each function has one job
def handle_order(data):
    order = validate_order(data)           # Just validate
    payment = process_payment(order)        # Just charge
    update_inventory(order)                 # Just inventory
    send_confirmation.delay(order.id)       # Defer email

• One function = one responsibility
• Easier to test, debug, reuse, and modify

Pillar 3: Defer When Possible

"Don't block on slow operations"

# Slow: User waits for everything
@app.post('/order')
def create_order():
    order = save_order(data)
    send_email(order.user.email)      # 2 seconds...
    generate_invoice(order)            # 3 seconds...
    notify_warehouse(order)            # 1 second...
    return order.to_dict()             # User waited 6+ seconds!

# Fast: User gets immediate response
@app.post('/order')
def create_order():
    order = save_order(data)
    process_order.delay(order.id)      # Queue all the slow stuff
    return order.to_dict(), 202        # User waited < 100ms!

Pillar 4: Transform Explicitly

"Be intentional about data shape at each boundary"

# Data transforms at EVERY step - be explicit!
json_input = request.get_json()         # Bytes → Python dict
validated = UserSchema().load(json_input)  # Dict → Validated
user = User(**validated)                # Validated → ORM model
db.session.add(user)                    # Model → SQL INSERT
db.session.commit()                     # SQL → Database row

# Going back out:
result = UserSchema().dump(user)        # Model → Dict (controlled!)
return jsonify(result)                  # Dict → JSON bytes

• .load() controls what comes IN
• .dump() controls what goes OUT
• Never accidentally expose internal data

The Four Pillars in Your Stack

Your stack has tools for every pillar.

With Your Stack, You Can Build...

With Your Stack, You Can Build...

With Your Stack, You Can Build...

With Your Stack, You Can Build...

Stripe, Twilio, SendGrid - all started with stacks like yours.

The Pattern is Always the Same

• Different apps, same architecture
• Learn the pattern, apply it anywhere

What Your Stack Doesn't Cover (Yet)

Redis as Cache: Speed Up Reads

# The problem: database queries are slow for hot data

# Every page view queries PostgreSQL
product = Product.query.get(product_id)  # 50ms each time

# Solution: Cache in Redis (same Redis you already have!)
from redis import Redis
redis_client = Redis.from_url('redis://redis:6379/0')

# Check cache first
cached = redis_client.get(f"product:{product_id}")
if cached:
    product_data = json.loads(cached)  # 1ms - much faster!
else:
    product = Product.query.get(product_id)  # Cache miss
    redis_client.setex(f"product:{product_id}", 3600, json.dumps(product.to_dict()))

• Redis is in-memory: microseconds vs database milliseconds
• Same Redis you use for queues - different use case
• Common for: sessions, hot data, API rate limiting

File Storage: The Missing Piece

# The problem: where do user profile pictures go?

# NOT in PostgreSQL (too large, wrong tool)
# NOT in Redis (ephemeral, memory-based)

# Solution: Object storage (S3)
import boto3
s3 = boto3.client('s3')
s3.upload_file('avatar.jpg', 'my-bucket', 'users/123/avatar.jpg')

# Store the URL in PostgreSQL
user.avatar_url = 'https://my-bucket.s3.amazonaws.com/users/123/avatar.jpg'
db.session.commit()

• Store the file in S3
• Store the URL in PostgreSQL
• Best of both worlds

Vector Databases: The AI Piece

# The problem: "Find products similar to this one"

# PostgreSQL can't do semantic similarity
# Redis doesn't understand meaning

# Solution: Vector database
from pinecone import Pinecone
pc = Pinecone(api_key="...")
index = pc.Index("products")

# Store embeddings (numbers that represent meaning)
index.upsert([(product.id, product.embedding, {"name": product.name})])

# Query by similarity
results = index.query(query_embedding, top_k=10)  # "Find 10 similar"

• Embeddings = numbers that capture meaning
• Vector DBs find similar items by meaning
• Powers: recommendations, semantic search, RAG

Coming Next: System Design

• Week 11: System Design Fundamentals
• The "Seven Building Blocks" framework
• Categorizing every technology you know
• A vocabulary for designing any system
• Week 12: Technology Mapping
• Week 13: Case Studies

What We'll Build

• Flask API to create and redirect short URLs
• Marshmallow validation for URL format
• PostgreSQL to store URL mappings
• Redis + rq for background click tracking
• Docker Compose running all four services

Your Mission

You'll receive a complete, working notification system.

1. Analyze (15 min):
   • How does data flow through the system?
   • Where are the four pillars applied?
   • What does each file do?
2. Modify (25 min):
   • Add a new feature (choose from options)
   • Apply the four pillars in your changes
3. Test (5 min): Verify your changes work

The System: Notification Hub

• API to send notifications (email, SMS, push)
• Background delivery via workers
• Status tracking and history

Modification Challenges

Choose one:

Getting Started

1. Clone the repository (link in bCourses)
2. Run docker-compose up
3. Test the existing API with curl
4. Read through the codebase
5. Identify where to add your feature
6. Implement and test

What We Covered Today

1. Your stack is complete - Flask, PostgreSQL, Redis, rq, Docker
2. Functions are the unit of work - Decorators change execution context
3. Four pillars guide everything - Validate, Separate, Defer, Transform
4. You can build most systems - E-commerce, social, SaaS, APIs
5. What's missing - File storage, vector DBs (specialized needs)

Your Technology Stack

Next Week

• Week 11: System Design Fundamentals
• The "Seven Building Blocks" framework
• Categorizing all technologies
• A vocabulary for any system
• Prep work:
• Universal Building Blocks Lesson 1: Systems Thinking
• Universal Building Blocks Lesson 2: The Seven Blocks