Building in Public: The Technical Debt That Didn't Happen

by Alien Brain Trust AI Learning
Building in Public: The Technical Debt That Didn't Happen

Building in Public: The Technical Debt That Didn’t Happen

Meta Description: We spent 15 extra minutes implementing Azure Key Vault instead of environment variables. That decision saved months of migration pain.

The fastest way to build the Learn Labs enrollment system was environment variables. Create a .env file. Add three secrets. Reference them in the Azure Function. Ship it. Total setup time: 5 minutes.

We spent 20 minutes setting up Azure Key Vault instead.

That 15-minute investment prevented technical debt we’d otherwise pay for months. Here’s why “just ship it” isn’t always the right answer, and how to decide when to do it right vs when to cut corners.

The Environment Variable Temptation

Fast Path (5 minutes):

// .env file
AIRTABLE_API_KEY=pat_xxxxxxxxxxxxxxxx
AIRTABLE_BASE_ID=appRJlF5pg8jzkynB
GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxx

// Azure Function
const apiKey = process.env.AIRTABLE_API_KEY;
const baseId = process.env.AIRTABLE_BASE_ID;
const githubToken = process.env.GITHUB_TOKEN;

Add these to Azure Portal → Configuration → Application Settings. Done.

This works perfectly for:

  • Proof of concepts
  • Internal tools with single users
  • Projects that will never need secret rotation
  • Temporary integrations
  • Learning exercises

We could have shipped in 5 minutes. Why didn’t we?

The Key Vault Decision

Why we chose Key Vault:

1. Expiration Tracking

Airtable Personal Access Tokens expire. GitHub tokens expire. With environment variables, expiration is silent. Your function breaks. Users see errors. You debug for 20 minutes before realizing the token expired.

Key Vault sends alerts 30 days before expiration. You rotate the secret before it breaks.

Cost of reactive vs proactive:

  • Proactive (Key Vault alerts): 5 minutes to rotate before expiration
  • Reactive (environment variable breaks): 30 minutes debugging + user-facing downtime

2. Audit Logging

Who accessed which secrets when? With environment variables, you don’t know. With Key Vault, every secret access is logged.

This matters when:

  • You have multiple team members
  • Compliance requires access trails
  • You’re debugging unusual behavior
  • You need to prove data access patterns

Example scenario: We add a second developer. They need access to test the enrollment system. With environment variables, we send them the secrets via Slack (insecure) or email (worse). With Key Vault, we add an access policy. They pull secrets via Managed Identity. All access is logged.

3. Centralized Rotation

What happens when we need to rotate the GitHub token?

Environment Variables:

  1. Generate new token in GitHub
  2. Update Azure Static Web App → Configuration
  3. Update local .env file
  4. Update CI/CD secrets if applicable
  5. Redeploy to pick up new environment variables
  6. Test to confirm it works

Total time: 15-20 minutes

Key Vault:

  1. Generate new token in GitHub
  2. Key Vault → Secrets → Create new version
  3. Done

No redeployment. No configuration updates. All apps using that secret get the new version automatically (after cache expiry). Total time: 2 minutes.

4. Version History

Key Vault keeps old versions. If the new secret doesn’t work, rollback to the previous version. Click “Set as current.” Fixed.

With environment variables, if you overwrite the old value and don’t have it saved elsewhere, you’re generating a new token.

5. Tagging for Organization

We’re building more than one project. Eventually we’ll have:

  • Learn Labs (Airtable, GitHub, SendGrid)
  • Client Project A (OpenAI, Stripe, Twilio)
  • Client Project B (Azure SQL, Blob Storage, Redis)

Key Vault lets us tag secrets:

airtable-api-key:
  tags:
    project: learn-labs
    service: airtable
    environment: production
    owner: jared@alienbraintrust.ai
    expires: 2027-01-19

Now we can:

  • Query all secrets for Learn Labs
  • Find all production secrets across projects
  • Identify secrets owned by specific team members
  • Generate reports for compliance audits

Environment variables don’t support this. You’d need external documentation that drifts from reality.

The “15 Extra Minutes” That Saves Months

Key Vault setup time:

  1. Create Key Vault resource: 5 minutes
  2. Add secrets with tags: 8 minutes
  3. Enable Managed Identity: 4 minutes
  4. Configure access policy: 3 minutes

Total: 20 minutes vs 5 minutes for environment variables

Time saved over 12 months:

  • Secret rotation: 4 rotations × 13 minutes saved = 52 minutes
  • Debugging expired secrets: 30 minutes saved (1 incident avoided)
  • Access management: 15 minutes saved (audit trail vs email threads)
  • Documentation: 20 minutes saved (tags vs external docs)

Total saved: 117 minutes

ROI: 15 minutes invested → 117 minutes saved = 7.8x return

And that’s just time. The real value is avoiding production downtime from expired secrets and having audit trails when you need them.

When to Do It Right vs When to Ship Fast

Do it right from the start when:

1. Security Matters

If secrets access production data, user data, or payment systems, set up proper secrets management immediately. The time to learn you should have used Key Vault is not during a security incident.

2. It’s Infrastructure You’ll Reuse

Key Vault isn’t just for this project. Once it’s set up, every future Azure project uses it. The marginal cost of adding secrets is ~2 minutes.

3. Multiple People Will Touch It

Solo project? Environment variables might be fine. Team project? Proper secrets management prevents sharing credentials via Slack.

4. Compliance Is Coming

If you’ll eventually need SOC 2, HIPAA, or other compliance frameworks, start with proper secrets management. Retrofitting is painful.

5. It’s a Pattern You’ll Teach

We’re building Learn Labs to teach secure AI prompt engineering. Using environment variables would be hypocritical. We model the patterns we teach.

Ship fast when:

1. It’s Truly a Prototype

If you’re validating an idea and will throw away the code, use environment variables. No need to optimize infrastructure you’ll delete.

2. It’s a Learning Exercise

Building a tutorial? Environment variables are simpler to explain. Key Vault adds cognitive overhead that distracts from the lesson.

3. You’re Solo and Short-Term

Personal project that won’t scale? Environment variables are fine. Just document your secrets somewhere secure.

4. The Tool Doesn’t Support Better Options

Some platforms don’t have Key Vault equivalents. Use what they provide.

Documentation: The Other Investment

We spent 10 minutes documenting Key Vault setup while implementing it. Not after. During.

What we documented:

  • Step-by-step Key Vault creation (KEY-VAULT-SETUP.md)
  • Tagging strategy with examples
  • Access policy configuration
  • Managed Identity setup
  • Cost monitoring
  • Expiration alert setup
  • Secret rotation process

Why document during implementation:

  1. Context is fresh (you remember why you made decisions)
  2. Catches gaps (if you can’t explain it, you don’t understand it)
  3. Future you won’t remember the details
  4. Onboarding new developers takes minutes, not hours

Time invested: 10 minutes Time saved: Hours over the project’s lifetime

The Security-First Mindset

This isn’t perfectionism. It’s pragmatism informed by 25+ years in cybersecurity.

What we’ve seen:

  • Production secrets leaked in GitHub commits (rotation takes hours)
  • API keys exposed in client-side JavaScript (discovered in security audit)
  • Expired credentials breaking production apps (discovered by users)
  • Shared secrets via email (compliance violation during audit)

The pattern: Most security issues aren’t sophisticated attacks. They’re:

  • Secrets in the wrong place
  • No audit trail when you need it
  • No expiration tracking
  • No rotation process

Key Vault solves all of these. The 15 extra minutes today prevents incidents tomorrow.

What We’d Do Differently

Local Development: We still use .env files locally. Key Vault access requires Managed Identity or service principals, which are complex for local dev. We could use Azure CLI authentication, but .env is simpler.

Trade-off: Local secrets in .env, production secrets in Key Vault. Document the difference clearly.

Alternative: Use Azure Key Vault references in local settings with Azure CLI auth. More secure but adds friction to local development.

Secrets Caching: Our Azure Function caches secrets for 5 minutes. This reduces Key Vault calls (cost) but means secret rotation takes up to 5 minutes to propagate.

Trade-off: Cost savings vs instant rotation. For our use case (low-frequency rotation), 5-minute delay is acceptable.

Alternative: Cache for 30 seconds (more Key Vault calls) or no cache (simplest, slightly higher cost).

The Takeaway

Fast ≠ Cutting corners. Fast = Doing it right the first time.

15 extra minutes on Key Vault saved:

  • Future migration from environment variables (2-3 hours)
  • Secret rotation complexity (13 minutes per rotation)
  • Debugging expired credentials (30+ minutes)
  • Setting up audit logging later (1 hour)
  • Documentation drift (ongoing pain)

Total saved: 5+ hours over 12 months

The rule: If it’s infrastructure you’ll use again, spend the extra time to do it right. If it’s a throwaway prototype, ship fast and iterate.

For Learn Labs, Key Vault was the right choice. For your next project, the answer might be different.

Ask yourself: Will this outlive the prototype phase? If yes, do it right. If no, ship it and learn.


Related: In the next post, we’ll dive deep into Azure Managed Identity and how DefaultAzureCredential eliminates credentials from code entirely.

The 15-minute rule: If investing 15 minutes today saves an hour tomorrow, it’s worth it. If it might save an hour someday maybe, ship fast and iterate.