ScrollSafe

ScrollSafe Monorepo

ScrollSafe is a full-stack system that flags likely AI-generated short-form videos in real time. It combines a Chrome extension, FastAPI services, Redis/Celery workers, residential resolvers, and a Hugging Face inference endpoint to overlay trustworthy signals on YouTube Shorts, Instagram Reels, and other feeds while you browse. This repository is the home for all future ScrollSafe development and the project I present to recruiters.

Recognition: Built for Google Developer Group Windsor’s Innovate with AI challenge (107 participants / 15 teams). ScrollSafe was selected as one of 6 finalists to demo live on Nov 7 and won 2nd place during GDG DevFest on Nov 8. Demo + write-up: Devpost.

Stable Release (YouTube-first): The publicly shared v1 Chrome extension-which is the version most recruiters will evaluate-focuses on YouTube Shorts and is proven stable there. It is available on GitHub releases as a packed .rar for manual loading: ScrollSafe Chrome Extension v1.0.0. Instagram Reels and TikTok adapters are in active development inside this monorepo. Once those implementations are production-ready, we plan to publish ScrollSafe to the Chrome Web Store so users can install updates seamlessly.

System Design (At a Glance)

ScrollSafe High-Level System Design

Product Screenshots

ScrollSafe popup stats and history ScrollSafe badge on YouTube Shorts

Deep scan progress on Instagram Reels Deep scan result badge

Recent videos history view ScrollSafe system components overview

Why ScrollSafe

Repository Layout

Path Description
scrollsafe-frontend/ Chrome Extension (Manifest V3) content scripts, adapters (YouTube, Instagram, TikTok), badge UI, and background service worker that proxies API requests and captures frames.
scrollsafe-backend/ FastAPI service exposing heuristics + deep scan APIs, admin metrics, and cache/database integration (Redis/Postgres). Includes Celery job helpers and heuristics.
resolver/ Residential resolver tooling that runs yt-dlp → ffmpeg frame capture behind a Cloudflare tunnel to avoid cloud IP throttling; provides proxy-aware pipelines and local smoke tests.
scrollsafe-doomscroller/ Discovery/analyzer workers, DB schema, and scripts that crawl trending shorts, run batch inference, and backfill caches (“doomscroller pipeline”).
scrollsafe-inference-api/ FastAPI inference endpoint that batches frames and serves the fine-tuned haywoodsloan/ai-image-detector-dev-deploy SwinV2 model (GPU-friendly).
scrollsafe-inference-api-cpu/ CPU-optimized variant of the inference API for low-cost deployments.
out/, test/ Local artifacts and system tests (ignored in git).

Key Features

Architecture Overview

ScrollSafe High-Level System Design

  1. Chrome Extension (MV3):
    • Content scripts inject the badge, adapters detect the active video, and the pipeline coordinates heuristics, caching, and deep scans.
    • The background service worker proxies API requests (avoids PNA restrictions) and captures frames via chrome.tabs.captureVisibleTab.
  2. Backend API (FastAPI):
    • /api/analyze runs metadata heuristics (regex + rules).
    • /api/deep-scan enqueues deep scans and returns job IDs.
    • /api/deep-scan/{job_id} polls Redis for status/results.
    • Admin endpoints expose queue depth, verdict metrics, and manual overrides.
  3. Task & Data Layer:
    • Redis: Queue + cache; dedupe deep scans; store transient verdicts.
    • PostgreSQL: Long-term storage for analyses, admin overrides, telemetry.
    • Celery Workers: Deep scans (yt-dlp → ffmpeg → inference → aggregation), discovery sweepers, analyzer jobs, and admin-label syncing.
  4. Inference Endpoint:
    • FastAPI microservice hosting the SwinV2 classifier (GPU or CPU build).
    • Accepts multipart frames, batches inference, returns {"real": p, "artificial": q} per frame plus batch timings.
  5. Resolver Apps:
    • Proxy-aware Python services (with IPRoyal creds & optional cookies) running on a trusted residential device. They expose /extract-and-infer, ensuring reliable downloads even when cloud IPs are throttled.
  6. Design Diagram: See design.png (also embedded above) for end-to-end data flow.

Tech Stack

Challenges & Solutions

Challenge Solution
Dynamic DOMs (YouTube/Instagram re-rendering) Implemented modular adapters + MutationObservers. The pipeline re-attaches badges, refreshes metadata, and prevents result bleed-over.
Cloud IP throttling of yt-dlp / ffmpeg Separated frame extraction into the resolver/ app running on a residential IP via Cloudflare Tunnel. Added progressive-MP4 preference, HLS/DASH fallbacks, cookie-aware retries, and idempotent deep-scan locks.
Balancing latency vs. accuracy Debounced heuristics (2 s), cache-aside (Redis/Postgres), and asynchronous deep scans. Badge always shows immediate heuristics while deep scans run in the background.
Frame extraction robustness Multi-stage extractor: metadata probe → fast path (yt-dlp pipe) → fallback B (direct ffmpeg with headers) → fallback C (temp download). Captures 16 evenly spaced frames per clip.

Getting Started

  1. Clone & Install 
    git clone <repo-url>
cd Hackathon-project
python -m venv .venv && . .venv/Scripts/activate  # or source .venv/bin/activate
pip install -r scrollsafe-backend/requirements.txt
  2. Environment
    • Copy .env.example files inside each subproject (frontend uses runtime config).
    • Provide API keys:
      • YOUTUBE_API_KEY, HUGGING_FACE_API_KEY, INFER_API_URL, ADMIN_API_KEY.
      • Resolver proxies (e.g., IPRoyal) and YTDLP_COOKIES_FILE.
  3. Run Backend 
    cd scrollsafe-backend
uvicorn main:app --reload
  4. Start Workers / Resolver 
    cd scrollsafe-backend
celery -A deep_scan.tasks worker -Q deep_scan -l info

cd ../resolver
python main_run.py  # smoke-test pipeline / resolver connectivity
  5. Load Extension
    • Go to chrome://extensions, enable Developer Mode, “Load unpacked” → scrollsafe-frontend/.
    • Pin “ScrollSafe” and open YouTube Shorts to see the badge in action.

Tip: For production, deploy the inference API (scrollsafe-inference-api), backend, and workers via Docker Compose. Residential resolver can run on a local machine using start-resolvers.ps1.

Metrics & Observability

Demo & Media

Roadmap

  1. Model Enhancements: Expand dataset with newly labeled clips, add explainability snippets (saliency / textual cues), and support additional modalities (audio cues).
  2. Platform Coverage: TikTok adapter parity with YouTube/Instagram, plus support for Facebook Reels and Twitter videos.
  3. Rate Limiting & Quotas: Per-platform throttling, user opt-in analytics, and dashboard for admin overrides.
  4. Reporting & Sharing: Allow users to share verdict cards, integrate with newsroom workflows, and add contextual education links.
  5. Mobile & Edge: Explore Firefox/Safari extensions and serverless inference for low-latency bursts.

30-Second Pitch

“ScrollSafe is a Chrome extension and backend that flags likely AI-generated short-form videos in real time. I built it end-to-end—extension UI, FastAPI services, Redis/Celery queues, yt-dlp → ffmpeg frame extraction, and a SwinV2 inference endpoint. We engineered around cloud IP throttling by tunneling a trusted edge and added resilient fallbacks. The MVP achieved ~80% internal accuracy, 80% fewer heuristics calls via debouncing, and 100% badge attach reliability across QA. We placed 2nd at GDG Windsor’s Innovate with AI after live demos to judges and DevFest attendees.”

Contact

Thanks for checking out ScrollSafe! Let me know if you’d like a guided walkthrough or deployment assistance.