Aws

Part of the Sanchayam series. Overview Sanchayam has two deployable components: Backend: Node.js process running as a systemd service behind nginx Frontend: Static React build deployed to S3, served via CloudFront Prerequisites A Linux server with Node.js 18+ and PostgreSQL An AWS account with an S3 bucket and CloudFront distribution for the frontend A domain with DNS pointing to your server (for TLS) A Twelve Data API key (free tier at twelvedata.com - 8 calls/minute) Backend Clone and install: ...

Part of the MediaBridge series. Overview MediaBridge has two deployable components and two AWS-side components: Backend: Node.js process running as a systemd service behind nginx Frontend: Static React build deployed to S3, served via CloudFront Thumbnail Lambda: AWS Lambda handling S3 object events Archive restore pipeline: CloudTrail, EventBridge, and two Lambda functions Prerequisites A Linux server with Node.js 18+, PostgreSQL, and nginx An AWS account with at least one S3 bucket and IAM credentials for it A domain with DNS pointing to your server (for TLS) Backend Clone and install: ...

Part of the MediaBridge series. Why Thumbnails in Lambda Thumbnail generation on the backend server would mean every file upload triggers a download from S3, a resize operation, and an upload back to S3 - all inline with the upload flow. That adds latency to every upload, burns bandwidth on the server, and blocks the upload response until the thumbnail is ready. Lambda is a better fit. S3 fires an event for every object creation. The Lambda processes it asynchronously, after the upload has already completed and the user has their presigned URL. The backend is not involved. ...

Part of the MediaBridge series. The Problem S3 Glacier and Glacier Deep Archive are cheap cold storage tiers. Files in Glacier take 3-5 hours to restore. Files in Deep Archive take up to 12 hours. When a user tries to access an archived file, S3 returns a 403 (or the object is simply missing from the listing depending on how the bucket is configured). The user gets no feedback about what happened or when the file will be available. ...

Part of the MediaBridge series. Per-Bucket Search vs Global Search The single-bucket search described in the previous post runs one WebSocket and searches one bucket from a given prefix. Global search is different: it searches every bucket the user has access to simultaneously, merging results from all of them into a single view. A user assigned to 10 buckets gets 10 WebSocket connections opened in parallel. Results from all 10 stream in at the same time. Each bucket runs its own DFS traversal independently. ...