Java, Spark, AWS, EC2, Git (12396 Lines)
I implemented the following components from scratch in Java:
- Cloud-hosted HTTPS web-server
- Server-side persitent key-value store
- Spark-style distributed analytics framework
I then built a web-crawler, indexer, and ranker on top of my system. Following this, I unified all of these components and produced a production-quality search engine in a team of four. We engineered and optimized our design, deploying it to EC2 and processing over 400k pages.
Video Demo Read More