Hello Everyone! 👋 Welcome to this interesting journey where we demystify a complex project using an everyday analogy. If you’ve ever been to a library, you’re halfway there in understanding this project! 📚
Why This Project? 🎯
Imagine a world where you can store and retrieve data from multiple computers without even knowing where the data is physically located. Sounds complex? Think about it like a library, a massive one with books stored in multiple rooms. 🏢
I embarked on this project to learn socket programming, which is the backbone of data communication over the internet. In our library analogy, think of socket programming as the walkie-talkies used by librarians to coordinate across different rooms! 📞🗨️
The Library Analogy 📚
Our memory storage system is like a vast library. But, instead of storing books, we store data values on shelves in different rooms, where each room represents a node or a computer. The goal? To ensure you can find your data without having to know where exactly it is stored. In simpler terms, you walk into the library, ask for what you want, and the library guides you to it. 🤓
Requesting Data 📥
When you want to find a book in a library, you’d typically look it up in a catalog. In our project, when you want a piece of data, you make a ‘GET’ request. 📖
Storing Data 📤
When donating a book to a library, the librarian decides where it belongs. Likewise, to store data in our system, you make a ‘PUT’ request, and the system decides its rightful place. 🗃️
Finding the Right Room (Node) 🏢
Each room in our library has a responsible librarian. When you ask for data, the first node you hit checks its storage. If the data isn’t there, the request is passed to the next node. This process continues until your data is found. 🔄
Confirmation ✅
Once the data is located or stored, a message is sent back confirming the successful completion of the task. 🎉
Actual Problem Statement 🤔
The fundamental problem this project tackles is creating a system that allows transparent data storage and retrieval across multiple nodes (computers), emulating Hadoop’s Distributed File System (HDFS) but in a simpler and more understandable manner.
How This Project Simulates Hadoop’s Distributed File System 🌐
This project uses a token-ring architecture to create a decentralized system where each node is aware only of its neighbors. It effectively simulates Hadoop’s HDFS by distributing the data across multiple nodes and using algorithms to locate and retrieve the data as needed. 🔄
Code Overview: Making it Work like Distributed Transparent Memory 🖥️
The project uses socket programming to facilitate TCP and UDP communication between nodes. Each node is capable of handling ‘GET’ and ‘PUT’ requests, either resolving them locally or forwarding them along the ring until they are fulfilled. Messages between nodes also indicate the type of operation to be performed, such as “GET forward” or “PUT forward,” making the system truly transparent. 🗨️
Behind the Tech Scene 🛠️
- Hashing: We use hashing to decide which room (or node) should store which piece of data. 🗄️
- Messages: Types of messages like “PUT forward,” “GET forward,” etc., facilitate communication between nodes. 💌
- Protocols: TCP and UDP protocols are used for reliable communication between the nodes. 🌐
Why is this Useful? 🌟
This system allows for expansion across multiple nodes while simplifying data retrieval. This is similar to how big data systems like Hadoop’s Distributed File System (HDFS) operate, storing massive amounts of data across multiple machines. 📈
Github Repository 📦
To dive into the nitty-gritty details, check out the GitHub repository here: Your GitHub Repository Link.
Final Thoughts 🌟
Whether you’re interested in Hadoop, distributed systems, or socket programming, this project serves as an engaging guide. Using the library analogy, we’ve simplified complex technical concepts, making them digestible and relatable. 🎉
Thanks for reading, and as always, happy coding! 🎈👩💻👨💻
Feel free to share your thoughts below. 🗨️ Until next time! 👋