Distributed Systems & Distributed Computing Part II

Why not to use one single supercomputer that could do everything we want and save ourselves from the trouble? Why to use that several computers and add the overhead of managing and maintaining them? That’s actually a good question and the answer is the motives of a distributed system which are many.

One of the most common one is the need for a solution where we need to reach a consensus among parties that are dispersed geographically. Examples of such cases are online banking, blockchain, bidding platforms and more.

Resource sharing is another reason why we need distributed system such as databases or distributed file systems. Another example that falls under the same reason are P2P (peer-to-peer) systems like the BitTorrent Network.

Another case where we need a distributed system is when it’s inefficient to replicate data using one computer or scalability has crucial importance, distributed system is probably the optimal solution. Of course, there are other reasons why one would choose a distributed system such as availability and reliability.

These are the main reasons why one would prefer to go for a distributed system. Related to the motives for using such systems are the advantages of distributed computing. Before talking about the advantages and challenges of distributed systems, let’s briefly talk about their types.

When talking about types of distributed systems or any kind of systems, it’s important to understand that classification depends on the context and on the level of complexity we’re taking into account. Are we talking about architecture? Are we classifying the system based on the end goal? Is the context related to the topology? Is it related to how couple nodes are? In this section, I’m going to address two types of classification. First is general and relates to the type of Coupling & Scale. The second is related to the Architectural Model of the whole system.

When talking about distributed systems in terms of Coupling & Scale, there are two main types of distributed computing; Cluster Computing and Grid Computing.

In Cluster Computing systems, the underlying infrastructure is composed of identical computers that are closely connected, and the management is local and centralized. Cluster computing is used to achieve high performance and minimize downtime.

Grid Computing, on the other hand, is a type of system in which heterogeneity is the norm in terms of hardware, software and technology. Nodes are dispersed over a very large area and administration is decentralized. Such systems are used when a large repository of data is involved, and a lot of computing power is required.

If we are talking about Architectural Model, distributed systems fall under one of the following categories.

First, Layered Architecture. In this architectural model, nodes are grouped into separate layers each with a specific goal to achieve. An example of a simple layer distributed system is the client-server model.

Next is Object-based Architecture. In this model, subsystems are less structured and loosely coupled than the layered model. Communication is a synchronous, and the elements of the system can directly interact with other elements through direct calls.

The third architectural model is the Data-centered Architecture. In such systems nodes communicate through a common repository. The system is based on a data center through which communication happens.

The last one which is more common one is the Event-driven Architecture. Event-driven systems achieve their goals by means of events. Nodes communicate and perform operations through the propagation of and the reaction to events.

(For more information about Architectural Models, you can check my previous articles).

Now, it is time to talk about the pros and cons of distributed computing. As always, let’s start with the advantages.

First advantage I want to talk about is reliability. A defining characteristic of distributed systems is the reliable interconnection and cooperation between nodes in the whole system. This makes it easy to share data between nodes.

Another great advantage is scalability. In fact, this is one of the main reasons to why someone or an organization would opt for a distributed system. In distributed computing, scalability is a matter of adding more nodes to the system at the correspondent layer or level. Or what is referred to as horizontal scaling.

Another common advantage is fault tolerance. it means the system and its services will still be operational and reliable even when parts of the system goes down.

Last but not least, increased performance.

As all matters of life, distributed systems are not a silver bullet and have disadvantages.

First failure detection. Failure detection is almost impossible in distributed systems, especially if the system is large and evolves over time. Very elaborate measures need to be taken to reduce the risks of this issue.

Redundancy is a common issue among distributed systems as well.

Another challenge or problem is the difficulty to achieve consistency among nodes.

Finally, performance bottlenecks is a serious issue also.

When it comes to distributed systems the more we design it at all levels, the less problems we will have. Most important pitfalls when building distributed systems is the assumptions made by developers such as “the network is reliable and secure”, “topology doesn’t change”, or “zero latency”. This leads to a poor and fragile system that breaks easily. Actually, one of the crucial principles developers and architects abide by when designing and developing distributed systems is the “Design for Failure” principle. Under this rule, there are a lot of considerations to take into account. But generally speaking, “Design for Failure” is expecting the worst-case scenario in each and every aspect when the system is being built.

This should be enough for a brief overview. We have reached the end of 2nd and last part of the article “Distributed Systems & Distributed Computing”.