Blog Post: Complexity #12

Merged
jeff merged 13 commits from BlogPost_Complexity into master 2020-04-19 22:57:46 +00:00
Showing only changes of commit f45c4db91f - Show all commits

View File

@ -24,36 +24,33 @@ Before being able to manage complexity, it must be identified. In addition to t
* Chess: At a glance, chess is straightforward. There are only six types of pieces, and the board is 8x8. The complexity comes from the [permutation of games](https://www.youtube.com/watch?v=Km024eldY1A). It generally takes [a dedicated decade](https://www.quora.com/How-long-does-it-take-to-become-a-chess-grandmaster) to become a grand master.
* Humans: No two humans are the same. DNA on its own is complex; factoring in the expression of that DNA is beyond the understanding of any single person.
* Humans: No two are the same. DNA on its own is complex; factoring in the expression of that DNA is beyond the understanding of any single person.
## Complex Software is Undesirable
All software is complex; it is one of the most complex man-made things. Physical creations such as space ships, submarines, and skyscrapers may be complex, but they are limited by the laws of physics. Software has no such limit. [FreeBSD](https://www.freebsd.org/) is an operating system that was first released in 1993. Using [scc](https://github.com/boyter/scc), a tool to evaluate the size of codebases, it estimates about 16 years with 500 people to recreate FreeBSD 12.1.
All software is complex; it is one of the most complex man-made things. Physical creations such as space ships, submarines, and skyscrapers may be complex, but they are limited by the laws of physics. Software has no such limit. [FreeBSD](https://www.freebsd.org/) is an operating system that was first released in 1993. Using [scc](https://github.com/boyter/scc), a tool to evaluate the size of codebases, it estimates it would take about 16 years with 500 people to recreate FreeBSD 12.1.
While some complexity is unavoidable, it is best minimized. Here are a few reasons to minimize the complexity of software:
* Expensive changes: As software gets more complex, the cost to change increases exponentially. A module bound to several business concerns is more expensive than a module that does one thing. Engineers, business representatives, and managers need to work together instead of just handing a simple task to an engineer.
* Time: While money can be acquired, time keeps ticking. If a project becomes expensive, some larger companies can handle it. If the software is complex enough, the bottleneck becomes time and at that point it is too late. By analogy, if the task is to carry a gallon of water 100 meters, it could be put in a car(expensive, but fast) and cover the distance pretty quickly. If the water must be converted to steam and then back to water, it will take much longer, and the car will not help with delivering it.
* Time: While money can be acquired, time keeps ticking. If a project becomes expensive, some larger companies can handle it. If the software is complex enough, the bottleneck becomes time and at that point it is too late. By analogy, if the task is to carry a gallon of water 100 meters, it could be put in a car(expensive, but fast) and cover the distance quickly. If the water must be converted to steam and then back to water, it will take much longer, and the car will not help with delivering it on time.
* Engineer retention: Engineers who work on simple software run into less frustrating walls. This is more fulfilling work, and will keep engineers on the team. Engineers are less likely to take ownership of problems and code if it is part of a complex mess.
* Engineer retention: Engineers who work on simple software run into less frustrating walls. This is more fulfilling work, and will keep engineers on the team. Engineers are less likely to take ownership of problems and code if it is part of a complex mess.
## Software Engineers Program, but Programmers do not Engineer
## Simple is Difficult, but Effective
An antonym of complexity is [simplicity](https://www.thesaurus.com/browse/complexity). Although it is easy to explain a [simple system](https://simplesystems.tech/blog/post/simplicity), it is not easy to create one. This is the classic investment mindset applied to time. It is better to think twice and code once. Here are a few examples of simplifying principles:
* Encapsulation: This is a very powerful idea, especially when some complexity cannot be mitigated -- it can still be encapsulated. The inner workings of an automatic transmission are quite complex, but it is encapsulated so well that the driver simply selects "D". As [Rob Pike](https://twitter.com/rob_pike) points out in this [video](https://www.youtube.com/watch?v=rFejpH_tAHM&t=789), the simplest interface is no interface at all; garbage collection is completely encapsulated.
* UNIX philosophy: "Make each program do one thing well." This can save software from becoming an indomitable mess. Constantly adding features to the same system will eventually make it collapse in on itself.
* Composition: Software should be composed of interchangeable modules. The boundaries between modules must be well-defined. Chess software should have 2 players, but those players do not need to be humans on a computer. By keeping the player module separate, it can be swapped for an AI player module. This enables better testing, and more interesting games.
These principles are not easy to incorporate into an engineering process. There is both a science and art aspect of designing and building something in a simple manner. Once a system becomes simple, the benefits are exponential. One important benefit for teams to think about is number of engineers who can work on a project. If a system becomes an interconnected mess, usually only 1 or 2 people can work on it simultaneously. If a system is well designed, especially using modularity, the number of engineers who can work on it simultaneously is practically unlimited, because each module is independent.
## Software Engineers Manage Complexity
A programmer understands software languages and can program a computer to accomplish tasks. A software engineer will do the same, with the addition of minimizing the program's complexity. In fact the act of minimizing the complexity takes up most of the engineer's time. Many other tasks such as designing, testing, documenting, refactoring, and code reviews are much more time intensive -- and important -- than the actual programming.
Software engineers must create programs that are simple enough to be understood by other engineers. This enables others to work on the project and create something greater than the sum of the engineers' abilities.
## Simple is Difficult, but Effective
[An antonym of complexity is simplicity.](https://www.thesaurus.com/browse/complexity) Although it is easy to explain a [simple system](https://simplesystems.tech/blog/post/simplicity), it is not easy to create one. This is the classic investment mindset applied to time. It is better to think twice and code once. A few examples of simplifying principles:
* Encapsulation: This is a very powerful idea, especially when some complexity cannot be mitigated -- it can still be encapsulated.
* UNIX philosophy: "Make each program do one thing well." This can save software from becoming an indomitable mess. Constantly adding features to the same system will eventually make it collapse in on itself.
* Composition: Software should be composed of interchangeable modules. The boundaries between modules must be well-defined.
None of these principles are easy to incorporate into an engineering process. There is both a science and art aspect of designing and building something in a simple manner. Once a system becomes simple, the benefits are exponential. One important benefit for teams to think about is number of engineers who can work on a project. If a system becomes an interconnected mess, usually only 1 or 2 people can work on it simultaneously. If a system is well designed, especially using modularity, the number of engineers who can work on it simultaneously is practically unlimited, because each module is independent.
##### ? Monero provides a simple interface, but it was not easy to create. There is a lot going on behind the scenes.
##### Reference Rob Pike's comments on GC (The simplest interface is none at all) https://www.youtube.com/watch?v=rFejpH_tAHM
##### ? design patterns
##### Simplicity is so effective in software that many books and blogs are written on the topic.