Productive programming - psychology and habits

Navaneesh Kumar -

I previously discussed building quality engineering tools as a group. This post will outline certain programming habits I found helpful over the years. Individual excellence is a crucial factor for the quality of the group. Being a developer is not a simple matter of one’s ability to write working programs at 100 lines per hour. It is a complex blend of quality of thinking, psychological ability to persist, attitude towards solving hard problems, and most important - an “aspiration” to the field.

First, I will go through a few attitudes to acquire as a builder. Next, I will enumerate the personal investments over a decade that continuously boost my ability to learn and solve engineering problems. Finally, I attempted to summarize approaching complex problems inspired by legendaries.

Shifting your attitudes towards engineering

Computer Science or engineering courses at Universities and colleges teach certain basic skills necessary. However, the reality is much more complex, time-consuming, and goes beyond engineering skills. It takes a lot of effort to acquire certain meta-skills, habits, and attitudes to be practically worth your skills.

“The more you sweat in peace, the less you bleed in war.” - Hyman G. Rickover

1. Think beyond your allocated time

We run our daily working sessions for 2-3 hours every day. We dedicate this slot to working on the projects, building tools, discussing, and thinking about solving problems. During the session, we make many technical decisions (big or small) based on various factors. However, it is rare that the thoughts put in to make those decisions are entirely formed during the session. Problems keep lingering in the brain all the time - when trying to sleep, while watching random YouTube videos, or when eating food.

It may seem to be an overkill to impose this kind of practice. In reality, forcing your brain on and off abruptly in a time-bound manner is ineffective. It is unlikely to get faster and high-quality results. Dedicating a lot of time to thinking is necessary to solve complex engineering problems, whether you like it or not.
In simple words, it is really an “obsession.” The obsession comes from your aspiration for the game.

2. An aspiration is necessary

If you are a beginner at programming, you are bound to lose focus and motivation. Writing code can feel like a burden imposed on you. You will be limited by your current ability to deal with the complexity and thus procrastinate. Only your aspiration for “wanting to improve” can sustain you in this situation. Stop and think again - what made you interested in programming in the first place? Maybe you haven’t exposed yourself to the fun yet? In that case, seek the expertise of the group.

What can the aspiration give you back? If you have the right mindset, you will always seek to improve your ability. Higher potential means you can build more powerful things, more achievements to your name, a bigger and better portfolio, acquire more wealth (monetary, knowledge, and peace), and ultimately improve your quality of life. This is a flywheel - once you trigger it with your aspiration, it will keep spinning and growing bigger. Your skills are compounded on every improvement attempt, and so are the results.

Code for fun, code for crafting, not as a “job”.

“Educated Indians who embrace the twenty- first-century mindset will have to follow a few canonical attributes. First, we need high aspirations. Aspirations energize us to overcome limitations posed by the context we are in. They engender and sustain hope, the main fuel for progress. They help us achieve miracles.”

- N. R. Narayana Murthy, founder of Infosys, “A Better India: A Better World”

3. Be Utilitarian

Writing complex programs takes time - you cannot finish it in one shot. Assume that you take about 30 days to complete your program. You can build it in two ways:

  1. Build all the necessary pieces separately, and integrate them at once. Your program is practically usable only at the end of day 30.
  2. Start with a bare minimum “workable” version of the program on Day 1. Keep enhancing it every day. You will have some “usable” version all the time, getting better and better every day.

In the Software industry, the second method is roughly equivalent to Iterative or incremental development.

In the Software industry, the second method is roughly equivalent to Iterative or incremental development.

Let’s walk through how I evolved woof using the above philosophy.

Iteration 1

I started with one simple Python function def echo that prints Hello world.

$ python woof.py echo Hello world 
Hello world

$ python woof.py echo blah blah
Hello world

Iteration 2

Next, I need to pass the command-line arguments to the echo function using sys.argv, and print them back

$ python woof.py echo Hello world 
Hello world

$ python woof.py echo blah blah
blah blah

$ python woof.py phpbb read all
read all

Note that this ignores the names echo and phpbb altogether.

Iteration 3

I experimented a little and learned how to use Python decorators - say @woof, how to pass arguments - @woof('echo'), and how to register all decorated functions into a dict - WOOF_FUNCTIONS = {}.

@woof
def echo(*args): 
    …

@woof
def brew(shots, sugar): 
    print(f"Making coffee with {shots} shots and {sugar} sugar packets")

$ python woof.py echo hello world
hello world

$ python woof.py brew 2 3
Making coffee with 2 shots and 3 sugar packets

Iteration 4: Enabled shell scripting, instead of Python interpreter.
./woof.sh brew 2 3

Iteration 5: Created a web UI to execute woof commands, meanwhile introduced a concept of “woof router” in the library.

Note that at every iteration, we have a usable version of woof. It gets better and better at every iteration. Your program is utilitarian at every step.

I could have taken another approach:

  • Session 1: Build a web UI with logo, input box, etc.
  • Session 2: Build a backend API to accept commands.
  • Session 3: Figure out how to use decorators. Write a simple @woof decorator.
  • Session 4: Connect backend API with @woof decorator
  • Session 5: Integrate all pieces and get the first version of woof working.

In this approach, only at the end of 5th session, woof is usable by other developers.

What are some benefits of the iterative and utilitarian approach?

  • You always have a working version - something that can keep you interested
  • Get small feedback from others at every step.
  • Another developer could start contributing easily at any stage.

Personal investments I made so far

Typing

10 years ago, I started playing Typeracer with 20-30 Words per minute (WPM). I watched the keyboard as I type. Today, I can go upto 70-80 WPM without seeing the keyboard  - i.e. Touch Typing. Many good programmers invest their ability to type faster. Here is a related interesting article - We Are Typists First, Programmers Second.

Programming involves a lot of thinking, and executing your thoughts. Execution involves typing the program using the keyboard. In reality, the human brain runs faster than your execution speed. As you type, your thoughts will be a few steps ahead. The faster you can type and execute, the better the feedback loop will be: think → execute → fail → think → execute → succeed → ... → fail → ... → finish.

Experts solve the Rubik’s cube within seconds in their brain. However, the actual execution involves rotating the sides through the fingers. For speedcubing competitions, a lot of investment goes into manufacturing professional Rubik’s cube that reduce the friction between pieces. Participants also train their hands and fingers for faster execution.

What can you try?

  • Take a physical book or newspaper, and try to digitize it.
  • Game with friends to improve your typing speed - http://play.typeracer.com/
  • Validate: Are you seeing your keyboard as you type?

Tools

  • I use the command line every day. There are a lot of utilities available to be a power user. I use iTerm as the default terminal on my Mac.
  • Practice how to use a debugger
  • Oh My Zsh with autosuggestions - my default shell. I don’t need to memorize the commands - it will autocomplete from history.
  • Fuzzy Jump around directories (instead of cd with full path) - github.com/rupa/z - I don’t memorize the project paths anymore
  • Use a clipboard manager. I use CopyClip for Mac.
  • Utilize the power of a Window manager. You can use i3wm if you are on Ubuntu.
  • Learn keyboard shortcuts - whichever app you are on. For example, VSCode has plenty of keyboard shortcuts as you write code. Learn and use them everyday - be productive.
  • Create your own bash aliases. Find repetitive commands you run, and create aliases.
    • Ex: I run python manage.py runserver on a regular basis. I create an alias alias pmr='python manage.py runserver'
  • Create shell scripts to automate multiple steps.
    • Ex: We have a build_and_run.sh file on many of our projects. I don’t want to run 2-3 docker commands to test every code change!

A useful heuristic: If you have repeated something more than 3 times, find a way to simplify (or automate) it.

Read and experiment

Approaching complex problems

Once you are out of the comfort zone in programming (tailored YouTube tutorials, MOOCs, etc.), you face the reality of complexity and fuzziness. Your method of approaching complex problems has a big impact on the efficiency and outcome.

I try to tackle complex problems in a “bottom-up” fashion. Peter Norvig’s notebook - A Concrete Introduction to Probability (using Python) is an incredible example. How did he approach the concepts?

  • He starts with the simplest step, “definition of probability”.  A simple one-line function P tackles the implementation: look at the elegance!
  • He goes on building tiny pieces (almost all single-line code) - modeling die roll, card decks, etc. as he learns more and more information.
  • Use the smaller pieces to build a slightly bigger piece. And then use the slightly bigger pieces to construct even more bigger pieces, and so on.This approach is more instinctive, i.e. closest to evolution favored by nature. Human body was not planned, but rather evolved from smaller organisms!

Peter Norvig’s notebook on Economic simulation is an yet another example of exploring unfamiliar concepts - Economics Simulation - norvig/pytudes · GitHub

Paul Graham summarizes the Programming Bottom-Up strategy, a key design principle of Lisp programming language.

Here are some tips you can incorporate in solving complex problems:

  • Make a small change (could be as small as one line of code, or even a single keyword), test it, make another small change, test, and so on.
  • Do not perform “big-bang” changes. You will likely end up spending hours fixing the whole mess. It is harder to refactor and much harder to test. Our brain can only remember 7 ± 2 at a time.
  • Seek feedback quickly (self or from others) - fail fast and fail simple. Simple failures are easier to fix. More importantly, press for negative feedback as well - what is irritating? What can be removed?
  • Sometimes, we tend to over-engineer and complicate certain aspects. Look out for simplification opportunities.
  • “Make it work, and then make it better”. Don’t aim for perfection at the first go.
Which way would you rather learn to play the piano: the normal, interactive way, in which you hear each note as soon as you hit a key, or "batch" mode, in which you only hear the notes after you finish a whole song? Clearly, interactive mode makes learning easier for the piano, and also for programming. Insist on a language with an interactive mode and use it. - Teach Yourself Programming in Ten Years (Peter Norvig)
“Everything should be made as simple as possible, but not simpler” – Albert Einstein

Programmer’s way of building a car

On a high level, how does a car manufacturer produce cars?

  • Order all required materials
  • Build all the parts separately
  • Test individual parts (motors, wheels, etc.)
  • Prepare an assembly line and assemble parts according to the design

The knowledge of “building a car” is already acquired. If you are an employee working on the assembly line, a clear instruction is readily available. Your efficiency and perfection are highly valued.

How does programming fundamentally differ from the “manufacturing a car” analogy?

  • Every time, you are building a new and different car.
  • You do not have an instruction manual. Using the documentation of the tools (programming language, library, IDE, etc.), you are pretty much on your own designing, testing, shipping, and even providing service to the car.
  • You traverse through different levels of abstractions and phases - How to become a Fool Stack Programmer
  • Every day, every hour, you learn something about the problem, and implement it. It is a continuous process of learning, rather than - “you take a 4 year training and never look back”. Discover new techniques, new tools and adapt the mental model accordingly.

Approaching problems as a group

Although this post is focused on individual practices, I would like to highlight a few pointers about our attitude as a group.

Adding human resources to a late software development project makes it later. - Brook’s law

Without appropriate coordination, we tend to slow down with more people working on interdependent pieces. Individually, I favor the “bottom-up” approach. However, we approached the design of Hexmos Karma onboarding flow in an almost top-down non-iterative fashion. We were unaware of the complexity involved and how many people would be involved. Here are some aspects we are unhappy about:

  • We don’t have good experience building products as a group.
  • We implicitly are aware of the benefits of iterative approach. Despite that, we steered our approach. Less energy, our prejudiced dislike for certain platforms (JavaScript and Expo) lead us to slow down the progress.
  • We never had smaller working versions of the onboarding flow. As a result, we found many issues during integration - divergence of data models between components, issues in different mobile devices (iOS, Android, screen resolutions, etc.), and required patching several times in all the components.

At this point, we are inefficient at scaling the “bottom-up iterative” approach. As we learn more from our failures, we should come up with better heuristics.