Scale Changes Everything

 Originally Published on LinkedIn on 15th Oct 2024

One lesson I emphasize to my team is "Scale Changes Everything." Solutions that work for tens of items may not work for hundreds, and usually fail for thousands or more. This is a universal rule, not just in software.

Examples Across Contexts

• Geographical Scale: Solutions that work in small European countries can fail when applied to larger countries like India or China (e.g., BRTS implementation in Pune).
• Business Growth: Startups and companies often struggle to scale. Moving from tens of employees to 200-300 requires significant changes in ways of working. Growing beyond 1,000 employees necessitates rethinking tools, processes, procedures, and practices in finance, HR, and projects.

Technical Examples

• Sorting Algorithms:
• Algorithms for sorting 10 items (e.g., Bubble sort) do not work well for 1,000 items.
• Algorithms for sorting 1,000 items (e.g., QuickSort) don't perform efficiently for 1 million items.
• Algorithms for 1 million items (e.g., Merge Sort) may struggle with 1 trillion items.

Software Development Considerations

The question of scale arises at various levels:

• Microservice vs Monolith:
  For a corporate application used by 200-300 users a day, a microservice architecture might be overkill, like using a Bofors gun to kill a mouse
• Technology Choices:
  
  For a small startup with 10 people, using F# for fast development is fine.- However, if you plan to grow to 100 people, F# might be a poor choice since finding 100 developers proficient in F# can be challenging.

**We need a Mindset Shift**

Unfortunately, very few senior developers and software architects consciously think in terms of scale. They often blindly select a "microservice" architecture.

If the justification for a decision is "everyone is using it", then your company has a serious problem.

Elevating DevOps: A Journey from novice to Master craftsman

 In the dynamic world of DevOps, the gap between theory and practice can sometimes seem as vast as space itself. Reflecting on my journey through countless interviews with DevOps engineers, a pattern emerges: a disconnect between claimed experience and practical knowledge. It's a concerning trend that sees the essence of DevOps, a craft of precision and innovation, being diluted by superficial engagement.

Rewind to the days before DevOps became the industry buzzword. I was at the helm of creating a comprehensive DevOps platform for Geometric Ltd., integrating both commercial and open-source tools. This experience was more than just 'clicking buttons'; it was about architecting a seamless workflow that propelled projects forward.

For those aspiring to master DevOps, here's a distilled essence of my experience:

1. Begin with the roots. Watch John Allspaw and Paul Hammond's seminal presentation at the Velocity 2009 conference. Let it be your DevOps genesis, and absorb its insights multiple times.
2. Dive deep into Version Control Systems (VCS). Whether it's Git, Mercurial, or Subversion, understanding the intricacies of VCS is non-negotiable. They are the backbone of any robust DevOps strategy.
3. Ensure everything is traceable. From source code to CI/CD configurations, every element should be version-controlled, allowing you to pinpoint changes with precision.
4. Immerse yourself in the wisdom of pioneers. My recommended reads include 'Release It' by Michael Nygard, 'The Phoenix Project' by Gene Kim, and 'Accelerate' by Gene Kim and Jez Humble.
5. Learn from the best. Study the engineering blogs of tech giants like Netflix, Google, and Facebook, Uber to stay abreast of cutting-edge practices.

DevOps is not just a role; it's a mindset of continuous improvement and relentless pursuit of excellence. Let's commit to upholding the true spirit of DevOps and nurturing the next generation of software craftsmen. 

Posted on LinkedIn  https://www.linkedin.com/posts/activity-7215942686271238144-7XjI

Transforming a Struggling Software Team into a High-Performance Team - Part 3

In this  third installment of our series, "Transforming a Struggling Software Team into a High-Performance Team," we delve into the pivotal third step of our transformation: improving our version control practices.

Our team's journey with Git, took a dramatic turn as we shifted from the somewhat chaotic 'developer branches' to the harmony of a unified 'trunk' based development approach. This strategic move was not just a change in process, but a cultural shift towards collective code ownership and streamlined workflows for a typical small to medium-sized team.

The creation of two  branches, 'develop' and 'bugfix', from the 'main' branch, marked the beginning of a new chapter. Developers merged their individual efforts (i.e. the individual 'developer branches' into the single 'develop' branch, resolving any conflicts along the way, and then bid farewell to their isolated 'developer branches'. This unification meant that all developers were now committing directly to the 'develop' branch, fostering an environment of collaboration and immediate integration.

To ensure a seamless workflow, we introduced critical practices such as syncing with the latest code at the start of the day and post-lunch, as well as pushing changes before breaks. This routine not only kept each developer codebase fresh but also minimized integration headaches.

Our build strategy also saw a transformation with the introduction of a daily build from the 'develop' branch, a QA build from the 'bugfix' branch for our diligent testers, and a customer release build triggered upon merges to the 'main' branch. This structured approach led to a more stable codebase, with integration issues being swiftly identified and rectified during local feature testing.

Join us as I continue to share this journey in the quest to achieve software excellence. Stay tuned for more insights and strategies that can help transform your team from struggling to high-performing.

Elevating our software team's game, we've changed our version control protocols, catapulting code stability to new heights. Developers now unearth and rectify integration snags swiftly through meticulous local feature testing. We've instituted  release naming conventions, build number tracking, and a streamlined hot fix process. This strategic overhaul has slashed the influx of trivial bugs to QA, empowering our engineers to zero in on pivotal feature and behavior anomalies. The result? A seismic reduction in customer-facing bugs, showcasing our commitment to excellence. (P.S. These insights are versatile, meshing seamlessly with any version control system, be it Subversion, Mercurial, or Perforce.)

NOTE :  This article was originally posted on LinkedIn and expanded.

• Part 1 of this series
• Part 2 of this series