Scale Changes Everything

 Originally Published on LinkedIn on 15th Oct 2024

One lesson I emphasize to my team is "Scale Changes Everything." Solutions that work for tens of items may not work for hundreds, and usually fail for thousands or more. This is a universal rule, not just in software.

Examples Across Contexts

• Geographical Scale: Solutions that work in small European countries can fail when applied to larger countries like India or China (e.g., BRTS implementation in Pune).
• Business Growth: Startups and companies often struggle to scale. Moving from tens of employees to 200-300 requires significant changes in ways of working. Growing beyond 1,000 employees necessitates rethinking tools, processes, procedures, and practices in finance, HR, and projects.

Technical Examples

• Sorting Algorithms:
• Algorithms for sorting 10 items (e.g., Bubble sort) do not work well for 1,000 items.
• Algorithms for sorting 1,000 items (e.g., QuickSort) don't perform efficiently for 1 million items.
• Algorithms for 1 million items (e.g., Merge Sort) may struggle with 1 trillion items.

Software Development Considerations

The question of scale arises at various levels:

• Microservice vs Monolith:
  For a corporate application used by 200-300 users a day, a microservice architecture might be overkill, like using a Bofors gun to kill a mouse
• Technology Choices:
  
  For a small startup with 10 people, using F# for fast development is fine.- However, if you plan to grow to 100 people, F# might be a poor choice since finding 100 developers proficient in F# can be challenging.

**We need a Mindset Shift**

Unfortunately, very few senior developers and software architects consciously think in terms of scale. They often blindly select a "microservice" architecture.

If the justification for a decision is "everyone is using it", then your company has a serious problem.