Originally Published on LinkedIn on 15th Oct 2024
One lesson I emphasize to my team is "Scale Changes Everything." Solutions that work for tens of items may not work for hundreds, and usually fail for thousands or more. This is a universal rule, not just in software.
Examples Across Contexts
- Geographical Scale: Solutions that work in small European countries can fail when applied to larger countries like India or China (e.g., BRTS implementation in Pune).
- Business Growth: Startups and companies often struggle to scale. Moving from tens of employees to 200-300 requires significant changes in ways of working. Growing beyond 1,000 employees necessitates rethinking tools, processes, procedures, and practices in finance, HR, and projects.
Technical Examples
- Sorting Algorithms:
- Algorithms for sorting 10 items (e.g., Bubble sort) do not work well for 1,000 items.
- Algorithms for sorting 1,000 items (e.g., QuickSort) don't perform efficiently for 1 million items.
- Algorithms for 1 million items (e.g., Merge Sort) may struggle with 1 trillion items.
Software Development Considerations
The question of scale arises at various levels:
- Microservice vs Monolith:
For a corporate application used by 200-300 users a day, a microservice architecture might be overkill, like using a Bofors gun to kill a mouse - Technology Choices:For a small startup with 10 people, using F# for fast development is fine.- However, if you plan to grow to 100 people, F# might be a poor choice since finding 100 developers proficient in F# can be challenging.
**We need a Mindset Shift**
Unfortunately, very few senior developers and software architects consciously think in terms of scale. They often blindly select a "microservice" architecture.
If the justification for a decision is "everyone is using it", then your company has a serious problem.
