Announcement

Saturday, April 06, 2019

D10X - Starting a new exciting and challenging journey

On this auspicious day of Gudhi Padava (6th April 2019) I am starting a new , exciting, challenging journey with some new friends. D10X is an outsourced product development company.

D10X is founded on 3 key principles 
  • We will make profit when our customer makes money 
  • Quality Products, Delivered Fast at Fair Cost (not always cheap) 
  • Start small, plan and grow as needed.   
It is going to be exciting venture. 

WE ARE HIRING. There are exciting opportunities ahead with different domains and interesting technologies. We are looking for new comers and also few seniors.  If you are interested, contact me.  

#hiringnow, #startupcompany

Saturday, December 15, 2018

Sane Branch Management of Version Control Systems for Teams

[NOTE : This is still a 'draft', please point out mistakes and suggest changes/improvements]
Some time back Nagaraj Mali asked a question about 'best practices for repository branch management' on one of our whatsapp groups. This is commom query. 'bad branching strategy' or 'no strategy about branching' are very common mistakes in project teams. These mistakes can seriously impact the teams productivity and quality. Unfortunately very few manager/scrum masters or tech leads really understand devastating impact of 'bad version control practices'. In this blog, I am attempting to explain my views and logic behind various practices that I recommend.

Mistake : Too many branches


A developer need to understand the difference between a 'branch' and a 'tag'. Many times teams use 'branch' where tag is sufficient. 'branch' is an 'active line' of development. Common practice is product teams is to support 3 previous versions (i.e major releases) for bug fixes and one new release'. For example your current release is Ver10 and you may supporting major Versions9, 8 and 7. Essentially you have 4 'lines of development'. Then you need 4 branches. Ideally you should checkout 'only' the  branch that you are working on. There is no need to checkout all branches. Since 'branches' represent 'live lines of development', once you stop supporting a particular major release, you should 'delete' the branch for that release. Typically you will have 4 to 6 branches and many/many tags.

Mistake : Creating new branch for every minor release


Assume a major release is 'version 9'. Then 'version 9.1' is developed on 'version 9'  branch ONLY. Remember 'Version 9' is the supported major release and 'live development line'. Version 9.1 is usually a 'NOT live development line'. Any code changes/bug fixes for Version 9.1 should be developed in 'Version 9' branch'

Common complaint : We spend long time in merging.

This is another symptom of bad branching and bad practices followed by team. Consider following branch scenarios.

  1. Release branches for past releases

    Lets assume that new releases are developed in 'master' and team is supporting 2 previous release 'Version1' and 'version2'. Now team have done a bug fix in 'Version1'. Obviously customers will expect that same bug fix is available on 'patch release on Version2' and also available when new Version3 becomes available. So every day branches must be merged 'upstream'. i.e. merge Version 1 commits to Verson 2. And merge version 2 commits to 'master'. If you get any merge failure fix them. Daily changes are typically small and can be easily merged. This simple practice ensure that time required to do merges are drastically reduced and bug introduced because of merge problems are almost entirely eliminated.

  2. Feature branches

    Many teams create 'feature' branches for every new feature or bug fix. However, they don't delete the feature branch one the feature development is over. At end of 'feature' development feature branch must be merged into 'master' and then delete the feature branch. It is best to use 'git flow' plugin/workflow for working on feature branches.

    Typical feature branch flow will be 
    • create feature branch from 'master' (so 'parent branch' will be 'master')
    • keep making changes and commiting in feature branch. Its ok to push feature branch.
    • every day merge 'parent branch' to 'feature' branch. This way any changes to 'parent' branch done by other developers are available to you and any conflicting changes are detected early.
    • once the feature is done, merge the 'feature branch' to 'parent branch' (usually 'master'). Since you are regularly merging the 'master' to 'feature', you will not see any conflicts when you merge 'feature' to master. Now delete the feature branch

    In both scenarios 'key' is the regular (preferably every day) merge from 'release branch' to master' or 'master to feature' branch.

Remember It is OK to delete a branch

Remember in version control there is nothing like 'permanent delete'. When a branch is deleted, you will not see it 'branches list'. But that does not mean all the history of such branch is also deleted. Recommended practices is to create a 'tag' from branch and then delete the branch. This will keep the history intact. It will also allow to restore a branch from the tag, if you need to do some critical bug fix in an older (now unsupported) release

Bottomline


Remember 'version control' is 'productivity tool' for the development team and NOT just a backup tool. Learn how to take advantage of the version control system and you will see signficant increase in productivity of your team.  Start by defining policy for 'branch creation/branch deletion and daily merge'.

Thursday, October 11, 2018

Manual Code Review of every single change is OVERRATED

Many software companies have this policy that every change has to be manually reviewed and approved by some peer (or senior) developer either before it is commited or before it is merged in main development branch. This practice is recommended by many software luminaries. However, when implemented in a project/product, I found many practical issues with 100% manual code reviews and reached a conclusion that "manual code review of every single change is OVERRATED"

Over years, I realized that manual code review of every single change is OVERRATED. It does not appreciably reduce post shipment defects but does introduce many inefficiencies in your development process. Manual Code Review of every single change usually have -ve ROI. I find that nobody writes about bad side effects of forcing developers into 100% manual code reviews. Recently I found one article and that triggered me to write this blog post.

So here is the list of problems that I found in 100% Manual Code Review process

Problems in Manual Code Review of every single change

  1.  ROI on Code Review - 
    Is reviewer finding kind bugs in code review such that if they remain undetected will take more efforts than the time spent in Code Review ? If yes, then ROI of Code Review is +ve else ROI of Code Review is -ve.  Most of the Code Reviews have -ve ROI.  
    Reviewer may be a good developer but he may not be really good or may not have an 'eye' for defect. Some years back I did an informal study of 'code review' comments reported in my company's code review system. Majority of the code comments were about 'coding style' or 'comments' or 'naming conventions'. Very rarely a serious bug is detected. Effectively Such code reviews have a '-ve ROI'.
  2. Reviewers time and attention.
    Typically Reviewer is mostly one of the senior developers. He/She also has their own commitments on new feature development. Usually these commitments take priority over code reviews. Hence Code Reviews remain stuck in queue and total feature development time increases unnecessarily (elapsed time from start to feature is delivered in end user's hand).
  3. Team's Most productive developers are 'reviewing' rather than 'writing new code'.
    On the other hand, if Senior developers spend majority of their time doing reviews, then 'most productive developers' are effectively not writing any code. Either case results in lower productivity for the entire team.
  4. Lower developer productivity while fixing bugs reported during code reviews.
    Code review is helps only if review happens within short time writing the code (preferably within a day). However, in general actual review happens after many days. In this time, developer loses the context of code. When finally the code is reviewed and he/she has to fix something in the code, it takes longer because now developer has to 'get in the context' again. Effectively it lowers the developer's productivity again.
Over year's almost every team that I have worked with has made this mistake and effectively reduced the productivity. How do we fix this ? Whats the alternative ?

How to regain the productivity lost by "manually review every code change policy" ?

  1. Institute "Zero Warnings" policy for your code.
    This is easiest and most effective way to eliminate bugs. Every new version of compiler improves in ability to detect potential problems and report them as 'warnings'.  Set your compiler to 'highest warning' level. If you are using Visual Studio, turn on 'report warnings as errors" setting. Fixing warnings is 'practically no-brainer'. Warnings have to fixed immediately. DO NOT wait till end of sprint or end of release to fix warnings.
  2. Take advantage of Static Analysis Tools for your programming language.
    Every programming language now has static analysis tools (linters, tools like Klocwork, Coverity etc). This Wikipedia article will give you a list of static analysis tools. Make static analysis part of your 'CI/CD' pipeline. Make sure that static analysis bugs are fixed within day. DO NOT wait till end of sprint or end of release for fixing static analysis errors.
  3. Identify your critical files and review only those manually.Stop reviewing very single line of change. Identify your critical files (use complexity analysis, analyze version history to find most frequently changed files etc).  Usually 80-20 rule applies 20% are your critical files. Mark those files in system. Any commits in these files should trigger your manual code review process. Manually review these critical files ONLY after warnings and static analysis bugs are fixed.
Development processes/practices followed in a typically software development team usually have serious inefficiencies. By using 'common sense' (rather than following common practices) it is possible to achieve an 'order of magnitude' (2x to 10x) improvement in the team's productivity and delivery quality. It DOES NOT require a change in methodology (like move to Agile) or any new expensive tools. It requires some disciple and taking advantage of many low hanging fruits. The results are usually visible in about 3 months. 

[Shameless Plug]If you want to know how to improve your team's productivity by 2x to 10x, I am available for 'consulting/coaching'.

References

  1. Code Reviews Do Not Find Bugs - How the Current Code Review Best Practice Slows Us Down
  2.