As reported on CNET, Facebook has gone to a twice daily code release cycle. Is this too much & too often? For a company with nearly 1 billion users, that is a good question. With a user base that size & being a publicly traded company, such a rapid release cycle raises the question of quality and what exactly is being released.
On what schedule should production level code be released? What are the ramifications of releasing code too often or not often enough? What schedule should be followed? This all depends upon the situation. But figuring out the answer is a relatively simple process.
From a general standpoint, a release cycle schedule should be based on the type of changes being applied. It may be necessary & prudent to use different release cycles, for different types of changes. Prioritizing the types of changes being rolled out, will help define the schedule for you.
It should go without saying, no code or change should be released to production, until it has been thoroughly reviewed for quality assurance purposes. All changes should also be fully beta tested by the developer, before the code is reviewed & tested yet again for quality control. If the changes do not pass all necessary quality control tests, said code should be rejected & subsequently declined for roll-out.
A great way to look at this is to take a page from the airline industry, when it comes to aircraft repair & maintenance. When a jet engine (say from from a Boeing 747 or Air Bus) comes in for maintenance or repair, it needs to follow a predefined set of standards, procedures and processes that have been approved by the airline & signed off by the FAA. Until all of those steps are met, that engine cannot be mounted back on an aircraft & used again for commercial flights.
For the sake of the example, I will dumb a lot of it down. But the overall process is the same, when you apply the same processes to changes for production level coding.
- The mechanic troubleshoots the engine to determine cause of problem or issue.
(The developer troubleshoots the code to determine cause of problem or issue.)
- The mechanic finds the root cause & applies the necessary repairs to address it.
(The developer finds the root cause & applies the necessary repairs to address it.)
- The inspector ensures the repair addresses the given situation, tests said work & ensures the changes were in accordance to the guidelines.
(The developer ensures the repair addresses the given situation, tests said work & ensures the changes were in accordance to the guidelines.)
- The quality control manager reviews/tests the changes again as necessary, and ensures all necessary paperwork is in place.
(The developer's manager reviews/tests the changes again as necessary, and ensures all necessary paperwork is in place.)
- The quality assurance manager finalizes the approval of the repairs & allows the release of the engine.
(The CTO finalizes the approval of the changes & allows the release of the new/updated code.)
- Should everything go ok, and should all necessary paperwork filed & if all approvals were given, the engine is rolled back out to active use.
(Should everything go ok, and should all necessary paperwork filed & if all approvals were given, the code change is rolled to production level systems.)
As you can see, the processes mimic one another. The basic flow ensures that the developer adjusts the code, its inspected & tested to work, then reviewed again for quality purposes. Then only if its all is ok, it can finally be released under their given roll-out release schedule.
As to release schedules, it's my belief that different schedules be used, based on the severity of the change.
- Security Patches - Emergency
- Bug Fixes - High Priority
- New Features & Abilities - Medium Priority
- Code Clean-up & Optimization - Medium Priority
- Cosmetic Changes - Low Priority
The company I work for, use to apply changes as they were completed - without using a release schedule. At first it was ok. But proved to get more crazy & out of hand, as the size of the code base grew & as multiple people began making large sweeping changes to the same libraries -- to remove legacy abilities, rehab older code, apply security updates and perform other code base redesigns for greater usage/efficiency. They since moved to a weekly release schedule for all non-mission critical changes. The only exception to this rule are those emergency security changes that need to be applied immediately, to correct a security issue or a large bug fix.
For this website, I tend to use a same day release schedule; primarily due to myself being the sole coder, the code base is relatively small & it uses a straight forward design/approach. However even being a single coder, I'm starting to see a need to schedule the roll-out of all the upgrades & adjustments, simply to ensure that the quality of the code remains constant.
The ultimate goal is to figure out when changes are being applied & to ensure that proper safeguards, restrictions & time-tables are put in place to ensure the code being delivered to production is always top notch. With the rate in which changes are being applied, such as in the case of Facebook, it does make you wonder if they are really doing the right thing. If the drop-off of their IPO continues, and more people become frustrated with constant UI/access issues (where I am one of them) & if Facebook keeps releasing code at this expedited rate, I makes me wonder whether they will be around for the long haul -- since there is no way sufficient beta testing, inspection, quality control & quality assurance measures are being taken.