How it starts

“we don’t have time to write tests” or “writing tests will slow us down”.

How it ends

“we develop for 2 weeks, after which QAs do sanity testing for 2 to 4 days, then we slowly roll out to production in increments of 1%, 5%, 25%, 50%, 100%”.

Stress free app releases

Then comes the tough question. Why does it take 7-10 days to release 2 weeks of development work?

We then try to lamely escape responsibility

“because… we don’t have enough QAs to test the app”.

“because… if there is a bug we have to again go through the review process on the App/Play stores, hence it’s good to be more thorough and spend that time testing the app”.

“because… we have too much workload, PMs keep asking us to release new features all the time, we have no time to write tests”.

“only if… we could do OTA updates without going through the app review process on App/Play stores”.

“only if… the team that worked on this codebase, before I joined, wrote tests”.

We also come up with cool sounding solutions

“let’s do server driven development (or use web views) so that we can fix some things on the fly”.

Road to stress-free releases

Ranjan goes more into the topic of “Why is test automation the backbone of Continuous Delivery?” , it’s a beautiful article, must read.

Apart from automated testing, there are other common areas which teams end up creating mundane work for themselves or create stressful situations:

  1. Uploading builds manually on Play/App store.
  2. Having a manual step to create a release branch & upgrading version numbers in code.
  3. Following a release train process, but allow for multiple commits after code-freeze without scrutiny.
  4. Having Long-running branches.
  5. Tying feature delivery with app release dates, creating unwanted pressure on engineers.

Automate testing

  1. If you don’t have experience writing tests, don’t delay it any further. A snippet from something I had shared with a team someone recently:
  • You do not need permission to write tests. Do it as part of your workflow.
  • It will feel slower in the beginning if you haven’t practiced writing tests.
  • Practice to get better & faster. Simplest way to start is to write tests for something simple like a calculator, part of a simple game like tic-tac-toe, snake, sudoku, game of life, etc. Then use that knowledge in the mobile framework you use for app development.

Resources

  1. If you don’t have a CI setup to run tests and block merging if tests fail, do it.

Automate release process

Trigger a release from click of a button or by running a script. If there is any more ceremony around releasing the app which requires meeting, manual intervention, get rid of it over time. Automate everything, including version update, release branch creation, uploading builds to the store, uploading dSYM/source mapping files to crash reporting tools, etc.

Don’t hack release trains

Most teams today follow a release train process for releasing mobile apps. If your app release is mostly delayed from the set date for the release, you’re doing it wrong. Some pitfalls which you should avoid:

  • Do not tie publishing of an app version to feature delivery. This delays every app release if you find even one issue in everything was changed. Release trains work effectively if you have alternate ways to roll out or toggle features. This can be done easily for most cases with Firebase Remote Config.
  • Do not allow merges after code freeze. Remotely toggle off features which have critical bugs. For any critical bug raised after code freeze which delays the release train, have an RCA.

Don’t do long-running branches

Why are we still doing this?

Also, do not invent a new branching strategy. A brilliant detailed post on branching for mobile apps on Tramline Blog

Automation is the key for stress-free releases. If you have engineers working on releasing the app, and they do anything apart from automating the process or triggering a few scripts, I’m coming for you 🔫

Wishlist

  1. Automated or data informed rollout based on key metrics like crash rate, ANRs, performance, adoption, etc.
  2. Better testing frameworks, specially for instrumentation testing. Writing, running & maintaining instrumentation tests can be really expensive and time-consuming. All these add complexity: multilingual app testing, multiple themes, device fragmentation, frustrating testing APIs, flakiness, complexity in reproducing issues locally, etc. Looking forward to what mobile.dev team is doing in this space.
  3. Faster feedback loop when writing tests. Build speeds generally come in the way. >10s incremental build time does not go well with testing. Flutter has pushed the bar very high on this .