Lessons From Launching Chelsea Handler: Gotta Go

When launching an application we typically gain users organically and scaling issues are managed over time as the user base grows. With Chelsea Handler’s Gotta Go, it was a unique situation because we immediately had thousands of people downloading and using the app, after it’s debut on Netflix.

The app helps users create excuses to leave uncomfortable situations by sending them real text messages and phone calls from a contact they set-up in their phone’s contact list. By the end of two days, we were up to tens of thousands of users out of the gate... and still growing! The massive early traction the app received presented us with some unique opportunities and challenges, some of which are outlined in this post.

Gotta Go’s on Fire!

As alluded to above, the early success brought some technical challenges along with it. Saturday morning we launched the app  only to realize that verification codes and excuse text messages were not reaching our new users. Due to some analytics we had set-up we saw that users were creating accounts, but only half of them made it through the verification flow. Gotta Go was broken and the documentary had only been on the air for 8 hours! What was going wrong?

Upon digging deeper into the issue, we realized that the phone number we were sending verification codes from was marked as spam by the major phone carriers. They were properly being sent, but being blocked and filtered as spam before reaching our user’s phones. Carriers do this when they detect a phone number is sending 100’s or 1000’s of text messages rapidly. In hindsight, this makes complete sense! What steps did we take to alleviate this situation?

Step #1 - More Numbers!

With thousands of users streaming in, we needed to get creative to ensure they had a positive experience despite the technical issues - so we came up with a short term fix. In an attempt to patch the issue while we figured out a longer term solution, we purchased 10 more phone numbers to send verification codes from. We then coded a hot fix for our server, which would round robin a number to send the verification code from. This would hopefully buy us some time and allow users to still login to Gotta Go. Unfortunately, within an hour of implementing this solution, some of the new numbers we were using started to be marked as spam. So while this approach did buy us a short block of time, we quickly needed to find and implement a more sustainable solution.

Step #2 - No More Spam

Fortunately, during the temporary relief provided by the new phone numbers, we found out that we needed to purchase a short-code phone number in order to send messages that would not be marked as spam. Short-codes are those 5-digit numbers that show up as the sending phone number when receiving text messages from an application. Again, this completely makes sense in hindsight, but when we were implementing SMS and Voice functionality for GottaGo we never came across anything recommending getting a short-code; #1 Likely because this is only necessary once you have a certain amount of active users at a time and #2 it costs a significant amount of money just to register one! Lesson learned!

Step #3 - Wrong Number

Now that our verification codes and excuses were being delivered to 100% of our users, we still had a problem. Excuses were now coming from this new short-code number, not from the phone number that we had saved in the user’s contact! We needed to come up with a solution that would enable the excuse messages to technically come from the short-code number while still appearing to come from the actual contact in the user's phone. We quickly put together a new version of the iOS application that would save this new short-code into the contact as well. We quickly got some beta users to test that it worked properly and submitted the build to Apple for approval. Apple was able to approve this for us in under 24 hours and we started Sunday off a-new with a fresh new version of the application for everyone to download.

Takeaways - Testing & Monitoring!

The big success of the app brought to light areas for technical improvement that likely wouldn’t have surfaced under different, less publicized circumstances. Obviously, it would have been more ideal to have learned about short-codes and hit the spamming threshold during beta testing or load testing. During early simulations and tests, what we estimated to be a “normal” amount of users was not enough to cause the carriers to mark our phone number as spam. We did however test the application and spend time bug fixing pre-launch, which has led to less than 1% of users experiencing issues within the application.

In retrospect, given the large volume of early app users, it would’ve been optimal to have implemented a systematic process for changing the phone numbers saved in the user’s contact. In a future build, we’re implementing the ability to make the phone numbers which are saved in the contact configurable so that it  isn’t necessary to push out an emergency build of the app if for some unforeseeable reason we need to change the numbers again.

As a positive, it was invaluable that we had put in place a group of tools to help us monitor the launch of Gotta Go. If we had not put in a series of error, performance, and analytics monitoring tools (Sentry, New Relic, Fabric, etc) we would not have been as quick to notice when our number was marked as spam. These tools continue to be helpful as we’ve been working on some bug fixes and slowly scaling up different parts of our back-end with load.

Next Steps

Even after successfully launching Gotta Go, seeing early & sustained traction, and patching up unforeseen technical issues, there is still plenty to be done to ensure the app can continue to scale. Many users are excited about the app’s future, and are sending in suggestions & requests for new functionality. There are also sporadic bugs relating to contacts & verification codes which are important to resolve efficiently. Ultimately, our goal is to continuously optimize the user experience so that even as the app scales, the overall quality of experience continues to move forward as well. To discover more about Gotta Go and how Yeti approached the project, read more here.