How Mailbox Scaled To One Million Users In Six Weeks

Orchestra, the developers of the wildly popular Mailbox mobile app, have a problem every app developer dreams of having: they need to seriously scale, really fast. Few developers, however, can claim to have scaled to one million users in just six weeks with hardly a glitch. 

To learn from Mailbox's success, ReadWrite sat down with Orchestra's Mailbox engineering lead, Sean Beausoleil. Among some now-common refrains like the need to continually iterate on a project, Beausoleil offers other advice, like the need to significantly limit the number of moving infrastructure parts, and the company's reservation system, which may be novel to many.

Planning For Scale

ReadWrite: Orchestra took Mailbox to 1 million users in just six weeks. Did you expect that level of success?

Beausoleil: We always planned for a large scale system because email is a high-volume data problem, but we weren't expecting the demand that we saw.  When we launched our video back in December, we were hoping for 100,000 views as we ramped towards product launch, but the video received that in under four hours, much to our surprise.  This initial interest made it quite apparent that we would need to support more ambitious scale than we had been expecting. 

Credit: Orchestra

ReadWrite: How do you plan for that kind of growth? You were very deliberate about how you staged your launch. Please walk through the process by which you've scaled up the infrastructure behind Mailbox.

Beausoleil: I think there were three critical phases in the evolution of the Mailbox infrastructure that led to our current scale.

1. Designing, iterating and building with scale and correctness in mind;
2. Simulating large scale as best we could; and
3. Reacting to production load: developing and executing rapidly to scale just in time.

Since we were dealing with email and email is business critical, we designed the system with scalability, availability and correctness in mind. Our goal was to design a scalable system while forcing ourselves to move quickly and iterate on our system and product. In order to do so, we built a modularized system and relentlessly iterated on each component. 

In order to find as many incorrect assumptions and bottlenecks as we could before launch, we built a clone of our system and an IMAP server that simulated production load. This allowed us to find limitations and problems with our backend that would have been painful to fix while trying to keep the system alive.

However, we knew that we weren't going to build a perfect system on day one. When building software, assumptions and constraints change rapidly as the problem and your understanding of the problem evolves, so you need to bake in learning and adjustment time as a necessary part of the process. After seeing the initial demand driven by the video and realizing that the system needed time to evolve, we decided that we had to build a reservation system to help us control the load on our system.

Our highest priority was ensuring that everyone already using the app to manage their email continued to have a great experience.

For several weeks after we launched our entire engineering team worked literally around the clock to identify issues and fix them so that we could continue allowing people into the app. This phase was the raw horsepower behind scaling so quickly. Various pieces of the core infrastructure were either tweaked, sharded, or removed entirely as we learned how our data and users behaved.

Limiting The Number Of Moving Parts

ReadWrite: How did you select the components of your infrastructure? Was it technology that you had used on the to-do app?

The components of our infrastructure is another story of iteration and evolution. Since Mailbox itself was really just an iteration on our to-do app, the technology similarly evolved out of our Orchestra To-Do backend. We were privileged with an opportunity that very few startups are fortunate to have: the chance to completely re-write our entire system. This allowed us to take the things we knew worked and scrap the parts that we knew didn't (both technologies and code that we had written) and start fresh.

However, as we were developing Mailbox, we discovered that some previous technologies we were using either weren't going to cut it given our new constraints or were simply not the right fit for what we needed. So we spent quite a bit of time vetting all of the options available.

For example, I remember one weekend where we built a huge whiteboard matrix with a dozen database options and the pros/cons of each. That allowed us to make the best decision we could at the time for our system. And then we just ran with it. 

Our iPhone app was a pretty direct evolution of Orchestra To-Do, though. We took the data parsing and networking frameworks that we had built to facilitate real-time messaging for Orchestra To-Do and iterated on the pieces that needed improvement. We also took our learnings on how to build an efficient and responsive iOS UI and applied those learnings into our own custom front-end framework that makes the app draw quickly and feel fast. 

One principal we stuck to was that we tried to keep the number of different technologies to a minimum. We didn't want to have to become experts in 20 different things while building out our system. We wanted to become really good at three things and focus as much as we could on our product. 

ReadWrite: Mailbox's infrastructure runs in the cloud. Did you ever consider building out the infrastructure in your own data center? Any thoughts of moving to a dedicated data center as you grow?

Beausoleil: A dedicated data center requires a lot of resources and up-front commitment. We were just a small team trying to build out a large-scale backend. We didn't have the resources to manage a dedicated data center. AWS [Amazon Web Services] was an awesome partner, giving us the flexibility to iterate and scale out our system. The platform proved to be both cost-effective and efficient for our team to build on top of, which was a necessity given our limited resources and tight timeline. 

Expecting The Unexpected

ReadWrite: Your launch wasn't without glitches. At one point, messages wouldn't load, which Orchestra blamed on an "unusual server issue." Looking back, is this something you could have anticipated? Should you have anticipated it, or was it a known unknown?

Beausoleil: I think hindsight is always going to be 20/20. When you look back at any issue with any piece of software that you write, you could say that you might have prevented the issue or should have caught the bug because of reason X. But that's because you now understand part of the problem or have insight into some code path that you didn't have before. It's nearly impossible to write perfect software on your first try and while I think there are things we could have done to prevent various issues or catch them earlier, I'm sure something else would have popped up. When you launch something with any kind of scale, there will invariably be things that fail and need to be fixed.

ReadWrite: Were you to launch over again, what would you have done differently? Are there components of your stack that you've found work less well than you had hoped, that you're hoping to replace?

Beausoleil: Again, hindsight is 20/20. Knowing what we know now, knowing which parts broke and how we were able to fix them, we would definitely have improved certain things. But then we wouldn't have had the opportunity to experience building and scaling our system in such a short amount of time. The end result is awesome, but the journey is what really matters. 

We are continually improving everything that we're doing and trying to make our system more efficient and faster for the end user. I think that will be a forever effort. There's always a better way to do something, some are just harder than others to achieve.

Advice To Other Startups Hoping To Scale

ReadWrite: Any other advice you'd share with startups hoping to get to Mailbox scale?

Beausoleil: Iterate, iterate, iterate. Whatever your current state is, it can be better. Just keep going at it and keep making it better. It will lead to a better overall architectural design as you iterate through your initial assumptions and eventually lead to a more scalable system as you learn how your implementation and data behaves. 

Details matter. Be obsessed with the details, but don't let them get in the way of executing quickly. It requires a lot of really hard work to do so, but is worth every ounce of effort.

View the original article here

Five Mistakes Wireless Users Make, and How to Avoid Them

When it comes to the world of technology, while there are plenty of exciting moves forward, there are also pitfalls to avoid. And for average computer users, feeling too savvy can sometimes lead to major disappointments. After all, there's nothing less pleasurable than assuming that one's time spent online is productive and efficient, only to end up on the phone with the local internet company for hours, trying to figure out why a computer cannot get online in the first place. Or even worse, sometimes people decide that the best way to get motivated is a change of scenery, only they end up making a faux pas at the local Wi-Fi hot spot without even realizing it, only to never snag a table in time again. Here are five mistakes that wireless internet users often make, and ways to avoid them in the future.

#1 - Being too self-assured. Sure, there are plenty of people out there who are legitimately skilled in the digital world. But since most of them are happily employed or getting online from an office at a fancy start-up, the rest of the people who assume that they are doing a great job sometimes are falling short of the mark, causing themselves a slower network speed or too many steps between turning a computer on and connecting to the world wide web. If you assume you're great at surfing the net, it's always good to keep those expectations in check and actually spend time learning what to do.

#2 - Not remembering to click the box that allows one's computer to save passwords for wireless internet networks. Anyone who is feeling rushed or stressed will notice what a godsend this simple move can be for going to places more than once and then needing to pester the waitstaff for the password. Instead of doing that, just remember to click the box in one's web settings that remembers all of that information. It's a much better experience after figuring that out.

#3 - Trying to get too much done with major file uploading or downloading in a public place. It's just not going to work to upload a major chunk of data if there are a ton of people on the same network, because the fact of the matter is that a single router only has a limited amount of space to send the transactions back and forth. Be smart, and make it so that the major chunks of information that need to be sent effectively on the first try are being processed through a superior connection.

#4 - Being confused with the free Wi-Fi available in a spot suddenly dips out or lags. Sure, it's a bummer when there are too many people on a single network to get things done, but the truth is that free wireless internet does have its limitations, and should be no reason to get cranky at the establishment that happens to be providing it! Realize that a ton of people on the same network means this is going to happen, and deal with it--or do tasks elsewhere.

#5 - Not realizing that at-home wireless internet networks should be password encrypted. People can access files in a computer or even poach the signal, both of which are frustrating things to deal with. So be sure that one's own method of getting online wirelessly at home comes with a password that the neighbors--or anyone with a laptop--cannot figure out.

View the original article here

Transatlantic Cable Overrun By Internet Users

Everyone is using internet these days. It is one of the cheapest forms of communication available. It is where most are getting their news, their letters and their advertising these days. What most people may not know is that soon our internet prices may just be going up because we are quickly using up all of the transatlantic cable. In fact, they may be all used up within the next five years.

The boom of telecommunications has brought many fiber optic cables across the Atlantic. The prices have steadily been coming down but now that carriers need to invest in new cables consumers may see these numbers rising again. This is not the news most people want to hear, especially in such a weak economy.

Service providers will soon need to charge more money to cover their costs. In the past of about last twenty years, there have been many new transatlantic cables that were installed. With the new demand of technology, there is now a need for more cables. Laying new cables is not cheap, however.

Since most providers are still using cables that were laid down about twenty years ago, they no longer have to charge extra for these costs. Also, with the high demand for faster internet and a lot of competition most providers have had to dramatically lower the cost of their internet service. This may not last long, however.

Another issue with these cables is that they can only be upgraded on either end. This has provided some higher capacity but they just are not up to par with where they should be. When new cables are laid they will be tremendously upgraded and much faster. This is actually good news for the consumer.

Cables that are laid on land are actually much faster and easier to service. They can support more bits per second per wavelength and upgrading is so much cheaper. The process to install them is so different from transatlantic that they cannot even be compared.

The actual cable that is used is not much bigger than a hose you would use in your garden. It is amazing at how much information can be transferred through it, though. They can be damaged by earthquakes, fishing boats, sea life and more. They constantly need to be maintained and repaired. These cables have been making a lot of companies a lot of money, however, and they have done their job well for a long while.

Transatlantic cable is a great tool that has brought millions of people information and communication that they otherwise may not have had. It has revolutionized internet and the way we get internet. When it needs to be upgraded it may raise our internet prices some but most can agree that it will be worth the extra money. We live in a society that wants more and wants it faster. There is not much more we can ask than new technology to help us achieve just what we want when we want it.

View the original article here