290 likes | 383 Views
Is it hard to build a service for 100M user?. Short answer – yes . What would you do if you had 1,000,000 users after 3 days?. 1. Run away 2. Get drunk 3. Pray 4. All of the above. Some numbers…. 100M registered users 2.5B text, 1.7B voice minutes monthly
E N D
Is it hard to build a service for 100M user? Short answer – yes
What would you do if you had 1,000,000 users after 3 days? 1. Run away 2. Get drunk3. Pray4. All of the above
Some numbers… • 100M registered users • 2.5B text, 1.7B voice minutes monthly • 50 employees in Belarus, about 120 worldwide
What do we do? • Free calls, messages and group messages (more coming!) • Your number is your “user id” • Always On…
Why Viber became popular • People love to get something for free • First one to do it “right” • It’s purple:)
Reception • 1M after 3 days • Very high growth ratio • Give us more (platforms, features) • “Where is the Blackberry version?”
What we saw as a result? PROBLEMS!!!
Building a service that can handle 100M users in 1.5 years is difficult. Really.
Why problems? • More usage - more load on backend • More platforms: • More development • Can’t deliver all features on all platforms at the same time.
Scaling “Scaling is replacing all components of a car while driving it at 100MPH” [Mike Krieger, Instagram]
When developers say “fully scalable” they don’t really mean it…
Backend • Got to 1M connected users in less than 2 weeks • Used Amazon EC2, but still you have to design for scalability • Started from 30 servers (and we thought it was a LOT) • So we started to rewrite
Why is it so hard? • Address books (100M) • Reverse address entries (3B) • 300,000 data requests per second • Active dataset 1.5TB • 15-20M concurrent connected users (TCP) • Over 100,000 concurrent phone calls
How to handle all the data • NoSQL (MongoDB, Redis)! • Sharding • MongoDB uses auto-sharding but it’s slow • Simple in-memory solutions require lot of work
We are our biggest enemy • A small mistake on a mobile client can lead to a massive DDOS • One bug in Android client = 100 times load on backend
Say goodbye to life • You are always in the office • Sometime the office is at home • Sometime at the beach • Sometime...
Shit happens • Stuff will break • Stay cool and fix • Communicate with your users
Monitoring • Collect tons of stats • Look at them • Look at them again • Trending matters. Things don’t just happen.
Backend now • Constantly rewriting and optimizing • Runs on 300 servers (and we think it's not too much) • Using and improving open source software
Backend - technologies • Proprietary front-end servers (C++, really fast) • Redis – in memory NoSQL, fast • MongoDB – persistent NoSQL, not so fast • MySQL – registrations, pretty slow • Always looking for alternatives (CouchBase, DynamoDB, etc)
Platforms • Users wants to communicate with their friends, but they keep buying different phones • Started from iOS and had Android community bashing us for a year • Now: iOS, Android, Blackberry, WindowsPhone7 • More coming…
Client challenges - iOS • Support numerous iOS versions (we’re finally dropping support for iOS 3) • Support several device type (3G, 3GS, 4, 4S) • Need to react very fast (iOS upgrade)! • Hard to test on large scale (no beta testing, although there is a one-time solution)
iOS Device Breakdown 6 users registered with iPhone 5
Mobile upgrade speed Desktop application upgrade Mobile application upgrade
Client challenges - Android • Devices/platform fragmentation (we saw more then ??? devices) • Lot of sound-related bugs caused by it • Different screens and device caps • APIs not always work as documented and not always documented
Client challenges - voice & video infrastructure • Have to choose/write a proper engine • Now we are using and modifying WebRTC • Has to work smoothly on mobile devices • 3G is a pain • Has to optimize for several platforms
Lessons learned • Always plan for success • Monitor your system • Hire best people • It’s VERY REWARDING! hr.blr@viber.com