Continuous Learner #15

MVC
Globalization, Internationalization and Localization in ASP.NET MVC 3, JavaScript and jQuery – Part 1
http://bit.ly/ieAiDS

Windows Azure Pricing Calculator
http://bit.ly/kb1pKD

NuGet for the Enterprise: NuGet in a Continuous Integration Automated Build System
http://bit.ly/kHvvPX

Udi Dahan on CQRS, DDD and NServiceBus
http://bit.ly/kA9hlz

Google to Drop Support in Google Apps for Older Browsers
http://bit.ly/lnzxZO

WCF does not support compression out of the box, so fix it
http://bit.ly/kqoF34

Sharding with SQL Azure – Introducing Federation in SQL Azure

Scalability often mean scaling out, and sharding the DB.
Until now, the sharding has always been managed at the application level, with a lot of manual work by developers with big scalability requirements.

Most of the NOSQL dbs like MongoDB, Raven etc, already support sharding, since it’s much easier to shard a non relational database than a relational database.

Finally MS is trying to add this concept of sharding built in into Sql, specifically into SQL Azure.
A single instance of SQL Azure can be up to 50GB, I believe this limit is due to the fact that each instance is replicated in 2 other locations, so it is be quite difficult to keep in sync a single massive instance bigger than that. For this reason Azure users where invited to shard their data across multiple SQL instances, adding a lot of work  to the developers implementing the solution.

MS released a whitepaper on sharding few days ago [1]. It’s a long and interesting document about the best practices of data sharding, and some details of the implementation of SQL Azure Federations and how we’ll be able in future to do it at the database level instead of doing it at the application level.

The following schematic outlines the concepts in SQL Azure with Federations:

6131.F1.bmp-550x0

 

Cihan Biyikoglu (Program Manager at Microsoft – SQL Azure) has the most interesting blog [2] about SQL Azure and has recently posted about this new Federation concept.

He has a very useful introduction to the new “keywords” (or key concepts if you want) [3] that will be useful to understand how everything works.

  • Federations represent all data being partitioned. It defined the distribution method as well as the domain of valid values for the federation key. In the picture below, you can see that the customer_federation is part of sales_db.
  • Federation Key is the key used for partitioning the data.
  • Atomic Unit (AU) represent a single instance value of the federation key. Atomic units cannot be separated thus all rows that contain the same instance value of the federation key always stay together.
  • Federation Member (aka Shard) is the physical container for a range of atomic units.
  • Federation Root is the database that houses federations and federation directory.
  • Federated Tables refer to tables in federation members that contain partitioned data, as opposed to
  • Reference Tables refer to table that contain data that is repeated in federation members for lookup purposes.

2627.image_thumb_4451A556

It’s all very exiting to see such a technology being implemented at the database level, keeping the ACID (http://en.wikipedia.org/wiki/ACID) properties inside a single Shard, and at the same time allowing the horizontal scalability needed in most modern systems.

1 – Sharding with SQL Azure
http://social.technet.microsoft.com/wiki/contents/articles/sharding-with-sql-azure.aspx

2 – SQL Azure – Your Data in the Cloud
http://blogs.msdn.com/b/cbiyikoglu/

3 – Building Scalable Database Solution with SQL Azure – Introducing Federation in SQL Azure
http://blogs.msdn.com/b/cbiyikoglu/archive/2010/10/30/building-scalable-database-solution-in-sql-azure-introducing-federation-in-sql-azure.aspx

Continuous Learner #4

Continuous Learner #3

AppFabric
http://microsoftpdc.com/Sessions/FT26
http://msdn.microsoft.com/en-gb/windowsserver/ee695849.aspx

Highscalability
HOW MYSPACE TESTED THEIR LIVE SITE WITH 1 MILLION CONCURRENT USERS: http://highscalability.com/blog/2010/3/4/how-myspace-tested-their-live-site-with-1-million-concurrent.html
Scaling Out Facebook: http://www.facebook.com/note.php?note_id=23844338919&id=9445547199&index=0
Understanding the Real-Time Web for Web Developers: http://www.25hoursaday.com/weblog/2010/02/15/UnderstandingTheRealTimeWebForWebDevelopers.aspx
http://www.infoq.com/presentations/The-State-and-Future-of-JavaScript
http://www.infoq.com/presentations/From-Concept-to-Product-Backlog
http://adamyan.blogspot.com/2010/02/aspnet-mvc-2-localization-complete.html
http://blogs.msdn.com/devkeydet/archive/2010/02/09/screencast-managing-browser-history-using-client-script-without-scriptmanager.aspx

Android
Motorola Droid vs Nexus One: Multitouch Test: http://www.youtube.com/watch?v=qzhUzq6bTPg
Android Developer: http://developer.android.com/index.html

Flash vs HTML5
http://techcrunch.com/2010/02/10/hulu-ipad/
http://techcrunch.com/2010/02/05/the-future-of-web-content-html5-flash-mobile-apps/

Hack and Spy
http://cryptome.org/
http://www.youtube.com/my_speed

Cool stuff
Microsoft Pivot: http://channel9.msdn.com/posts/LarryLarsen/The-Tech-Behind-The-Hard-Rock-Cafe/
Mousepaths: http://blog.iso50.com/2010/02/15/mousepaths-hiroyuki-hamada/

F#
LAgent: an agent framework in: http://code.msdn.microsoft.com/LAgent

Ted Talks
Bill Gates on energy: Innovating to zero!: http://www.ted.com/talks/bill_gates.html
James Cameron: Before Avatar … a curious boy: http://www.ted.com/talks/james_cameron_before_avatar_a_curious_boy.html
Blaise Aguera y Arcas demos augmented-reality maps: http://www.ted.com/talks/blaise_aguera.html

Events
Agenda for Microsoft Architect Insight Conference 2010: http://msdn.microsoft.com/en-gb/architecture/ee959262.aspx
TechDays: http://www.microsoft.com/uk/techdays/