1 / 52

.NET and MongoDB

Building Applications with NoRM and MongoDB Fairfield/Westchester Code Camp 2010 2010-11-06 John C. Zablocki Development Lead, MagazineRadar Adjunct, Fairfield University. .NET and MongoDB. Agenda. What is NoSQL? NoSQL Databases Introducing MongoDB .NET Drivers for MongoDB

Download Presentation

.NET and MongoDB

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Building Applications with NoRM and MongoDB Fairfield/Westchester Code Camp 2010 2010-11-06 John C. Zablocki Development Lead, MagazineRadar Adjunct, Fairfield University .NET and MongoDB

  2. Agenda • What is NoSQL? • NoSQL Databases • Introducing MongoDB • .NET Drivers for MongoDB • Introducing NoRM • Case Study: RateMySnippet.com • Questions?

  3. NoSQL

  4. What is NoSQL? • Coined in 1998 by Carlos Strozzi to describe a database that did not expose a SQL interface • In 2008, Eric Evans reintroduced the term to describe the growing non-RDBMS movement • Broadly refers to a set of data stores that do not use SQL or a relational model to store data • Popularized by large web sites such as Google, Facebook and Digg

  5. Wide Column, Key/Value and Document Store... Oh My! NoSQL Databases

  6. NoSQL Databases • NoSQL databases come in a variety of flavors • XML (myXMLDB, Tamino, Sedna) • Wide Column (Cassandra, Hbase, Big Table) • Key/Value (Redis, Memcached with BerkleyDB) • Object (db4o, JADE) • Graph (neo4j, InfoGrid) • Document store (CouchDB, MongoDB)

  7. Why NoSQL?

  8. RDBMSs and the Environment • RDBMS Administrators are highly paid • Highly paid individuals often buy larger than average homes or cars • Larger than average homes and cars require more energy than smaller home and cars • Therefore RDMBSs contribute to global warming more than NoSQL databases which typically do not require the addition of a DBA

  9. RDBMSs and the Environment • RDBMSs often require high end servers that are taxing on disks • High end servers consume more electricity than mid-range servers • Taxed disks fail more often than untaxed disks • Therefore RDBMSs require more energy and produce more waste (lots of hard drives in landfills) than NoSQL DBs, which run on mid-range servers.

  10. Why NoSQL?

  11. NoSQL and Healthcare • The current healthcare crisis requires talented software engineers to fix the outdated or non-existent IT systems of the hospital system • Talented software engineers spend a great deal of time mapping objects to tables in RDBMSs • Talented software engineers are unable to fix healthcare because they are mapping objects to tables • Therefore RDBMSs are causing illnessnes

  12. Dropping ACID

  13. Dropping ACID • CAP Theorem and Eventual Consistency • A distributed computer system cannot provide Consistency, Availability and Partition Tolerance at the same time • Immediate consistency is expensive • Some systems can tolerate dirty reads • Removing relational storage structure helps to reduce transactional requirements • Inserts/Updates written to fewer places

  14. Dropping ACID • ACID support is not required for blogs, status updates, product listings, etc. • Would a 2 second delay in finding out who the mayor of Turnpike Spirits is matter? • Or that your friend is embarrassed to admit that she loves Jersey Shore? • Modern relational databases simply do not scale to Internet proportions • Sites like Digg, Facebook and Ebay have data sets 10s or 100s of TB or even PB large

  15. NoSchema Databases • NoSQL databases generally do not impose a formal schema • Optimized storage for unstructured data

  16. NoSchema Documents • Related data is stored in a single document • The ubiquitous Blog example • Each post is a document • Post might have reference to Blog or User document or redundantly store blog/user data • East post document has a nested collection of tags and comments • Application layer must be smarter about referential integrity

  17. NoSchema Documents varpost = { Author : { Name : "John Zablocki", _id : "497ce96f..." },Title : "On Running NerdDinner on MongoDB with NoRM",Tags : ["mvc", "mongodb", "norm"], CreatedDate : newDate('6-14-2010'),Text : "At the Hartford Code Camp..." }

  18. NoSchema Documents <post> <author> <name>John Zablocki</name> <id>497ce96f395f2f052a494fd4</id> </author> <title>On Running NerdDinner on MongoDB - Part 1</title> <tags> <tag>mvc</tag> <tag>mongodb</tag> <tag>norm</tag> </tags> <created_date>6-14-2010</created_date> <text>At the Hartford Code Camp...</text> </post>

  19. Something Completely Different

  20. Introducing MongoDB • Open source, document-oriented database • 10gen corporate entity behind development • 10gen supports official drivers for many platforms, but not .NET!

  21. MongoDB – The Basics • Schema-less documents stored in collections • Documents are stored as BSON (Binary JSON) • JavaScript used to query and manipulate documents and collections • Each document in a collection has a unique BSON ObjectId field named _id • Collections belong to a database

  22. MongoDB – Server Features • Indexes • Secondary, compound and unique • Replication • Master/slave and replica sets • Auto-sharding (1.6) • MapReduce • GridFS • Large file storage

  23. MongoDB – Hidden Features • MongoDB will help you: • Make more friends • Be more confident in social situations • Be more attractive to members of the opposite (or same) sex

  24. MongoDB on Windows

  25. Installing MongoDB on Windows • Download the binaries from mongodb.org • Extract to Program Files directory (or wherever) • Create a directory c:\data\db • Run mongod.exe from the command line with the --install switch • See http://bit.ly/aed1RW for some gotchas • To run the daemon without installing, simply run mongod.exe without arguments • Run mongo.exe to verify the daemon is running

  26. MongoDB - Shell • The MongoDB interactive JavaScript shell (mongo.exe) is a command line utility for working with MongoDB servers • Allows for CRUD operations on collections • May be used for basic administration • Creating indexes • Cloning databases • Also useful as a test-bed while building apps

  27. MongoDB - Shell /*Connect to a server:port/database (defaults are localhost:27017/test ):*/ mongo.exelocalhost:27017/AltNetGroup //Switch database: use CodeCamp //View collections in a database: showcollections //create an index on Name field db.Artists.ensureIndex({ Name : 1 }); //copy one database to another db.copyDatabase("CodeCamp", "AltNetGroup")

  28. MongoDB – Shell CRUD //Insert an item into a collection db.Artists.insert({ Name : “TheShins” }); //Find an item in a collection: db.Artists.findOne({ Name: “Radiohead”}); //Find items in a collection: db.Artists.find({ Name : /The/i}); //Count items in a collection db.Artists.count();

  29. MongoDB – Shell CRUD //Update an item in a collection db.Artists.update({ Name : “JackJohnson” }, $set : { Name : “JackJohnson” } }); /*Update items in a collection Without $set in this example, whole document would be replaced!*/ db.Artists.update({ Name : { $ne : null } }, { $set : { Category : “Rock” } }, false, true); /*$ denotes special operators and operations $push, $pop, $pull, etc.*/

  30. MongoDB – MapReduce • MapReduce is used for batch manipulation and aggregation of data • A Map function transforms a collection into a series of key/value pairs • A Reduce function operates on that key/value pair series to produce output based on those input values • An optional Finalize function is called after the reduce function is invoked

  31. MongoDB – Shell MapReduce //Create some data db.Posts.insert( { Name : "On Installing MongoDB as a Service On Windows", Tags : ["mongodb"] }); db.Posts.insert( { Name : "On Running NerdDinner on MongoDB with NoRM", Tags : ["mongodb", "norm", "mvc"] }); db.Posts.insert( { Name : "On A Simple IronPython Route Mapper for ASP.NET MVC", Tags : ["mvc", "ironpython"] });

  32. MongoDB – Shell MapReduce /* Create the map function This function creates a view that looks like{ “mvc”, [1, 1] }, { “norm”, [1] }*/ varmap = function() { if (!this.Tags) { return; } for (varindexinthis.Tags) { emit(this.Tags[index], 1); } };

  33. MongoDB – Shell MapReduce /* Create the reduce function conceptually, reduce gets called like: reduce("mvc", [1, 1]); reduce("norm", [1] /* varreduce = function(key, vals) { varcount = 0; for (varindexinvals) { count += vals[index]; } returncount; };

  34. MongoDB – Shell MapReduce /* Run the mapreduce command on the Posts collection using the map and reduce functions defined above store the results in a collection named Tags */ varresult = db.runCommand( { mapreduce : "Posts", map : map, reduce : reduce, out : "Tags" }); db.Tags.find()

  35. .NET and MongoDB

  36. .NET and MongoDB • No officially supported 10gen driver • mongo-csharp • Actively being developed and supported • Supports typed collections • Simple-mongodb • Json-centric • Actively developed, but less so than other drivers like mongo-csharp and NoRM

  37. NoRM

  38. NoRM • OK, so it is pronounced No R M! • Actively developed by Andrew Theken and James Avery, among others • Active community with reliable support • I received help even as I prepared this slide! • Support for typed and untyped collections, MapReduce, Property Mapping, LINQ, Expando, nearly all CRUD operations

  39. NoRM – The Basics //Connections managed with IDisposable pattern //Connect to test database on localhost using (IMongomongo=Mongo.Create("mongodb://localhost/test")) { Console.WriteLine(mongo.Database.DatabaseName); } //Mongo instance has MongoDatabase property /*MongoDatabase has GetCollection<T> methods for accessing MongoCollection instances*/ //CRUD operations performed on collections

  40. NoRM - CRUD //Class properties map to document structure publicclassArtist { //ObjectId property mapped to _id field in document publicObjectIdId { get; set; } publicstringName { get; set; } //IList property will be mapped to JavaScript Array privateIList<string>_albums=newList<string>(0); publicIList<string>Albums { get { return_albums ; } set { _albums =value; } } }

  41. NoRM - CRUD varartist=newArtist() { Name="The Decembrists" }; //Inserting a document into a typed collectionmongo.Database.GetCollection<Artist>("Artists").Insert(artist); //Updating (replacing) a document in a typed collection artist.Name="The Decemberists"; mongo.Database.GetCollection<Artist>("Artists").Save(artist); //Updating a nested collection mongo.Database.GetCollection<Artist>("Artists").UpdateOne( new { Name="The Decemberists"}, new { Albums=M.Push("Picaresque") } );

  42. NoRM - CRUD //Find all documents in a typed collection varartists=mongo.GetCollection<Artist>("Artists").Find(); Console.WriteLine(artists.FirstOrDefault().Name); //Query with a document spec varartist=mongo.GetCollection<Artist>("Artists") .FindOne( new { Name="The Decemberists" }); Console.WriteLine(artist.Albums.Count); //Count the documents in a collection longcount=mongo.GetCollection<Artist>("Artists").Count(); Console.WriteLine(count);

  43. NoRM - MapReduce //Add a Tags collection added to Artist class privateIList<string>_tags; publicIList<string>Tags { get { return_tags; } set { _tags=value; } } //Add some tags – use Set not PushAll, since Tags was nullmongo.Database.GetCollection<Artist>("Artists").UpdateOne( new { Name="The Decemberists" }, new { Tags=M.Set(newList<string>() { "Alternative", "Folk Rock }) });

  44. NoRM - MapReduce //Create map and reduce functons stringmap=@"function() { if (! this.Tags ) { return; } for (index in this.Tags) { emit(this.Tags[index], 1); } }"; stringreduce=@"function(previous, current) { var count = 0; for (index in current) { count += current[index]; } return count; }";

  45. NoRM - MapReduce //MapReduce class is responsible for calling mapreduce command MapReducemr=mongo.Database.CreateMapReduce(); //Represents the document passed to the //db.runCommand in the shell example MapReduceOptionsoptions=newMapReduceOptions("Artists") { Map=map, Reduce=reduce, OutputCollectionName="Tags" }; MapReduceResponseresponse=mr.Execute(options); varcollection=mongo.Database.GetCollection<Tag>("Tags"); Console.WriteLine(collection.Count());

  46. NoRM - LINQ //LINQ provider exposed via AsQueryable method of MongoCollection varartists=mongo.Database.GetCollection<Artist>("Artists") .AsQueryable(); //Find items in typed collection varartistsWithDecInName=fromainmongo.Database.GetCollection<Artist>().AsQueryable() wherea.Name.Contains("Dec") selecta; Console.WriteLine("First containing Dec in name: "+artistsWithDecInName.First().Name);

  47. NoRM - LINQ //Find artists without pulling back nested collections //Note use of Regex name search varartistsWithoutAlbums= fromainmongo.Database.GetCollection<Artist>().AsQueryable() whereRegex.IsMatch(a.Name, "ber", RegexOptions.IgnoreCase) selectnew { Name=a.Name }; Console.WriteLine(artistsWithoutAlbums.First().Name); //Find artists with a given tag varartistsWithFolkRockTag=mongo.Database.GetCollection<Artist>("Artists") .AsQueryable().Where(a=>a.Tags.Any(s=>s=="Folk Rock")); Console.WriteLine(artistsWithFolkRockTag.First().Name);

  48. Case Study: RateMySnippet.com

  49. Final Thoughts Eat food. Not too much. Mostly plants. - Michael Pollan

  50. Final Thoughts Write Code. Not too much. Mostly C#. - John Zablocki

More Related