If You are interested to learn create backup In MongoDB
Map-reduce is a data processing methodology for reducing massive amounts of data into useful aggregated output, according to the MongoDB documentation. For map-reduce operations, MongoDB employs the mapReduce command. Large data sets are typically processed using MapReduce.
Syntax of Mongo mapReduce()
Following is the syntax of mapReduce() function that could be used in Mongo Shell
> db.collection.mapReduce(
function() {emit(key,value);}, // map function
function(key,values) {return reduceFunction}, // reduce function
{ out: collection }
)
MapReduce Command
Following is the syntax of the basic mapReduce command −
>db.collection.mapReduce(
function() {emit(key,value);}, //map function
function(key,values) {return reduceFunction}, { //reduce function
out: collection,
query: document,
sort: document,
limit: number
}
)
In the above syntax −
- map is a javascript function that maps a value with a key and emits a key-value pair
- reduce is a javascript function that reduces or groups all the documents having the same key
- out specifies the location of the map-reduce query result
- query specifies the optional selection criteria for selecting documents
- sort specifies the optional sort criteria
- limit specifies the optional maximum number of documents to be returned
Using MapReduce
Consider the following document structure storing user posts. The document stores user_name of the user and the status of post.
{
"post_text": "futurefundamentals is an awesome website for tutorials",
"user_name": "mark",
"status":"active"
}
Now, we will use a mapReduce function on our posts collection to select all the active posts, group them on the basis of user_name and then count the number of posts by each user using the following code −
>db.posts.mapReduce(
function() { emit(this.user_id,1); },
function(key, values) {return Array.sum(values)}, {
query:{status:"active"},
out:"post_total"
}
)
The above mapReduce query outputs the following result −
{
"result" : "post_total",
"timeMillis" : 9,
"counts" : {
"input" : 4,
"emit" : 4,
"reduce" : 2,
"output" : 2
},
"ok" : 1,
}
The result shows that a total of 4 documents matched the query (status:”active”), the map function emitted 4 documents with key-value pairs and finally the reduce function grouped mapped documents having the same keys into 2.
To see the result of this mapReduce query, use the find operator −
>db.posts.mapReduce(
function() { emit(this.user_id,1); },
function(key, values) {return Array.sum(values)}, {
query:{status:"active"},
out:"post_total"
}
).find()
The above query gives the following result which indicates that both users tom and mark have two posts in active states −
{ "_id" : "tom", "value" : 2 }
{ "_id" : "mark", "value" : 2 }
In a similar manner, MapReduce queries can be used to construct large complex aggregation queries. The use of custom Javascript functions make use of MapReduce which is very flexible and powerful.
Example 1 – MongoDB mapReduce()
In this example, we’ll use a school database in which students are a collection, and the collection has documents, each of which contains the student’s name and the marks they received in various subjects. To tally the grades for each student, we’ll use the mapReduce function.
Following is the students collection.
> db.students.find({});
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec39"), "name" : "Midhu", "subject" : "science", "marks" : 68 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec3a"), "name" : "Midhu", "subject" : "maths", "marks" : 98 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec3b"), "name" : "Midhu", "subject" : "sports", "marks" : 77 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec3c"), "name" : "Akhil", "subject" : "science", "marks" : 67 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec3d"), "name" : "Akhil", "subject" : "maths", "marks" : 87 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec3e"), "name" : "Akhil", "subject" : "sports", "marks" : 89 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec3f"), "name" : "Anish", "subject" : "science", "marks" : 67 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec40"), "name" : "Anish", "subject" : "maths", "marks" : 78 }
{ "_id" : ObjectId("5a1f9ce431c157f3ec2aec41"), "name" : "Anish", "subject" : "sports", "marks" : 90 }
mapReduce() In Mongo Shell
Following is a step by step guide to prepare mapReduce function for the use case in Mongo Shell :
1. Prepare Map function
Our map function should emit key-value pair. And in this case, name is key and value is marks.
varmap = function() {emit(this.name,this.marks);}; |
2. Prepare Reduce function
Our map function should emit key-value pair. And in this case, name is key and value is marks.
varreduce = function(name,marks) {returnArray.sum(marks);}; |
3. Prepare mapReduce function
Our map function should emit key-value pair. And in this case, name is key and value is marks.
db.students.mapReduce( map, reduce, { out: "totals"}); |
out: “totals” : the output is written to totals collection in the same database.
4. Start Mongo Daemon
Run the following command in terminal to start mongo daemon.
~$ sudo mongod --port 27017 --dbpath /var/lib/mongodb |
Now Mongo Daemon would be waiting for connections on port 27017.
5. Run mapReduce
Start a Mongo Shell and Run the above commands (in Step 1 to Step 3) in Mongo Shell.
> var map = function() {emit(this.name,this.marks);};
> var reduce = function(name,marks) {return Array.sum(marks);};
> db.students.mapReduce(
... map,
... reduce,
... { out: "totals" }
... );
{
"result" : "totals",
"timeMillis" : 599,
"counts" : {
"input" : 9,
"emit" : 9,
"reduce" : 3,
"output" : 3
},
"ok" : 1
}
> db.totals.find({})
{ "_id" : "Akhil", "value" : 243 }
{ "_id" : "Anish", "value" : 235 }
{ "_id" : "Midhu", "value" : 243 }
The value has been accumulated (aggregated) for the key values and the output is written to totals collection.
Using mapReduce Command in Mongo Script
Following is the Mongo Script file that runs mapReduce command and writes result to totals collection.
mongo-mapreduce-example.js
// equivalent for "use <db>" command in mongo shell
db = db.getSiblingDB('school')
db.runCommand( {
mapReduce: "students",
map: function() {
emit( this.name, this.marks );
},
reduce: function(name, values) {
var value = 0;
for (var index = 0; index < values.length; ++index) {
value += values[index];
}
return value;
},
out: {
replace: "totals"
}
} )
Run the JavaScript file using mongo command
~$ mongo mongo-mapreduce-example.js MongoDB shell version v3.4.10 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.4.10
You may check the totals collection
> db.totals.find({})
{ "_id" : "Akhil", "value" : 243 }
{ "_id" : "Anish", "value" : 235 }
{ "_id" : "Midhu", "value" : 243 }
Conclusion
In this MongoDB Tutorial, we have learnt how to use MongoDB MapReduce function in Mongo Shell and Mongo Script with examples.
