PBC MapReduce
Execute a MapReduce job.
Request
message RpbMapRedReq {
required bytes request = 1;
required bytes content_type = 2;
}
Required Parameters
request— MapReduce jobcontent_type- Encoding for MapReduce job
Mapreduce jobs can be encoded in two different ways
application/json— JSON-encoded MapReduce jobapplication/x-erlang-binary— Erlang external term format
The JSON encoding is the same as REST API and the external term format is the same as the local Erlang API
Response
The results of the MapReduce job is returned for each phase that generates a result, encoded in the same format the job was submitted in. Multiple response messages will be returned followed by a final message at the end of the job.
message RpbMapRedResp {
optional uint32 phase = 1;
optional bytes response = 2;
optional bool done = 3;
}
Values
phase— Phase number of the MapReduce jobresponse- Response encoded with the content_type submitteddone- Settrueon the last response packet
Example
Here is how submitting a JSON encoded job to sum up a bucket full of JSON encoded values.
{"inputs": "bucket_501653",
"query":
[{"map": {"arg": null,
"name": "Riak.mapValuesJson",
"language": "javascript",
"keep": false}},
{"reduce": {"arg": null,
"name": "Riak.reduceSum",
"language": "javascript",
"keep": true}}]}"
Request
Hex 00 00 00 F8 17 0A E2 01 7B 22 69 6E 70 75 74 73
22 3A 20 22 62 75 63 6B 65 74 5F 35 30 31 36 35
33 22 2C 20 22 71 75 65 72 79 22 3A 20 5B 7B 22
6D 61 70 22 3A 20 7B 22 61 72 67 22 3A 20 6E 75
6C 6C 2C 20 22 6E 61 6D 65 22 3A 20 22 52 69 61
6B 2E 6D 61 70 56 61 6C 75 65 73 4A 73 6F 6E 22
2C 20 22 6C 61 6E 67 75 61 67 65 22 3A 20 22 6A
61 76 61 73 63 72 69 70 74 22 2C 20 22 6B 65 65
70 22 3A 20 66 61 6C 73 65 7D 7D 2C 20 7B 22 72
65 64 75 63 65 22 3A 20 7B 22 61 72 67 22 3A 20
6E 75 6C 6C 2C 20 22 6E 61 6D 65 22 3A 20 22 52
69 61 6B 2E 72 65 64 75 63 65 53 75 6D 22 2C 20
22 6C 61 6E 67 75 61 67 65 22 3A 20 22 6A 61 76
61 73 63 72 69 70 74 22 2C 20 22 6B 65 65 70 22
3A 20 74 72 75 65 7D 7D 5D 7D 12 10 61 70 70 6C
69 63 61 74 69 6F 6E 2F 6A 73 6F 6E
Erlang <<0,0,0,248,23,10,226,1,123,34,105,110,112,117,116,115,34,58,32,34,98,
117,99,107,101,116,95,53,48,49,54,53,51,34,44,32,34,113,117,101,114,
121,34,58,32,91,123,34,109,97,112,34,58,32,123,34,97,114,103,34,58,32,
110,117,108,108,44,32,34,110,97,109,101,34,58,32,34,82,105,97,107,46,
109,97,112,86,97,108,117,101,115,74,115,111,110,34,44,32,34,108,97,
110,103,117,97,103,101,34,58,32,34,106,97,118,97,115,99,114,105,112,
116,34,44,32,34,107,101,101,112,34,58,32,102,97,108,115,101,125,125,
44,32,123,34,114,101,100,117,99,101,34,58,32,123,34,97,114,103,34,58,
32,110,117,108,108,44,32,34,110,97,109,101,34,58,32,34,82,105,97,107,
46,114,101,100,117,99,101,83,117,109,34,44,32,34,108,97,110,103,117,
97,103,101,34,58,32,34,106,97,118,97,115,99,114,105,112,116,34,44,32,
34,107,101,101,112,34,58,32,116,114,117,101,125,125,93,125,18,16,97,
112,112,108,105,99,97,116,105,111,110,47,106,115,111,110>>
RpbMapRedReq protoc decode:
request: "{"inputs": "bucket_501653", "query": [{"map": {"arg": null,
"name": "Riak.mapValuesJson", "language": "javascript", "keep": false}},
{"reduce": {"arg": null, "name": "Riak.reduceSum", "language":
"javascript", "keep": true}}]}"
content_type: "application/json"
Response 1 - result from phase 1
Hex 00 00 00 08 18 08 01 12 03 5B 39 5D
Erlang <<0,0,0,8,24,8,1,18,3,91,57,93>>
RpbMapRedResp protoc decode:
phase: 1
response: "[[9]]"
Response 2 - end of MapReduce job
Hex 00 00 00 03 18 18 01
Erlang <<0,0,0,3,24,24,1>>
RpbMapRedResp protoc decode:
done: true