Invention Grant
- Patent Title: Dynamic resource allocation in MapReduce
-
Application No.: US14176635Application Date: 2014-02-10
-
Publication No.: US09886310B2Publication Date: 2018-02-06
- Inventor: Nicholas C. Fuller , Min Li , Shicong Meng , Jian Tan , Liangzhao Zeng , Li Zhang
- Applicant: International Business Machines Corporation
- Applicant Address: US NY Armonk
- Assignee: International Business Machines Corporation
- Current Assignee: International Business Machines Corporation
- Current Assignee Address: US NY Armonk
- Agency: Ryan, Mason & Lewis, LLP
- Main IPC: G06F9/50
- IPC: G06F9/50 ; G06F9/48

Abstract:
Methods, systems, and articles of manufacture for dynamic resource allocation in MapReduce are provided herein. A method includes partitioning input data into one or more sized items of input data associated with a MapReduce job; determining a total number of mapper components, and a total number of reducer components for the MapReduce job based on said partitioning; dynamically determining an allocation of resources to each of the total number of mapper components and reducer components during run-time of the MapReduce job, wherein said dynamically determining the allocation of resources comprises monitoring one or more utilization parameters for each of the total number of mapper components and total number of reducer components during run-time of the MapReduce job; and dynamically determining a number of concurrently executing mapper components and reducer components from the total number of mapper components and the total number of reducer components for the MapReduce job.
Public/Granted literature
- US20150227393A1 Dynamic Resource Allocation in Mapreduce Public/Granted day:2015-08-13
Information query