Invention Grant
- Patent Title: Transforming and loading data utilizing in-memory processing
-
Application No.: US15227265Application Date: 2016-08-03
-
Publication No.: US10915544B2Publication Date: 2021-02-09
- Inventor: Lawrence A. Greene , Yong Li , Xiaoyan Pu , Yeh-Heng Sheng
- Applicant: International Business Machines Corporation
- Applicant Address: US NY Armonk
- Assignee: International Business Machines Corporation
- Current Assignee: International Business Machines Corporation
- Current Assignee Address: US NY Armonk
- Agency: Edell, Shapiro & Finnan, LLC
- Agent Steven Bouknight
- Main IPC: G06F17/00
- IPC: G06F17/00 ; G06F7/00 ; G06F16/25

Abstract:
A system includes at least one processor and processes an ETL job. The system analyzes a specification of the ETL job including one or more functional expressions to load data from one or more source data stores, process the data in memory, and store the processed data to one or more target data stores. One or more data flows are produced from the specification based on the one or more functional expressions. The one or more data flows utilize in-memory distributed data sets generated to accommodate parallel processing for loading and processing the data. The one or more data flows are optimized to assign operations to be performed on the one or more source data stores. The optimized data flows are executed to load the data to the one or more target data stores in accordance with the specification. Present invention embodiments further include methods and computer program products.
Public/Granted literature
- US20170075966A1 TRANSFORMING AND LOADING DATA UTILIZING IN-MEMORY PROCESSING Public/Granted day:2017-03-16
Information query