Invention Grant
US08949175B2 Meta-data driven data ingestion using MapReduce framework 有权
使用MapReduce框架进行元数据驱动的数据采集

  • Patent Title: Meta-data driven data ingestion using MapReduce framework
  • Patent Title (中): 使用MapReduce框架进行元数据驱动的数据采集
  • Application No.: US13466981
    Application Date: 2012-05-08
  • Publication No.: US08949175B2
    Publication Date: 2015-02-03
  • Inventor: Mingxi WuSongting Chen
  • Applicant: Mingxi WuSongting Chen
  • Applicant Address: US CA Redwood City
  • Assignee: Turn Inc.
  • Current Assignee: Turn Inc.
  • Current Assignee Address: US CA Redwood City
  • Agency: Kwan & Olynick LLP
  • Main IPC: G06F17/30
  • IPC: G06F17/30
Meta-data driven data ingestion using MapReduce framework
Abstract:
A generic approach for automatically ingesting data into an HDFS (Hadoop File System) based data warehouse includes a datahub server, a generic pipelined data loading framework, and a meta-data model that, together, address data loading efficiency, data source heterogeneities, and data warehouse schema evolvement. The loading efficiency is achieved via the MapReduce scale-out solution. The meta-data model is comprised of configuration files and a catalog. The configuration file is setup per ingestion task. The catalog manages the data warehouse schema. When a scheduled data loading task is executed, the configuration files and the catalog collaboratively drive the datahub server to load the heterogeneous data to their destination schemas automatically.
Public/Granted literature
Information query
Patent Agency Ranking
0/0