You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

1) Sending Queries to DW:

  • First download the following file:  QueryGenerator.jar
  • Run it using command line: java -jar QueryGenerator.jar, for example on Windows after downloading
    • Z:\>java -jar c:\Users\cottrell\Downloads\QueryGenerator.jar
Enter the query:
SELECT * FROM pinger.source_node;
Sending query:  SELECT * FROM pinger.source_node;
  • Please refer to the following data model to construct the queries
  • Example: SELECT * FROM pinger.source_node

  • PS: Please, refer to tables as pinger.[table_name]

2) Java Update and ETL:

  • E (Extract): This step of the application downloads and extract daily files from the FTP;
  • T (Transformation): This step of the application transforms the data following the relational model above;
  • L (Load): This step is responsible for loading the transformed data into Impala.


  • The cronjob responsible for update the data warehouse everyday is running once a day in the machine 134.79.129.194.

  • The code of the application responsible for running the ETL is in: /home/workspace_eclipse/PingERETL

 

 

 

 

 

  • No labels