Page History

...

• First download the following file: QueryGenerator.jar
• Run it using command line: java -jar QueryGenerator.jar, for example on Windows after downloading

  • Code Block
    Z:\>java -jar c:\Users\cottrell\Downloads\QueryGenerator.jar Enter the query: SELECT * FROM pinger.source_node; Sending query: SELECT * FROM pinger.source_node;

• Please refer to the following data model to construct the queries
• Example: SELECT * FROM pinger.source_node
  
  Image Modified
• PS: Please, refer to tables as pinger.[table_name]

2) Java Update and ETL:

• Image Added
• E (Extract): This step of the application downloads and extract daily files from the FTP;
• T (Transformation): This step of the application transforms the data following the relational model above;
• L (Load): This step is responsible for loading the transformed data into Impala.

• The cronjob responsible for update the data warehouse everyday is running once a day in the machine 134.79.129.194.

  • Image Added
• The code of the application responsible for running the ETL is in: /home/workspace_eclipse/PingERETL