Versions Compared

Old Version 3

changes.mady.by.user Unknown User (jlrulff)

Saved on

compared with

New Version 4

changes.mady.by.user Unknown User (jlrulff)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • First download the following file:  QueryGenerator.jar
  • Run it using command line: java -jar QueryGenerator.jar
  • Please refer to the following data model to construct the queries
  • Example: SELECT * FROM pinger.source_node

  • PS: Please, refer to tables as pinger.[table_name]

2) Java Update and ETL:

  • Image Added
  • E (Extract): This step of the application downloads and extract daily files from the FTP;
  • T (Transformation): This step of the application transforms the data following the relational model above;
  • L (Load): This step is responsible for loading the transformed data into Impala.

  • The cronjob responsible for update the data warehouse everyday is running once a day in the machine 134.79.129.194

  • To access or modify the cronjob you can access Trscrontab -e