1) Sending Queries to DW:
- First download the following file: QueryGenerator.jar
- Run it using command line: java -jar QueryGenerator.jar, for example on Windows after downloading
Z:\>java -jar c:\Users\cottrell\Downloads\QueryGenerator.jar Enter the query: SELECT * FROM pinger.source_node; Sending query: SELECT * FROM pinger.source_node;
- Please refer to the following data model to construct the queries
- Example: SELECT * FROM pinger.source_node
- PS: Please, refer to tables as pinger.[table_name]
2) Java Update and ETL:
- E (Extract): This step of the application downloads and extract daily files from the FTP;
- T (Transformation): This step of the application transforms the data following the relational model above;
- L (Load): This step is responsible for loading the transformed data into Impala.
The cronjob responsible for update the data warehouse everyday is running once a day in the machine 134.79.129.194.
- The code of the application responsible for running the ETL is in: /home/workspace_eclipse/PingERETL