Before you clone a Github Repo

  1. Create a GitHub account: https://github.com/
  2. On the Linux machine that you will clone the GitHub from, generate the SSH key (if not already done): https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/
  3. Add the new SSH key to your GitHub account: https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/

  4. Setup for large file systems on GitHub.

    git lfs install

  5. Verify that you have git version 2.13.0 (or later) installed.

    git version

  6. Verify that you have git-lfs version 2.1.1 (or later) installed.

    git-lfs version

Clone the git repository

Clone the example repository into the "convDemo" directory.

git clone -b Conv_Example --recursive https://github.com/slaclab/snl-Dave.git convDemo


Training your own model and transferring it to SNL

***Note: This repo assumes that you are using the default CNN model. If you are using a different model, make sure you define its structure in all necessary files and follow the instructions in this section.***

  1. Open the Jupyter notebook "Convo_Example.ipynb" from "convDemo/firmware/python/snl_Convo_example".

  2. This Jupyter notebook trains a conv-maxpool-conv-dense-dense model on the MNIST dataset. After you run the notebook, the weights and biases will be saved to .txt files, and the entire model, as well as the weights and biases, will be saved to a .h5 file.

  3. In "convDemo/firmware/shared/SnlConvoExample/src/Network.cc," ensure that the model is defined in class Wba. For example, for this model, the class looks like this:


  4. Now, modify the processStream() method to reflect more or fewer layers. For this model, it looks like this:


  5. Next, modify the processNetwork() method to reflect more or fewer layers. For this model, it looks like this:


  6. Now, go to "convDemo/firmware/shared/SnlConvoExample/include/Parameters_Set0.hh" and add/modify namespaces to reflect the correct layers. For this model, an example looks like this:

    ***Note: Currently, SNL only supports stride=1 and padding=Valid and odd kernel sizes for conv2d. For maxpool2d, it can support stride >= 1 but only supports padding=Same.
    Ensure that you clearly define the layer type. For this layer, it is "snl::LayerType::Middle." If you wish to add more layers, just copy the relevant layer and change the parameters.

  7. After modifying all namespaces, modify the following pieces of code to reflect the changed layers:


  8. Next, it is time to test the kernels. Modify the following piece of code to reflect the changed layers:


    For each layer in your model, change the name of the .txt file you will use to test in the following code:


    If you want to add more layers, just copy the example layer code and ensure that you change the variable names accordingly. Pay close attention to the difference between setting weights and setting biases for dense layers. For example:

    m_layer3.m_weights[0].set (x, cnt); # weights
m_layer3.m_bias[cnt]=x;				# biases

  9. In firmware/shared/SnlConvoExample/src/Test.cc, in line 298, feel free to modify the test image. Right now, it is an image of all 1's. You can modify it to be all zeros or some other number. If you do modify the test image, ensure you can recreate it in Keras and pass the image through your trained model to get the raw logits output before the last softmax activation layer. This will be used to compare the results in SNL against your software results.

    Now your new model is ready to move on to the next steps.


How to build the Vitis HLS Model

  1. Set up Xilinx licensing. If you are on the SLAC network:

    source convDemo/firmware/setup_env_slac.sh

    Otherwise, you will have to install Vivado/Vitis and install the Xilinx licensing.

  2. Go to the HLS target directory and build the HSL.

    cd convDemo/firmware/shared/SnlConvoExample
make clean

  3. Launch the GUI.

    make gui
  4. On the Flow Navigator, click on "Run C Simulation" to verify that your model structure is correct. Select "Clean Build" and "OK."



  5. You will then see a "processStream_csim.log" file, which will show you the structure of the model. Verify that everything looks correct.



  6. At the end of the csim log, you will see the results of passing your test image through the model (the raw logits before the final softmax activation layer). Check that this matches the software intermediate output of the layer right before the final softmax activation layer in "Convo_Example.ipynb." This verifies that you have successfully transferred your software model to SNL.

  7. Close the GUI and run the following commands in the terminal. This will generate an IP zip file.

    make clean
make

  8. After the process finishes, you will see two lines like this.

    INFO: [HLS 200-111] Finished Command export_design CPU user time: 29.47 seconds. CPU system time: 1.24 seconds. Elapsed time: 34.39 seconds; current allocated memory: 16.152 MB.
/u1/jaeylee/convDemo/firmware/shared/SnlConvoExample/ip/SnlConvoExample-v1.0-20240213234508-jaeylee-560c31c.zip

    Copy the part that begins with "/SnlConvoExample-v1..." to your clipboard. Make sure that the name does not contain "dirty," which means it was made using uncommitted code.

  9. Launch vim to edit the "ruckus.tcl" file under the "SnlConvoExample" directory.

    vim ruckus.tcl

    Replace the path that begins with "/SnlConvoExample-v1..." with the copied path in your clipboard. Save the modified file.

  10. Now, go to "convDemo/firmware/python/snl_Convo_Example/_Application.py" and create the correct variables that correspond to the weights and biases of your model. To get the correct offset and number fields, go to "convDemo/firmware/shared/SnlConvoExample/ip" and unzip the ip zip file by running

    unzip [IP_ZIP_FILE].zip

    Then, go to "convDemo/firmware/shared/SnlConvoExample/ip/drivers/processStream_v1_0/src" and run

    cat xprocessstream_hw.h

    The output will show the offset and number for each variable. Create your variables according to these values. *Note: "number" stands for the "DEPTH" values in the output.


  11. In the getWeights() method, load the correct .h5 file for your model and write the correct weights and biases to hardware. Ensure that the format of sending the weights matches up with the example. For this model, it looks like this:

How to build the Vivado firmware

  1. Go to 

    convDemo/firmware/targets

    and run "ls."

    This will output a list of available FPGAs. For example,

    ConvExampleKcu1500  shared_version.mk  SnlConvAlveoU200  SnlLenetKcu105  SnlLenetKcu105HlsBypass

    After you select an FPGA (in this case, I selected SnlConvAlveoU200), "cd" into it and run

    make clean
make

    This will take a long time. After it finishes running, run

    make gui

    This will bring up the Vivado GUI.

  2. Under the "Project Summary" tab, you can view the resource utilization graph, power consumption, and other information regarding the implementation. In the left panel, expand the "Open Implemented Design" and click on any reports to view the information in more detail. Opening the implementation reports provides more accurate data than the synthesis reports.


How to reprogram PCIe firmware via Rogue

  1. Setup the Rogue environment.

    cd convDemo/software
source setup_env_slac.sh

  2. Run the PCIe firmware update script with the path equal to "convDemo/firmware/targets/<Name of FPGA>/images"

    python scripts/updatePcieFpga.py --path .../convDemo/firmware/targets/<Name of FPGA>/images
  3. Select the image number (in this case, 0) and make sure that the FwTarget is set to the correct FPGA (the names might be different, but if it is the correct FPGA, enter "y").

  4. Reboot the computer.

    sudo reboot

How to run the software to load weights and images from HDF5 model 

Open "software/NoteBook/DataProcessing.ipynb."

After changing the paths to be correct, you will be able to run the cells and get the accuracy of your model compared to the ground truth labels at the end. You will also see a count of all labels your model generated, and you can compare this count to the count you got in "Convo_Example.ipynb" from "convDemo/firmware/python/snl_Convo_example" to see if your FPGA output matches your software output.

Note that the predicted class counter for the FPGA output (left) matches the predicted class counter for the software output (right), meaning that the model was successfully moved to the FPGA.

  • No labels