Content
- Software Environment
- Data Generation
- Codewords Generation
- CPU
- GPU
- Support Vector Machine
- Convolutional Neural Network (pending)
1.Software Environment
The required software are listed below:
- python 2.7
- ipython
- scipy
- numpy
- matplotlib
- pillow
- h5py
- scikit-learn
- tensorflow
It is recommanded to use linux or mac osx operating sytem. I use ubuntu 16.04. If you use different version of operating system, the command lines below may not work.
To set up software environment, you might need additional packages:
- pip
- git
You can get all these packages installed easily via command line "sudo apt-get install ...(name of the packages) " or "sudo pip install ... (package names)". It may also help to protect your system from unexpected mistakes by setting up a virtual environments. The simple tutorial for setting up a virtual environment can be found in the link:
http://docs.python-guide.org/en/latest/dev/virtualenvs/
2.Data Generation
Currently we don't have many images available for training. Thus I use two ways to generate new images.
- For images with a beam, first shift the beam spot to the center of the image, then rotate the image around the center, then cut out a small patch of images that contains the beam. Then arbitrarily chose a position paste the small patch to the new position. Cut out the small patch of image that is covered by the beam spot, and paste it to the original position of the beam spot. This proccedure aims to minimize the influence of our manipulation. It should be notice that to do this, one have to choose a large enough patch. Otherwise, it may not be able to cover the original beam spot.
- For all images, one can merely shift the whole images one or two pixels along x or y axis. In this way, one can create 24 times more data.
3.Codewords Generation
To generate codewords, one can do with CPU or GPU. But it is one hundred times faster to run on GPU.The original program come from blog:
https://www.cs.toronto.edu/~frossard/post/vgg16/
After you follow the steps described there and successfully run the prediction for the example animal, you only need to make minor modifications to get it produce your own codewords.
- CPU
- Change mean value
find the lines
- Change mean value
# zero-mean input
with tf.name_scope('preprocess') as scope:
mean = tf.constant([123.68, 116.779, 103.939], dtype=tf.float32, shape=[1, 1, 1, 3], name='img_mean')
images = self.imgs-mean
Replace the 'mean = tf.constant([123.68, 116.779, 103.939]' part with 'mean = tf.constant([0., 0., 0.]' . Because our data are not included in the ImageNet, so we have to do mean subtraction by hand. Our data mean are also different than their general mean.
b. Load target Image with zeros mean
Find line
img1 = imread('laska.png', mode='RGB')
img1 = imresize(img1, (224, 224))
Replace them with
middle_result_1=np.zeros((224,224,3))
mean_values_image[i]=np.mean(vcc_original_images[i,::,::])
img=vcc_original_images[i,::,::]-mean_values_image[i]
img=middle_result_1+img[::,::,np.newaxis]
img1=img.astype(np.float32)
In this way you successfully loaded your data in to a suitable form with zero mean. Of course there are other ways to do this, and this is not what I really write in the code due to the context. Real code are in file 'vgg16_modified_1.py'
c. Extract the codewords
Replace line
prob = sess.run(vgg.probs, feed_dict={vgg.imgs: [img1]})[0]
preds = (np.argsort(prob)[::-1])[0:5]
With
codewords_1[i,::] = sess.run(vgg.fc1, feed_dict={vgg.imgs: [img1]})[0]
codewords_2[i,::] = sess.run(vgg.fc2, feed_dict={vgg.imgs: [img1]})[0]
After this, codewords_1 and codewords_2 becomes will contain the codewords for the target images.
- GPU
GPU works in a similar way, you only have to make an extra change. Replace 'sess = tf.Session()' withwith tf.device('/gpu:0'):
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
config.allow_soft_placement = True
with tf.Session(config=config) as sess:
You will be able to run the codewords generation program on GPU0. The aim of this part is to constrain the program to a single GPU to improve the memory usage efficiency.
4.Support Vector Regression
For an elementary tutorial for support vector machine, please refer to the first three Notes on webpage: http://cs229.stanford.edu/materials.html
For a tutorial of doing support vector regression, please refer to the article on webpage: https://alex.smola.org/papers/2004/SmoSch04.pdf
Certainly you can implement the support vector machine youself. But it is kind of time consuming. So you can refer to the package: scikit-learn. It is very user-friendly. However, it is designed to do computation on CPU and it can be extremely slow when you have a moderate data set. So perhaps in the future, the next student on this project may want to write the SVR program with tensorflow, and use GPU to accelerate the computation.
Currently I use function
svr_rbf=SVR(kernel='rbf', C=1e3, gamma=0.1)
pca=PCA(n_components=num_pca)
to construct the model and reduce the dimension of the feature space. But I have not tuned the parameters carefully. These parameters come from the demonstration example of these function.
5. Convolutional Neural Network (pending)