Neural Networks in Python - Revision history

Erica: /* A Simple Feedforward Classifier Network */

2019-04-26T14:35:09Z

A Simple Feedforward Classifier Network

← Older revision		Revision as of 10:35, 26 April 2019
Line 54:		Line 54:

	#noise layer to mitigate overfitting		#noise layer to mitigate overfitting
	noise_1=keras.layers.noise.GaussianNoise(0.05, input_shape=(inodes,), name='dense_1_noise')(~~main_input~~)		noise_1=keras.layers.noise.GaussianNoise(0.05, input_shape=(inodes,), name='dense_1_noise')(dropout_1)

	#first hidden layer		#first hidden layer

Chris: /* A Simple Feedforward Classifier Network */

2019-03-08T16:59:39Z

A Simple Feedforward Classifier Network

← Older revision		Revision as of 12:59, 8 March 2019
Line 7:		Line 7:
	=A Simple Feedforward Classifier Network=		=A Simple Feedforward Classifier Network=
	Keras has defined network templates, but the only one that I've seen well-documented is the '''forward'' model. Fortunately, the forward model is well-suited to the classification task: it takes a set of input values, feeds the values forward over a set of layers, and checks the output values in a set of binary classification units. If the output decision is strictly binary (A vs B), then one output unit will suffice (since it's a zero-sum decision). If 3+ output categories are possible, then there needs to be an output unit for each possible category.		Keras has defined network templates, but the only one that I've seen well-documented is the '''forward'' model. Fortunately, the forward model is well-suited to the classification task: it takes a set of input values, feeds the values forward over a set of layers, and checks the output values in a set of binary classification units. If the output decision is strictly binary (A vs B), then one output unit will suffice (since it's a zero-sum decision). If 3+ output categories are possible, then there needs to be an output unit for each possible category.

			import numpy as np
			import keras
			import time
			import scipy.io as sio
			from keras.layers import Input,Dense,Dropout
			from keras.models import Model, Sequential, load_model
			from keras.regularizers import l1
			from keras.optimizers import Adadelta
			from sklearn.model_selection import StratifiedKFold as SKF
			from sklearn.model_selection import ShuffleSplit as SS

			import read_activations as reader

			seed=int(time.time())
			np.random.seed(seed)

			trainfile='av_hi.csv'
			traindata=np.genfromtxt(trainfile, delimiter=',')
			testdata=(traindata)

			x_train=traindata[:,0:-1]
			y_cat_code=testdata[:,-1]
			#this next line converts the category codes of 1,2,3 into 3 separate output units so that
			#a code of 1 maps to {1,0,0}, 2 maps to {0,1,0} and 3 maps to {0,0,1}
			y_cat_train=keras.utils.to_categorical(y_cat_code, num_classes=3)

			encoding_dim1=96
			encoding_dim2=32
			input_trials, inodes = x_train.shape

			foldnumber=0
			fp=open('perflog.txt', 'w')
			hfp=open('cat_traininghist.txt', 'w')

			kfold = SKF(n_splits=8, shuffle=True, random_state=seed)
			for train, test in kfold.split(x_train, y_cat_code):
			foldnumber=foldnumber+1
			ada=keras.optimizers.Adadelta(lr=1, rho=0.9, decay=1e-6)

			#input layer: a series of 1000 floats from 0.0 to 1.0
			main_input=Input(shape=(inodes,), dtype='float32', name='input')

			#first dropout layer, initially, lets randomly drop 20% of values
			dropout_1=Dropout(0.2, input_shape=(inodes,), name='dense_1_drop')(main_input)

			#noise layer to mitigate overfitting
			noise_1=keras.layers.noise.GaussianNoise(0.05, input_shape=(inodes,), name='dense_1_noise')(main_input)

			#first hidden layer
			dense_1=Dense(encoding_dim1, activation='relu', input_dim=inodes,
			name='dense_1', kernel_regularizer=l1(0.00001))(noise_1)

			#second hidden layer
			dense_2=Dense(encoding_dim2, activation='relu', name='dense_2')(dense_1)
			#category output layer
			cat_output=Dense(3, activation='softmax', name='cat_output')(dense_2)

			#assemble the model
			model=Model(inputs=[main_input], outputs=[cat_output])
			#compile the model
			model.compile(optimizer='adadelta',
			loss=['categorical_crossentropy'],
			metrics=['accuracy'], loss_weights=[1])

			X_train=x_train[train]
			X_test=x_train[test]
			Y_cat_train=y_cat_train[train]
			Y_cat_test=y_cat_train[test]

			xrows, xcols = X_test.shape
			history=model.fit(X_train, [Y_cat_train],
			validation_split=0.2,
			epochs=64,
			batch_size=32,
			shuffle=True,
			verbose=0)
			trainscore = model.evaluate(X_test, [Y_cat_test], verbose=0)
			outstr='%.2f\n' % trainscore[1]
			print outstr
			fp.write(outstr)
			fname='model_' + format(foldnumber, '02') + '.h5'
			model.save(fname) #save the trained model

			#training history
			vals=history.history['val_acc']
			strlist= ['{:.3f}'.format(x) for x in vals]
			hfp.write('\n' + ','.join(strlist))

			print model.summary()
			fp.close()
			hfp.close()

Chris at 16:44, 8 March 2019

2019-03-08T16:44:43Z

← Older revision		Revision as of 12:44, 8 March 2019
Line 5:		Line 5:
	pip install -r requirements.txt		pip install -r requirements.txt
	I've got the current list of dependencies on ubfs/Scripts/Python/requirements.txt		I've got the current list of dependencies on ubfs/Scripts/Python/requirements.txt
			=A Simple Feedforward Classifier Network=
			Keras has defined network templates, but the only one that I've seen well-documented is the '''forward'' model. Fortunately, the forward model is well-suited to the classification task: it takes a set of input values, feeds the values forward over a set of layers, and checks the output values in a set of binary classification units. If the output decision is strictly binary (A vs B), then one output unit will suffice (since it's a zero-sum decision). If 3+ output categories are possible, then there needs to be an output unit for each possible category.

Chris: /* Before You Begin */

2019-03-08T16:39:55Z

Before You Begin

← Older revision		Revision as of 12:39, 8 March 2019
Line 2:		Line 2:
	We've been using the Keras API with the TensorFlow backend for our simulations. Our code often also uses the numPy and SciKit libraries because I often steal code examples that happen to have been coded using those libraries. When you get started, you will want to make sure that all the requisite libraries are installed. Pip (the Python package installer) has a function called <code>freeze</code> that will list all your installed libraries. This list can be dumped to a text file and used to share the list of Python packages you'll need to run any of the code we've developed:		We've been using the Keras API with the TensorFlow backend for our simulations. Our code often also uses the numPy and SciKit libraries because I often steal code examples that happen to have been coded using those libraries. When you get started, you will want to make sure that all the requisite libraries are installed. Pip (the Python package installer) has a function called <code>freeze</code> that will list all your installed libraries. This list can be dumped to a text file and used to share the list of Python packages you'll need to run any of the code we've developed:
	pip freeze > requirements.txt		pip freeze > requirements.txt
	Now any lab member can use '''~~dependencies~~.txt''' to ensure they have the requisite Python packages installed:		Now any lab member can use '''requirements.txt''' to ensure they have the requisite Python packages installed:
	pip install -r requirements.txt		pip install -r requirements.txt
	I've got the current list of dependencies on ubfs/Scripts/Python/requirements.txt		I've got the current list of dependencies on ubfs/Scripts/Python/requirements.txt

Chris: Created page with "=Before You Begin= We've been using the Keras API with the TensorFlow backend for our simulations. Our code often also uses the numPy and SciKit libraries because I often stea..."

2019-03-08T16:39:25Z

Created page with "=Before You Begin= We've been using the Keras API with the TensorFlow backend for our simulations. Our code often also uses the numPy and SciKit libraries because I often stea..."

New page

=Before You Begin=
We've been using the Keras API with the TensorFlow backend for our simulations. Our code often also uses the numPy and SciKit libraries because I often steal code examples that happen to have been coded using those libraries. When you get started, you will want to make sure that all the requisite libraries are installed. Pip (the Python package installer) has a function called <code>freeze</code> that will list all your installed libraries. This list can be dumped to a text file and used to share the list of Python packages you'll need to run any of the code we've developed:
pip freeze > requirements.txt
Now any lab member can use '''dependencies.txt''' to ensure they have the requisite Python packages installed:
pip install -r requirements.txt
I've got the current list of dependencies on ubfs/Scripts/Python/requirements.txt