Create Classification model using Tensorflow

 Here also we need to follow typical steps in the training of a neural network

  1. Define the network
    • Here we define the deep neural network as having an input layer that takes 784 input using the functional API.
    • Then there are two dense layers each having 64 neurons each.
    • The Output layer is also a dense layer that gives 10 output probabilities relevant to each class.   
    • The activation function for each class is a softmax function. 
    • This model consists of 138 trainable parameters (64+64+10)
# Here we define the model 
# Here we use keras functional API note the passing of inputs
def base_model():
  inputs = tf.keras.Input(shape=(784,), name="fashions") # 28 x 28
  x = tf.keras.layers.Dense(64,activation='relu',name='dense_1')(inputs)
  x = tf.keras.layers.Dense(64,activation='relu',name='dense_2')(x)
  outputs = tf.keras.layers.Dense(10,activation='softmax',name='Prediction_layer')(x)
  model = tf.keras.Model(inputs=inputs,outputs=outputs)
  return model
   
     2. Prepare the training data pipeline
    • In here first, we load the training and testing data.
    • Then by using a function we extract the images from the dataset.
    • The extracted images are reshaped to be compatible with the input layer of the neural network (flatten to a 1D array).
    • Then normalize the pixel values to lie between 0 and 1
# load the data sets available with the tensorflow
train_data, info = tfds.load("fashion_mnist",split="train",with_info=True)
test_data = tfds.load("fashion_mnist",split="test")
# format the data to be accepted by neural network
def format_image(data):
  # extract the images
  image = data["image"]
  # flatten the images
  image = tf.reshape(image,[-1])
  # type casting
  image = tf.cast(image,'float32')
  # Normalize
  image = image / 255.0
  # return the formated data with label
  return image,data['label']
train_data = train_data.map(format_image)
test_data = test_data.map(format_image)
    
    Create test and train batches
batch_size = 64 
# buffer_size = 1024 maens it takes 1024 images and then create the 64 size batches
# if the 50 th image is taken then the 1025 th is become the 50th
train = train_data.shuffle(buffer_size=1024).batch(batch_size)
# just create batches without shuffling
test = test_data.batch(batch_size=batch_size)

    
    3. Specify Loss and Optimizer functions
    • Here we use the sparse categorical loss function.
    • Adam is used as the optimizer.
# define the optimizer as object
optimizer = tf.keras.optimizers.Adam()
# define the loss function as object
loss_object = tf.keras.losses.SparseCategoricalCrossentropy()
    
                Apply the Gradient decent
# get the loss and the partial derivative of the loss
def apply_gradient(optimizer,model,x,y):
  with tf.GradientTape() as tape:
    # take the prediction of the current item 
    logits = model(x)
    # calculate the loss values
    loss_value = loss_object(y_true=y,y_pred=logits)

    # calculate the gradients 
    gradients = tape.gradient(loss_value,model.trainable_weights)
    # pass the current values and the partial derivative to adjust the parameters
    # zip the tuple builder
    # zip -> zip the gradient of weights and bias of a given neuron together
    optimizer.apply_gradients(zip(gradients,model.trainable_weights))

    return logits, loss_value


    4. Train the model to minimize loss using an optimizer
    • In the process of the training process, we perform validation steps additional to the training process to validate the epoch with the test set while training.
    • Validation is significant due to "The validation data set provides an unbiased evaluation of a model fit on the training data set while tuning the model's hyperparameters (e.g. the number of hidden units—layers and layer widths—in a neural network) - Wikipedia"
    • In performing training the division of training set into equal size batches are essential to avoid the skewness of the average that can cause due to uneven batch size at the last. The overall biased will be in favour of the batch having less number of items.
model = base_model()
# iteraterations over the training dataset
epochs = 10
epochs_val_losses,epochs_train_losses = [],[]

for epoch in range(epochs):
  print('start of epoach %d' % (epoch,))

  losses_train = train_data_for_one_epoch()
  train_acc = train_acc_metric.result()

  losses_val = perform_validation()
  val_acc = val_acc_metric.result()

  losses_train_mean = np.mean(losses_train)
  losses_val_mean = np.mean(losses_val)
  epochs_val_losses.append(losses_val_mean)
  epochs_train_losses.append(losses_train_mean)

  print('\n Epoch %s: Train loss: %.4f Validation loss: %.4f, Train Accuracy: %.4f, 
        Validation Accuracy %.4f' % 
        (epoch, float(losses_train_mean),float(losses_val_mean),float(train_acc),
        float(val_acc)) )
  
  # reset to avoid calculating from previous batch
  train_acc_metric.reset_states()
  val_acc_metric.reset_states()

    5. Test the model
# Here we plot the matrics
def plot_metrics(train_metric,validation,metric,title,ylim=4):
  plt.title(title)
  plt.ylim(0,ylim)
  plt.gca().xaxis.set_major_locator(mticker.MultipleLocator(1))
  plt.plot(train_metric,color='blue',label=metric)
  plt.plot(validation,color='green',label='validation '+metric)

plot_metrics(epochs_train_losses,epochs_val_losses,
'Loss','Losses of train & validation',ylim=0.55)

        Perform testing with the test data set

Comments

Post a Comment

Popular posts from this blog

The Fractals, Infinity, Universe and Measurement Error

Image
Infinite division of universe and connection between fractals. The universe is a fascinating and mysterious place, full of wonders and mysteries that we are only beginning to unravel. One of the most intriguing questions that scientists have been trying to answer is whether the universe is infinite or finite and whether it has a simple or complex structure. One way to approach this question is to use the concept of fractals, which are geometric shapes that repeat themselves at different scales, creating patterns that look similar but not identical. Fractals are found in nature, such as in snowflakes, ferns, coastlines, and clouds, but they can also be generated mathematically by applying simple rules repeatedly. Some cosmologists have proposed that the universe itself is a fractal, meaning that it has a self-similar structure at different scales. For example, galaxies are composed of stars, which are composed of planets, which are composed of atoms, which are composed of subatomic part...
Read more

Intro to Deep Learning and Tensorflow Basics

Image
First of all, let's look at what is Machine learning? ML is simply converting physical representations/data into numbers and finding patterns in them. Deep learning is a subset of ML. In this article, I'll use ML and Deep learning interchangeably. To find patterns in the numbers computers use algorithms that are based on probabilistic methods. In conventional programming, we feed the computer with a set of inputs and rules to follow to get the desired output. But in ML we feed the set of inputs and desired outputs to generate the rules. These rules are figured out by an algorithm and used to deal with the unseen inputs to generate intended outputs which are used to generate the rule in the first place. If you can build a simple rule-based system that doesn't required machine learning, do that  - first rule of Google's Machine Learning Handbook It is advisable not to overuse Machine Learning when a rule-based system can fulfil the same functionality. So when to use Deep ...
Read more

Let's Create a Perceptron Using Tensorflow

Image
 There are 5 main steps in training a neural network which is even common for the atomic unit of its perception. Define the network: This could be a lot of layers of neurons. For this case just a perceptron. Prepare the training data: Here we will generate a zero-dimensional tensor with 1000 random values. Define loss and optimizer functions. Optimize use in the training phase to measure and minimize the loss. Train the model with training data to minimize the loss. This is the step we find for what values of weight and bias the training data will fit the model (Here just like y = mx + c ) Validate the model The perception consists of two trainable values namely Weight (W) and Bias (b).  As the loss function, we here going to use the Mean squared error loss. 01 Lets define the Perceptron class # Define the model  class   Perceptron () :    def   __init__ ( self ) :      # initializing the trainable...
Read more