In the process of back propagation, the neural network needs to calculate the partial derivative of the learning parameter corresponding to each loss. The calculated value is the gradient, which is used to multiply the learning rate to update the learning parameter. It is used through the gradients function in tensorflow

We parse the function prototype according to the official documents

The function prototype and parameters in the official document are as follows:

tf.gradients(
    ys,
    xs,
    grad_ys=None,
    name='gradients',
    colocate_gradients_with_ops=False,
    gate_gradients=False,
    aggregation_method=None,
    stop_gradients=None,
    unconnected_gradients=tf.UnconnectedGradients.NONE
)

Ys and XS are tensors or tensor lists. The function TF. Gradients is used to derive XS in ys. The return value of the derivation is a list, and the length of the list is the same as that of XS

Let’s introduce the usage of functions through examples (this is the example given in Teacher Li Jinhong’s book)

import tensorflow as tf
w1 = tf.Variable([[1,2]])
w2 = tf.Variable([[3,4]])

y = tf.matmul(w1, [[9],[10]])
grads = tf.gradients(y,[w1])

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    gradval = sess.run(grads)
    print(gradval)

Running this code will report an error as follows:

TypeError: Fetch argument None has invalid type <class 'NoneType'>

The reason is that tensorflow gradients are like int type tensorflow gradients that set W1 to float type, such as tf.float32 gards, and tensorflow gradients are generally of float32 type. So we modify the code to change the tensor of integer to floating point

import tensorflow as tf
w1 = tf.Variable([[1.,2.]])
w2 = tf.Variable([[3.,4.]])

y = tf.matmul(w1, [[9.],[10.]])
grads = tf.gradients(y,[w1])

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    gradval = sess.run(grads)
    print(gradval)

The output results are as follows

[array([[ 9., 10.]], dtype=float32)]

In the above example, since y is multiplied by W1 and [[9], [10]], its derivatives are [[9], [10]] (i.e. slope)

Note: if there is no variable requiring partial derivative in the gradient formula, the system will report an error. For example, write grads = TF. Gradients (y, [W1, W2])