Tag Archives: tensorflow

Weighted cross entropy loss function: tf.nn.weighted_cross_entropy_with_logits

tf.nn.weighted_cross_entropy_with_logits Function

tf.nn.weighted_cross_entropy_with_logits(
  targets,
  logits,
  pos_weight,
  name=None
)

Defined in: tensorflow/Python/OPS/NN_ impl.py 。

The weighted cross entropy was calculated.

0 Similar to sigmoid?Cross?Entropy?With?Logits () , except POS?U weight It allows people to weigh recall rate and accuracy by weighting up or down the cost of positive error relative to negative error.

0 Generally, the cross entropy cost is defined as:

targets * -log(sigmoid(logits)) +
  (1 - targets) * -log(1 - sigmoid(logits))

Value POS?U weights &> 1 The false negative count is reduced and the recall rate is increased. Instead, set POS?U weights & lt; 1 It can reduce false positive count and improve accuracy. You can see from the following It is introduced as the multiplication coefficient of the positive target term in the loss expression

targets * -log(sigmoid(logits)) * pos_weight +
  (1 - targets) * -log(1 - sigmoid(logits))

For simplicity, let x = Logits, z = targets, q = POS?U weight . The losses are:

 qz * -log(sigmoid(x)) + (1 - z) * -log(1 - sigmoid(x))
= qz * -log(1/(1 + exp(-x))) + (1 - z) * -log(exp(-x)/(1 + exp(-x)))
= qz * log(1 + exp(-x)) + (1 - z) * (-log(exp(-x)) + log(1 + exp(-x)))
= qz * log(1 + exp(-x)) + (1 - z) * (x + log(1 + exp(-x))
= (1 - z) * x + (qz + 1 - z) * log(1 + exp(-x))
= (1 - z) * x + (1 + (q - 1) * z) * log(1 + exp(-x))

Set L = (1 + (Q – 1) * z) to ensure stability and avoid overflow

(1 - z) * x + l * (log(1 + exp(-abs(x))) + max(-x, 0))

Logits and targets must have the same type and shape.

Parameters:

Targets: a tensor with the same type and shape as Logits.

Logits: a tensor of type float32 or float64.

pos_ Weight: the coefficient used in the positive sample.

Name: the name of the operation (optional).

Return:

Tensors with the same shape as Logits with component weighted logic loss.

Possible exceptions:

Valueerror: if Logits and targets do not have the same shape.

Detailed explanation of yolo2 — yolo9000, better, faster, stronger

Introduction

The full name of yolov2’s paper is yolov000: better, faster, stronger, which won CVPR 2017 best paper honorable meaning. In this paper, the author first proposes an improved yolov2 based on yolov1, and then proposes a joint training method of detection and classification. Using this joint training method, the yolov000 model is trained on coco detection data set and Imagenet classification data set, which can detect more than 9000 kinds of objects. Therefore, this article actually contains two models: yolov2 and yolo9000, but the latter is based on the former, and the main structure of the two models is consistent. Compared with yoov1, yoov2 has made many improvements, which also makes the map of yoov2 significantly improved. Moreover, the speed of yoov2 is still very fast, maintaining its advantages as one stage method. The comparison between yoov2 and fast r-cnn, SSD and other models is shown in Figure 1. This paper will first introduce the improvement strategy of yoolv2, and give the implementation process of tensorflow of yoolv2, and then introduce the training method of yool9000.

paper address : yolo9000: better, faster, stronger

Improvement strategy of yolov2

Although the detection speed of yolov1 is very fast, its detection accuracy is not as good as that of r-cnn system. Yolov1 is not accurate enough in localization and recall. Yolov2 proposes several improvement strategies to improve the positioning accuracy and recall rate of Yolo model, so as to improve map. Yolov2 follows one principle in the improvement: maintain the detection speed, which is also a major advantage of Yolo model. The improvement strategy of yolov2 is shown in Figure 2. It can be seen that most of the improvement methods can significantly improve the map of the model. The improvement strategies are as follows:

for details, please refer to:


≪ P &> & lt; Center &> * * [yolov2 of object detection]( https://zhuanlan.zhihu.com/p/42861239 )**</center&></p&> —————————————————————————————————————————–

≪ font color = Red &> I feel that when I read some good papers, I can learn on the basis of some big men, and think about whether these views are correct, which is conducive to learning the ideas in classic papers faster and easier. </font&>

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor ‘…

I found a problem when you use tensorboard for visualization: if you define

MERGED = tf.summary.merge_ all();

After this operation, if you use it alone SESS.run ([merge]), the above error will be reported

At this point, you should work with other pigs instead SESS.run ([train, merged]), this error will not be reported again after the change

It’s hard for me to explain the specific reasons. Before, I checked this error for a long time and found some solutions, but none of them solved my problem

https://stackoverflow.com/questions/35114376/error-when-computing-summaries-in-tensorflow

Later, I referred to a GitHub program and changed it according to its appearance.

# -*- coding: utf-8 -*-
"""
Created on Wed Oct 31 17:07:38 2018

@author: LiZebin
"""

from __future__ import print_function
import numpy as np
import tensorflow as tf

tf.reset_default_graph()
SESS = tf.Session()

LOGDIR = "logs/"

X = np.arange(0, 1000, 2, dtype=np.float32)
Y = X*2.3+5.6
X_ = tf.placeholder(tf.float32, name="X")
Y_ = tf.placeholder(tf.float32, name="Y")
W = tf.get_variable(name="Weights", shape=[1],
                    dtype=tf.float32, initializer=tf.random_normal_initializer())
B = tf.get_variable(name="bias", shape=[1],
                    dtype=tf.float32, initializer=tf.random_normal_initializer())
PRED = W*X_+B
LOSS = tf.reduce_mean(tf.square(Y_-PRED))
tf.summary.scalar("Loss", LOSS)
TRAIN = tf.train.GradientDescentOptimizer(learning_rate=0.0000001).minimize(LOSS)
WRITER = tf.summary.FileWriter(LOGDIR, SESS.graph)
MERGED = tf.summary.merge_all()

SESS.run(tf.global_variables_initializer())
for step in range(20000):
    c1, c2, loss, RS, _ = SESS.run([W, B, LOSS, MERGED, TRAIN], feed_dict={X_:X, Y_:Y})   ####If you write RS=SESS.run(MERGED) after it alone, it will report the same error as before
    WRITER.add_summary(RS)
    if step%500 == 0:
        temp = "c1=%s, c2=%s, loss=%s"%(c1, c2, loss)
        print(temp)
SESS.close()

:Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

1. Problems

Write a simple single-layer neural network run MNIST handwritten numeral set, the result of each fit will appear dead kernel

For many dead kernels, first of all, don’t rush to search the Internet to find out how to solve the problem. Because the reasons are different, you should go to the bash terminal of notebook to check the error information, which is as follows:

It’s better to see a blog explaining this problem:

https://blog.csdn.net/qq_ 41185868/article/details/79127838#comments

According to the error report, first understand what AVX is

AVX is a kind of extended instruction, which needs the cooperation of CPU and protocol, but my tensorflow doesn’t support it.

2. Solutions

Maybe it’s because the Mac version of tensorflow is lower (compared with windows, it can’t be upgraded disorderly), so how to solve it?

It’s OK. I also want to find out what’s behind this, but it seems that the more I look at it, the more complicated it becomes. -It’s better to ignore the warning and run the code