2. Download and unzip
wget https://www.python.org/ftp/python/3.7.1/Python-3.7.1.tar.xz
# Download
wget https://www.python.org/ftp/python/3.7.1/Python-3.7.1.tar.xz
# Unzip and compile.
tar -xvJf Python-3.7.1.tar.xz
cd Python-3.7.1
3. Compile and install
. /configure prefix=/usr/local/python3
make && make install
# After compiling, create a softlink file to the executable path.
ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3
# We can clear the previously compiled executables and configuration files && Clear all generated files:.
make clean && make distclean
bug: Failed to use pip command
2.1 Error messages
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting virtualenv
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)’: /simple/virtualenv/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)’: /simple/virtualenv/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)’: /simple/virtualenv/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)’: /simple/virtualenv/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)’: /simple/virtualenv/
Could not fetch URL https://pypi.org/simple/virtualenv/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host=’pypi.org’, port=443): Max retries exceeded with url: /simple/virtualenv/ (Caused by SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)) – skipping
Could not find a version that satisfies the requirement virtualenv (from versions: )
No matching distribution found for virtualenv
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host=’pypi.org’, port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)) – skipping
2.2 Reason
System version centos6.5, where openssl version is OpenSSL 1.0.1e-fips 11 Feb 2013, and python3.7 needs openssl version 1.0.2 or 1.1.x, need to upgrade openssl and recompile python3.7.0. The version of openssl installed by yum is relatively low.
2.3 Upgrade openssl
# 1. download openssl
wget https://www.openssl.org/source/openssl-1.1.1a.tar.gz
tar -zxvf openssl-1.1.1a.tar.gz
cd openssl-1.1.1a
# 2.Compile and install
. /config –prefix=/usr/local/openssl no-zlib #no-zlib
make
make install
# 3.Back up the original configuration
mv /usr/bin/openssl /usr/bin/openssl.bak
mv /usr/include/openssl/ /usr/include/openssl.bak
# 4. New version configuration
ln -s /usr/local/openssl/include/openssl /usr/include/openssl
ln -s /usr/local/openssl/lib/libssl.so.1.1 /usr/local/lib64/libssl.so
ln -s /usr/local/openssl/bin/openssl /usr/bin/openssl
# 5. Modify the system configuration
## Write the search path of the openssl library file
echo “/usr/local/openssl/lib” >> /etc/ld.so.conf
## To make the modified /etc/ld.so.conf take effect
ldconfig -v
# 6. Check the openssl version
openssl version
openssl version note:
/usr/local/openssl/bin/openssl: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
If your libssl.so.1.1 file is under /usr/local/openssl/lib/, you can do this
Today, when I write a simple Python class definition code, I encountered the problem of reporting an error: typeerror: drive() takes 2 positional arguments but 3 were given
The code is as follows
class Car:
speed = 0
def drive(self,distance):
time = distance/self.speed
print(time)
bike = Car()
bike.speed=60
bike.drive(60,80)
After investigation, it was found that it was the self parameter in the def drive (self, distance) method in the class definition
Now let’s take a brief look at the basic information of self in Python
self , which means that the created class instance itself and the method itself can bind various attributes to self, because self points to the created instance itself. When creating an instance, you can’t pass in empty parameters. You must pass in parameters that match the method, but you don’t need to pass in self. The Python interpreter will pass in instance variables by itself
so there are two solutions
method 1: transfer only one parameter. If you want to transfer two parameters, look at method 2
class Car:
speed = 0
def drive(self,distance):
time = distance/self.speed
print(time)
bike = Car()
bike.speed=60
bike.drive(80)
Method 2:
class Car:
speed = 0
def drive(self,distance,speed):
time = distance/speed
print(time)
bike = Car()
bike.drive(80,50)
Cause analysis: in Windows system, you can use \, but in Python string, \ \ has the meaning of escape. For example, \ \ T represents tab, and \ \% n represents newline. Therefore, you need to take some measures to prevent \ \% from being interpreted as escape character. There are currently three solutions
1. Add r in front of the path to keep the original value of the character
When creating canvas with Python and importing images on the canvas, an error occurs_ tkinter.TclError: couldn’t recognize data in image file “F:\Python\test\a.gif””
Tkinter can only load GIF images, that is. GIF extension of the image file, want to display other types of images, such as PNG or JPG, need to use other modules
def canvas_test():
import tkinter
window = tkinter.Tk()
window.geometry('600x400')
window.title('This is Canvas')
#Create a canvas of 550 * 300
canvas = tkinter.Canvas(window, bg='green', width=550, height=300)
# Create the image on the canvas and place the imported image
image_file = tkinter.PhotoImage(file="F:\\Python\\test\\a.gif")
image = canvas.create_image(300, 10, anchor='n', image=image_file)
canvas.pack()
window.mainloop()
Looking for a solution on the Internet, I learned that changing the image suffix can’t change the image format（ Online reference: https://stackoverflow.com/questions/28740462/tkinter-couldnt-recognize-data-in-image-file ）
So, search Baidu again for a GIF image, download it and name it c.gif (or d.jpg). As long as you save the image in GIF image format, run the following code:
def canvas_test():
import tkinter
window = tkinter.Tk()
window.geometry('600x400')
window.title('This is Canvas')
#Create a canvas of 550 * 300
canvas = tkinter.Canvas(window, bg='green', width=550, height=300)
# Create the image on the canvas and place the imported image
#image_file = tkinter.PhotoImage(file="F:\\gao\\Python\\test\\c.gif")
image_file = tkinter.PhotoImage(file="F:\\gao\\Python\\test\\d.jpg")
image = canvas.create_image(300, 10, anchor='n', image=image_file)
canvas.pack()
window.mainloop()
The code runs normally, the picture is displayed normally, only the static picture is displayed
photo image only depends on the type of the image itself, and has nothing to do with the suffix of the image name
The execution order of return statement in try except else finally in Python
Python version: 3.7
First, let’s take a look at the example of try except else finally
def exe_try():
try:
print("start")
raise KeyError
return 1
except KeyError as e:
print("key error")
return 2
else:
print("other")
return 3
finally:
print("finally")
# return 4
result = exe_try()
print(result)
# You can see that the result of its execution is:
# start
# key error
# finally
# 4
# exe_try function catches the KeyError exception, but why does it not return 2 but 4?
# Here is the use of the return statement in try except finally:
# If there is a return statement in finally, it returns the return statement in finally,
# If finally does not have a return statement, it returns the return statement from the previous call
# Execution process:
# When the exe_try function is executed, it prints start, and then encounters a KeyError exception.
# Then the exception is caught and the code block in except is executed: the key error is printed, and return 2 is executed
# But this return 2 does not return directly, but put it on the stack
# Then it goes on to the next block in finally: prints finally, and executes return 4
# Here it still doesn't return 4 directly, but puts it on the stack as well
# At this point the entire try except else finally are executed, it will take out the return at the top of the stack and return
# And the return in finally is put on the stack afterwards, so it's on the top of the stack, and returns 4.
# If there is no return statement in finally, it will return:
# start
# key error
# finally
# 2
Typeerror: super() takes at least 1 argument (0 given) error occurred when writing inheritance subclass
Source code (perfect to run in python3)
class Example(QWidget):
def __init__(self):
super().__init__()
self.initUI() #Interface drawing is given to the InitUi method
The reason is super ()__ init__() Function is supported in python3, which is correct, but it will cause problems in python2
If you want to inherit the construction method of the parent class in Python 2, you need to pass in the super parameter: Super (example, self)__ init__()；
In python2, it should be written as follows:
class Example(QWidget):
def __init__(self):
super(Example,self).__init__()
self.initUI() #Interface drawing is given to the InitUi method
Artificial intelligence, deep learning, convolutional neural network, reinforcement learning. These are revolutionary advances in the field of machine learning, which make many impossible tasks become possible. Despite these advantages, there are also some disadvantages and limitations. For example, because neural networks need a large number of training sets, it is easy to lead to over fitting. These algorithms are usually designed for specific tasks, and their capabilities can not be well transplanted to other tasks
One of the main challenges of computer vision is the amount of data involved: an image is usually represented as a matrix with millions of elements, while video contains thousands of such images. In addition, noise often appears in this kind of data. Therefore, unsupervised learning method which can reduce the dimension of data is a necessary magic weapon to improve many algorithms.
In view of this, tensor decomposition is very useful in the application of high-dimensional data. Using Python to implement tensor decomposition to analyze video can get important information of data, which can be used as preprocessing of other methods
High dimensional data
High dimensional data analysis involves a set of problems, one of which is that the number of features is larger than the number of data. In many applications (such as regression), this can lead to speed and model learning problems, such as over fitting or even unable to generate models. This is common in computer vision, materials science and even business, because too much data is captured on the Internet
One of the solutions is to find the low dimensional representation of the data and use it as the training observation in the model, because dimension reduction can alleviate the above problems. The lower dimensional space usually contains most of the information of the original data, so the reduced dimensional data is enough to replace the original data. Spline, regularization and tensor decomposition are examples of this method. Let’s study the latter method and practice one of its applications.
Project 3D data onto 2D plane, image source: May Morrison.
Mathematical concepts
The core concept of this paper is tensor
The number is a 0-dimensional tensor
A vector is a one-dimensional tensor
A matrix is a two-dimensional tensor
in addition, it directly refers to the dimension of tensor
This data structure is particularly useful for storing images or videos. In the traditional
RGB model
In [1] , a single image can be represented by a three-dimensional tensor
each color channel (red, green, blue) has its own matrix, and the value of a given pixel in the matrix encodes the intensity of the color channel
Each pixel has (x, y) coordinates in the matrix, and the size of the matrix depends on the resolution of the image
The representation of 3D tensor. For images,
, and
And
It represents the resolution of the image. Image sources: Kolda, Tamara g. and Brett W. Bader.
Further, a video is just a series of frames, each of which is an image. It makes it difficult to visualize, but it can be stored in 4D tensor: three dimensions are used to store a single frame, and the fourth dimension is used to encode the passage of time
Each slice is a 3D tensor representing a certain frame, and there are multiple slices along the time axis. Image source: kamran paynabar.
To be more specific, let’s take a 60 second video with 60 frames per second (frames per second) and a resolution of 800×600 as an example. The video can be stored in 800x600x3600 tensor. So it’s going to have five billion elements! That’s too much for building a reliable model. That’s why we need tensor decomposition
There are many literatures about tensor decomposition, and I recommend Kolda and balder’s
overview
[2]。 In particular, Tucker decomposition has many applications, such as tensor regression
Objectives
[3] or
forecast
[4] variable. The key point is that it allows the extraction of a kernel tensor, a compressed version of the original data. If this reminds you of PCA, that’s right: one of the steps in Tucker decomposition is actually an extension of SVD, that is, SVD
high order singular value decomposition .
Existing algorithms allow the extraction of kernel tensor and decomposition matrix (not used in our application). The super parameter is rank n. The main idea is that the higher the n value is, the more accurate the decomposition is. Rank n also determines the size of the kernel tensor. If n is small, the reconstructed tensor may not exactly match the original tensor, but the lower the data dimension is, the trade-off depends on the current application.
A. B and C are decomposition matrices, and G is kernel tensor whose dimension is specified by n. Image sources: Kolda, Tamara g. and Brett W. Bader.
It is very useful to extract this kernel tensor, which will be seen in the following practical application examples.
Application
As an example of a toy, I captured three 10 second videos on my mobile phone:
My favorite cafe terrace
Parking lot
Cars on the freeway during the afternoon commute
I uploaded them and notbook code to GitHub. The main purpose is to determine whether we can strictly rank potential video pairs according to the similarity under the premise that parking lots and commuting videos are the most similar.
Before analyzing, use opencv Python library to load and process this data. The steps are as follows,
Create videocapture objects and extract the number of frames for each object
I use a shorter video to truncate the other two for better comparison
After completing these steps, we get three 50x1080x1920x3 tensors.
Results
To determine how similar these videos are, we can rank them. The L2 norm of the difference between two tensors is a common measure of similarity. The smaller the value is, the higher the similarity is. In mathematics, the norm of tensor can be
Each
Represents a given dimension,
Is a given element.
Therefore, the norm of difference is similar to Euclidean distance
The result of this operation using complete tensor is not satisfactory.
Not only is there no clear ranking between the two videos, but the parking lot and terrace videos seem to be the most similar, in sharp contrast to the original assumption
Well, let’s see if Tucker decomposition can improve the results.
Tensorly library makes it relatively easy to decompose tensors, although it’s a bit slow: all we need is tensors and their rank n. Although AIC criterion is a common method to find the optimal value of this parameter, it is not necessary to achieve the optimal value in this particular case, because the purpose is to compare. We need all three variables to have the same rank n. Therefore, we choose n-rank = [2, 2, 2, 2], which is a good tradeoff between accuracy and speed. By the way, n-rank = [5,5,5,5] exceeds the function of LAPACK (low level linear algebra package), which also shows that these methods are computationally expensive.
After extracting the kernel tensor, the same comparison can be made.
These results make sense: Although terrace videos are different from parking and commuting videos, the latter two videos are closer to an order of magnitude.
Conclusion
In this article, I show how unsupervised learning methods provide insight into data. Only when the dimension is reduced by Tucker decomposition to extract the kernel tensor from the video, can the comparison be meaningful. We confirm that the parking lot is the most similar to the commuter video
As video becomes more and more popular data source, this technology has many potential applications. The first thought (due to my passion for TV and how the video streaming service uses data) is to improve the existing recommendation system by checking the similarities between some key scenes of the trailer or movie/TV program. The second is materials science, in which the heated metal can be classified according to the similarity between the infrared video and the benchmark. In order to make these methods fully scalable, we should solve the computational cost: on my computer, Tucker decomposition speed is very slow, although only three 10s small videos. Parallelization is a potential way to speed up processing.
In addition to these direct applications, the technique can also be combined with some of the methods introduced in the introduction. Using the core tensor instead of the whole image or video as the training points in the neural network can help to solve the over fitting problem and speed up the training speed, so as to enhance the method by solving these two main problems.
In this paper, through an example, let us initially understand the tensor decomposition tool, and master the actual usage through the code. Please click here [5] for children’s shoes that need to download this code to consolidate understanding. If you want to have a deep understanding of tensor decomposition, please read the references. The detailed interpretation of the theoretical knowledge of tensor decomposition will be introduced later
References
[1]
RGB model https://en.wikipedia.org/wiki/RGB_ color_ model
def gcd(x, y):
(x, y) = (y, x) if x > y else (x, y)
for factor in range(x, 0, -1):
if x % factor == 0 and y % factor == 0:
return factor
if __name__ == '__main__':
num1 = int(input("Please enter a positive integer: "))
num2 = int(input("Please enter a positive integer. "))
print("i" + gcd(num1, num2))
code error
PS G:\vscode_worksapce\vscode_python_workspace> python -u "g:\vscode_worksapce\vscode_python_workspace\test1.py"
Please enter a positive integer: 3
Please enter a positive integer: 9
Traceback (most recent call last):
File "g:\vscode_worksapce\vscode_python_workspace\test1.py", line 11, in <module>
print("i" + gcd(num1, num2))
TypeError: can only concatenate str (not "int") to str
causes and solutions
reference the blog https://blog.csdn.net/feng2147685/article/details/86494905, and https://www.cnblogs.com/Jimc/p/9606112.html
is a Python string concatenation problem;
if __name__ == '__main__':
num1 = int(input("Please enter a positive integer: "))
num2 = int(input("Please enter a positive integer. "))
print("i%d" % (gcd(num1, num2)) + "hahahha")
#Receive request data
def search(request):
request.encoding = 'utf-8'
if 'q' in request.GET:
message = 'You searched for: ' +request.GET['q'].encode('utf-8')
else:
message = 'You submitted an empty form'
return HttpResponse(message)
code marked red position, we can see that encode function is used to transcode, because encode transcode returns data of type Bytes, can not be directly added with data of type STR.
Since the request request has been transcoded in the first sentence of the function, we remove the following encode function here, and the error can be solved.
The updated code is:
#Receive request data
def search(request):
request.encoding = 'utf-8'
if 'q' in request.GET:
message = 'You searched for: ' +request.GET['q']
else:
message = 'You submitted an empty form'
return HttpResponse(message)
recently wrote the program using urllib. Request urlopen to read pages of text files, the results appeared a TypeError, problems and solving methods will now be recorded as follows, hope to help those encountered similar problems:
uses the Python version 3.5.2
1 # -*- coding: UTF-8 -*-
2 #!/usr/bin/python
3 # Filename: urllib.py
4 import random
5 from urllib.request import urlopen
6 import sys
7
8 WORD_URL = "http://learncodethehardway.org/words.txt"
9 WORDS = []
10
11 PHRASES = {
12 " class %%%(%%%):":
13 "Make a class named %%% that is-a %%%.",
14 "class %%%(object):\n\tdef__init__(self,***)":
15 "class %%% has-a __init__ that takes self and *** parameters.",
16 "*** = %%%()":
17 "Set *** to an instance of class %%%.",
18 "***.***(@@@)":
19 "From *** get the *** function, and call it with parameters self,@@@.",
20 "***.*** = '***'":
21 "From *** get the *** attribute and set it to '***'."
22 }
23
24 PHRASES_FIRST = False
25 if len(sys.argv) == 2 and sys.argv[1] == "english":
26 PHRASES_FIRST = True
27
28 for word in urlopen(WORD_URL).readlines():
29 WORDS.append(word.strip())
30
31 def convert(snippet, phrase):
32 class_names = [w.capitalize() for w in
33 random.sample(WORDS, snippet.count("%%%"))]
34 other_names = random.sample(WORDS, snippet.count("***"))
35 results = []
36 param_names = []
37
38 for i in range(0, snippet.count("@@@")):
39 param_count = random.randint(1,3)
40 param_names.append(', '.join(random.sample(WORDS, param_count)))
41 #param_names.append(', '.join('%s' %id for id in random.sample(WORDS, param_count)))
42
43 for sentence in snippet, phrase:
44 result = sentence[:]
45 print(result)
46
47 for word in class_names:
48 result = result.replace("%%%", word, 1)
49
50 for word in other_names:
51 result = result.replace("***",word, 1)
52
53 for word in param_names:
54 result = result.replace("@@@", word, 1)
55
56 results.append(result)
57
58 return results
59
60 try:
61 while True:
62 snippets = list(PHRASES.keys())
63 random.shuffle(snippets)
64
65 for snippet in snippets:
66 phrase = PHRASES[snippet]
67 question, answer = convert(snippet, phrase)
68 if PHRASES_FIRST:
69 question, answer = answer,question
70
71 print (question)
72
73 input ("> ")
74 print ("ANSWER: %s\n\n" % answer)
75 except EOFError:
76 print ("\nBye!")
= TypeError: Can’t convert ‘bytes’ object to STR IMPLICITLY
after the access to information found that urlopen () returns a bytes object, if you need to string operations, he need to explicitly convert it into a string, can appear otherwise the above problems.
we when reading web content directly convert them to encoding to utf-8, can continue to use. Change line 29 of the above code to
