Category Archives: Python

Python bug problems and solutions sorted out in recent days

TypeError: unsupported operand type(s) for ^: ‘float’ and ‘int’

Solution: Changed the multiplication sign ^ to **

https://stackoverflow.com/questions/34258537/python-typeerror-unsupported-operand-types-for-float-and-int

TypeError: type numpy.ndarray doesn’t define __round__ method

Solution: change round() to np.round()

TypeError: ufunc ‘bitwise_xor’ not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ”safe”

Solution: change ^ to **

TypeError: only size-1 arrays can be converted to Python scalars

Solution: change math.exp to np.exp()

Reason: sigma is a numpy matrix, use numpy.exp to handle it

Programming udp sockets in python

UDP sockets

UDP or user datagram protocol is an alternative protocol to its more common counterpart TCP. UDP like TCP is a protocol for packet transfer from 1 host to another, but has some important differences. UDP is a connectionless and non-stream oriented protocol. It means a UDP server just catches incoming packets from any and many hosts without establishing a reliable pipe kind of connection.

In this article we are going to see how to use UDP sockets in python.

Create udp sockets

A udp socket is created like this

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

The SOCK_DGRAM specifies datagram (udp) sockets.

Sending and Receiving

Since udp sockets are non connected sockets, communication is done using the socket functions sendto and recvfrom. These 2 functions dont require the socket to be connected to some peer. They just send and receive directly to and from a given address

Udp server

A udp server has to open a socket and receive incoming data. There is no listen or accept. Here is a quick example

'''
	Simple udp socket server
	Silver Moon ([email protected])
'''

import socket
import sys

HOST = ''	# Symbolic name meaning all available interfaces
PORT = 8888	# Arbitrary non-privileged port

# Datagram (udp) socket
try :
	s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
	print 'Socket created'
except socket.error, msg :
	print 'Failed to create socket. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
	sys.exit()


# Bind socket to local host and port
try:
	s.bind((HOST, PORT))
except socket.error , msg:
	print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
	sys.exit()
	
print 'Socket bind complete'

#now keep talking with the client
while 1:
	# receive data from client (data, addr)
	d = s.recvfrom(1024)
	data = d[0]
	addr = d[1]
	
	if not data: 
		break
	
	reply = 'OK...' + data
	
	s.sendto(reply , addr)
	print 'Message[' + addr[0] + ':' + str(addr[1]) + '] - ' + data.strip()
	
s.close()

The above will program will start a udp server on port 8888. Run the program in a terminal. To test the program open another terminal and use the netcat utility to connect to this server. Here is an example

$ ncat -vv localhost 8888 -u
Ncat: Version 5.21 ( http://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:8888.
hello
OK...hello
how are you
OK...how are you

Netcat can be used to send messages to the udp server and the udp server replies back with “OK…” prefixed to the message.

The server terminal also displays the details about the client

$ python server.py 
Socket created
Socket bind complete
Message[127.0.0.1:46622] - hello
Message[127.0.0.1:46622] - how are you

It is important to note that unlike a tcp server, a udp server can handle multiple clients directly since there is no connection. It can receive from any client and send the reply. No threads, select polling etc is needed like in tcp servers.

Udp client

Now that our server is done, its time to code the udp client. It connects to the udp server just like netcat did above.

'''
	udp socket client
	Silver Moon
'''

import socket	#for sockets
import sys	#for exit

# create dgram udp socket
try:
	s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
except socket.error:
	print 'Failed to create socket'
	sys.exit()

host = 'localhost';
port = 8888;

while(1) :
	msg = raw_input('Enter message to send : ')
	
	try :
		#Set the whole string
		s.sendto(msg, (host, port))
		
		# receive data from client (data, addr)
		d = s.recvfrom(1024)
		reply = d[0]
		addr = d[1]
		
		print 'Server reply : ' + reply
	
	except socket.error, msg:
		print 'Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
		sys.exit()

The client will connect to the server and exchange messages like this

$ python simple_client.py 
Enter message to send : hello
Server reply : OK...hello
Enter message to send : how are you
Server reply : OK...how are you
Enter message to send :

Overall udp protocol is simple to program, keeping in mind that it has no notion of connections but does have ports for separating multiple udp applications. Data to be transferred at a time should be send in a single packet.

PyCrypto Install error: Unable to find vcvarsall.bat

Today, there is a wechat SDK library called wechatpy, in which a dependency library is pycrypt

If you directly PIP install wechatpy, you will encounter error: unable to find vcvarsall.bat

Google, a bunch of solutions, are invalid, should be expired

Then I found two good articles

1. It introduces Microsoft Visual C + + compiler for Python 2.7

http://www.microsoft.com/en-us/download/details.aspx?id=44266

2. It introduces that if the version of setuptools is higher than 6.0, the compiler can be searched automatically

http://stackoverflow.com/questions/26140192/microsoft-visual-c-compiler-for-python-2-7

 

Then, first install 1, then check your own setup tools, version 3.6, and decisively upgrade to 7.0

pipinstall--upgradesetuptools

Then PIP install wechatpy again, everything goes well

Record it

Python TypeError: unbound method a() must be called with A instance as first argument (go…

TypeError: unbound method a() must be called with A instance as first argument (got nothing instead)

# encoding: utf-8

import time
import threading

class test:

    def follow(self,thefile):
        thefile.seek(0,2)
        while True:
            line = thefile.readline()
            if not line:
                time.sleep(0.1)
                continue
                yield line

if __name__ == '__main__':

    # obj = test()
    file1 = 'log.txt'
    file2 = 'result.txt'
    logfile = open(file2,"r")
    # loglines = obj.follow(logfile)
    loglines = test.follow(logfile)
    for line in loglines:
        print line,

output

C:\Python27\python.exe C:/Users/sys_syspifarm/.ssh/MagicBox/source/test.py
Traceback (most recent call last):
File “C:/Users/sys_syspifarm/.ssh/MagicBox/source/test.py”, line 44, in <module>
loglines = test.follow(logfile)
TypeError: unbound method follow() must be called with test instance as first argument (got file instance instead)

Process finished with exit code 1

Reason for error: The function a() is not a static method, so it needs to be instantiated before it can be used, corrected as follows.

# encoding: utf-8

import time
import threading

class test:

    def follow(self,thefile):
        thefile.seek(0,2)
        while True:
            line = thefile.readline()
            if not line:
                time.sleep(0.1)
                continue
                yield line

if __name__ == '__main__':

    obj = test()
    file1 = 'log.txt'
    file2 = 'result.txt'
    logfile = open(file2,"r")
    loglines = obj.follow(logfile)
    # loglines = test.follow(logfile)
    for line in loglines:
        print line,

Obj = test() needs to instantiate the method

Python SyntaxError: unexpected character after line continuation character

Python_ Syntax error: unexpected character after line continuation character

Reason: the content of the written file is incorrect and should be processed as a string

>>> import os
>>> os.makedirs(time_year+"\\"+time_month+"\\"+time_day)#time_year、time_month、time_day
>>> os.chdir(time_year\time_month\time_day)#Wrong
  File "<stdin>", line 1
    os.chdir(time_year\time_month\time_day)
                                          ^
SyntaxError: unexpected character after line continuation character

This is OK: OS. Chdir (time)_ year+”\\”+time_ month+”\\”+time_ day)

Refer to the writing method of creating directory in the first line

>>> os.chdir(time_year+"\\"+time_month+"\\"+time_day)#Right
>>> with open(time_hours+".txt","w+") as fp:
...     fp.read()
...

Python Open File SyntaxError: (unicode error) ‘unicodeescape’ codec can’t decode bytes in …

#path is the file path

f=open(‘C:\Users\jingqiu\Desktop\New Text Document.txt’)

SyntaxError: (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape

The file path plus r becomes f=open(r‘C:\Users\jingqiu\Desktop\New Text Document.txt’)

or f=open(‘C:\\Users\\jingqiu\\Desktop\\New Text Document.txt’)

the output of f.read() is …

Traceback (most recent call last):
File “<pyshell#4>”, line 1, in <module>
f.read()
File “C:\Users\xiuwe\AppData\Local\Programs\Python\Python37\lib\encodings\cp1252.py”, line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: ‘charmap’ codec can’t decode byte 0x8d in position 17: character maps to <undefined>

encode issue

f = open(r’C:\Users\jingqiu\Desktop\New Text Document.txt’,encoding=’utf-8′)

f.read()

OK ^_^

Python3: list dict set [UNK]unhashable type

 

code

coder@ubuntu:~$ source activate py37
(py37) coder@ubuntu:~$ ipython
Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: hash(list())                                                            
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-3e2eb619e4e4> in <module>
----> 1 hash(list())

TypeError: unhashable type: 'list'

In [2]: hash(dict())                                                            
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-7762fff637c6> in <module>
----> 1 hash(dict())

TypeError: unhashable type: 'dict'

In [3]: hash(set())                                                             
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-2699417ebeac> in <module>
----> 1 hash(set())

TypeError: unhashable type: 'set'

In [4]: exit                                                                    
(py37) coder@ubuntu:~$ conda deactivate
coder@ubuntu:~$

source_code

def hash(*args, **kwargs): # real signature unknown
    """
    Return the hash value for the given object.
    
    Two objects that compare equal must also have the same hash value, but the
    reverse is not necessarily true.
    """
    pass

 

python3 are installed through pip on Win 10, Unicode decodeerror: ‘UTF-8’ codec can’t decode byte

UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xd5 in position 2: invalid continuation byte

when installing Python3 via pip on Win10
I encountered UnicodeDecodeError when installing via pip.

The system is windows 10 and the python version is 3.6.0.

The complete message is as follows.

C:\Windows\system32>pip install SQLAlchemy
Collecting SQLAlchemy
  Using cached SQLAlchemy-1.1.4.tar.gz
Installing collected packages: SQLAlchemy
  Running setup.py install for SQLAlchemy ... error
Exception:
Traceback (most recent call last):
  File "c:\program files\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 34: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\program files\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "c:\program files\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "c:\program files\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "c:\program files\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "c:\program files\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "c:\program files\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 34: invalid start byte

I’ve tried to modify the code page with “CHCP 65001”, but it didn’t work. I have tried the method of “CHCP 866”, but it has no effect

I also found many ways to use sys. Set default encoding in various places, but I used Python 3

Update:

I saw on stackoverflow that it is suggested to modify the code page to 437. It is not only the above problem… But also the permissionerror: [winerror 32] another program is using this file and the process cannot access it

The error information is as follows:

C:\Users\59740>pip install SQLAlchemy
Collecting SQLAlchemy
  Using cached SQLAlchemy-1.1.4.tar.gz
Installing collected packages: SQLAlchemy
  Running setup.py install for SQLAlchemy ... error
Exception:
Traceback (most recent call last):
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 34: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 34: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\commands\install.py", line 385, in run
    requirement_set.cleanup_files()
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\req\req_set.py", line 729, in cleanup_files
    req.remove_temporary_source()
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\req\req_install.py", line 977, in remove_temporary_sou
    rmtree(self.source_dir)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 212, in call
    raise attempt.get()
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\six.py", line 686, in reraise
    raise value
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\utils\__init__.py", line 102, in rmtree
    onerror=rmtree_errorhandler)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\shutil.py", line 488, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\shutil.py", line 387, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\utils\__init__.py", line 114, in rmtree_errorhandler
    func(path)
PermissionError: [WinError 32] Another program is using this file and the process cannot access it. : 'C:\\Users\\59740\\AppData\\Local\\Temp\\pip-build-1djzmudb\\SQLAlchemy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\commands\install.py", line 385, in run
    requirement_set.cleanup_files()
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\utils\build.py", line 38, in __exit__
    self.cleanup()
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\utils\build.py", line 42, in cleanup
    rmtree(self.name)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 212, in call
    raise attempt.get()
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\six.py", line 686, in reraise
    raise value
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\_vendor\retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\utils\__init__.py", line 102, in rmtree
    onerror=rmtree_errorhandler)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\shutil.py", line 488, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\shutil.py", line 378, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\59740\appdata\local\programs\python\python36\lib\shutil.py", line 387, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\users\59740\appdata\local\programs\python\python36\lib\site-packages\pip\utils\__init__.py", line 114, in rmtree_errorhandler
    func(path)
PermissionError: [WinError 32] Another program is using this file and the process cannot access it. : 'C:\\Users\\59740\\AppData\\Local\\Temp\\pip-build-1djzmudb\\SQLAlchemy'

Solution:

Open
C: (program files \ \ python36 \ \ lib \ \ site packages \ \ PIP \ \ compat\__ init__. Py about 75 lines
return s.decode ('utf_ 8 ') to return s.decode ('cp936')

Reason:
coding problem, although PY3 uses UTF-8. But the terminal display under win still uses GBK code