Category Archives: Python

Ubuntu1.8 Install python3.7pip Error: “subprocess.CalledProcessError…lsb_release”

background

Install Python 3.7 on a Ubuntu 18.04. After that, try the PIP list command, and a long list of errors appear. I’ll post the last few lines:

File “/usr/local/lib/python3.7/subprocess.py”, line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘(‘lsb_release’, ‘-a’)’ returned non-zero exit status 1.

When executing the PIP list command, the LSB cannot be found_Release – a this command
solutions

1. LSB found_Release.py this file

sudo find/-name ‘lsb_release.py’

The results shown here are as follows. There may be differences between different system versions:

/usr/share/pyshared/lsb_release.py
/usr/lib/python2.7/dist-packages/lsb_release.py
/usr/lib/python3/dist-packages/lsb_release.py

2. Select the path containing python3 and copy the file to the error path/usr/local/lib/python3.7/

sudo cp /usr/lib/python3/dist-packages/lsb_release.py/usr/local/lib/python3.7/

Try to execute PIP list again and return to normal.

Python Files Error: SyntaxError: (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: tr

Syntaxerror was just reported when running the python file: ‘Unicode error)’ Unicode scape ‘codec can’t decode bytes in position 2-3: tr. in fact, the reason for this error is the escape problem.

sys.path.append('c:\Users\mshacxiang\VScode_project\web_ddt')

Cause analysis: in the windows system, the file path can be read by \, but in the Python string \, there is an escape meaning,

For example, \ t can represent tab and \ n represents line feed, so we need to take some measures to make \ not be interpreted as escape characters. There are currently 3 solutions

1. Add r before the path to keep the original value of the character.

sys.path.append(r'c:\Users\mshacxiang\VScode_project\web_ddt')

2. Replace with double backslash

sys.path.append('c:\\Users\\mshacxiang\\VScode_project\\web_ddt')

3. Replace with forward slash

sys.path.append('c:/Users/mshacxiang/VScode_project/web_ddt')

[Solved] Uncaught TypeError: Cannot read property ‘ownerDocument’ of null

/*********************************************************************

 * Uncaught TypeError: Cannot read property 'ownerDocument' of null
 * Description.
 * Trying to load the SVG file content via JS Ajax and then dynamically insert it into the div, reporting this error.
 * Solution, directly change the svg suffix to html, it works.
 *
 ********************************************************************/

For this reported error, did not understand much, anyway, after changing the file suffix (.svg -> .html), it works.

Python installs pandas library Error: Could not import the LZMA module appears when

environment

pyenv python =3.7.0
pandas = 1.3.2
OS = Ubuntu 16.04

report errors

There is a warning prompt after installing the pandas reference

Solution:

sudo apt-get install libbz2-dev
sudo apt-get install  lzma
sudo apt-get install  liblzma-dev	


pip install backports.lzma

Find the file lzma.py and modify line 27

find/-name lzma.py
/root/.pyenv/versions/3.7.3/lib/python3.7/lzma.py

try:
    from _lzma import *
    from _lzma import _encode_filter_properties, _decode_filter_properties
except ImportError:
    from backports.lzma import *
    from backports.lzma import _encode_filter_properties, _decode_filter_properties

Import pandas again.

No problem!

 

[Solved] TypeError: can’t convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

normal_traffic = np.concatenate((intrinsic_normal, content_normal, time_based_normal, host_based_normal, categorical_normal), axis=1)

report errors:

Traceback (most recent call last):
  File "test_wgan.py", line 165, in <module>
    main()
  File "test_wgan.py", line 24, in main
    test_ids(options)
  File "test_wgan.py", line 37, in test_ids
    data = reassemble(options.attack, adversarial, adversarial_ff, nor_nff, nor_ff)
  File "test_wgan.py", line 51, in reassemble
    adversarial_traffic = np.concatenate((intrinsic, content, time_based, host_based, categorical), axis=1)
  File "/root/miniconda3/envs/ids_attack/lib/python3.7/site-packages/torch/tensor.py", line 433, in __array__
    return self.numpy()
TypeError: can't convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Solution:

Change intrinsic_normal to intrinsic_normal.cuda().data.cpu().numpy() will report another error:

'numpy.ndarray' object has no attribute 'cuda'

Refer to ‘numpy.Ndarray’ object has no attribute ‘CUDA’, and set the intrinsic_ Convert normal to tensor type

intrinsic_normal = torch.tensor(intrinsic_normal).cuda().data.cpu().numpy()

Successfully solved

Error: failed building wheel for pycrypto (pycrypto installation in Python 3.6 environment under win)

python  3.6 install pycrypto 2.6.1 error
ERROR: Failed building wheel for pycrypto
Solution: pycrypto-2.6.1-cp36-cp36m-win_amd64.whl

    ERROR: Command errored out with exit status 1:
     command: 'c:\users\user\appdata\local\programs\python\python36\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\User\\AppData\\Local\\Temp\\pip-install-yh93wdr0\\pycrypto_ff3c5548ebc643dfa87630bf7a78a4b6\\setup.py'"'"'; __file__='"'"'C:\\Users\\User\\AppData\\Local\\Temp\\pip-install-yh93wdr0\\pycrypto_ff3c5548ebc643dfa87630bf7a78a4b6\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\User\AppData\Local\Temp\pip-record-3s3fcqco\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\user\appdata\local\programs\python\python36\Include\pycrypto'
         cwd: C:\Users\User\AppData\Local\Temp\pip-install-yh93wdr0\pycrypto_ff3c5548ebc643dfa87630bf7a78a4b6\
    Complete output (153 lines):
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.6
    creating build\lib.win-amd64-3.6\Crypto
    copying lib\Crypto\pct_warnings.py -> build\lib.win-amd64-3.6\Crypto
    copying lib\Crypto\__init__.py -> build\lib.win-amd64-3.6\Crypto
    creating build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\hashalgo.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\HMAC.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\MD2.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\MD4.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\MD5.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\RIPEMD.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\SHA.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\SHA224.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\SHA256.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\SHA384.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\SHA512.py -> build\lib.win-amd64-3.6\Crypto\Hash
    copying lib\Crypto\Hash\__init__.py -> build\lib.win-amd64-3.6\Crypto\Hash
    creating build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\AES.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\ARC2.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\ARC4.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\blockalgo.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\Blowfish.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\CAST.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\DES.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\DES3.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\PKCS1_OAEP.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\PKCS1_v1_5.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\XOR.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    copying lib\Crypto\Cipher\__init__.py -> build\lib.win-amd64-3.6\Crypto\Cipher
    creating build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\asn1.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\Counter.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\number.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\py3compat.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\randpool.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\RFC1751.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\winrandom.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\_number_new.py -> build\lib.win-amd64-3.6\Crypto\Util
    copying lib\Crypto\Util\__init__.py -> build\lib.win-amd64-3.6\Crypto\Util
    creating build\lib.win-amd64-3.6\Crypto\Random
    copying lib\Crypto\Random\random.py -> build\lib.win-amd64-3.6\Crypto\Random
    copying lib\Crypto\Random\_UserFriendlyRNG.py -> build\lib.win-amd64-3.6\Crypto\Random
    copying lib\Crypto\Random\__init__.py -> build\lib.win-amd64-3.6\Crypto\Random
    creating build\lib.win-amd64-3.6\Crypto\Random\Fortuna
    copying lib\Crypto\Random\Fortuna\FortunaAccumulator.py -> build\lib.win-amd64-3.6\Crypto\Random\Fortuna
    copying lib\Crypto\Random\Fortuna\FortunaGenerator.py -> build\lib.win-amd64-3.6\Crypto\Random\Fortuna
    copying lib\Crypto\Random\Fortuna\SHAd256.py -> build\lib.win-amd64-3.6\Crypto\Random\Fortuna
    copying lib\Crypto\Random\Fortuna\__init__.py -> build\lib.win-amd64-3.6\Crypto\Random\Fortuna
    creating build\lib.win-amd64-3.6\Crypto\Random\OSRNG
    copying lib\Crypto\Random\OSRNG\fallback.py -> build\lib.win-amd64-3.6\Crypto\Random\OSRNG
    copying lib\Crypto\Random\OSRNG\nt.py -> build\lib.win-amd64-3.6\Crypto\Random\OSRNG
    copying lib\Crypto\Random\OSRNG\posix.py -> build\lib.win-amd64-3.6\Crypto\Random\OSRNG
    copying lib\Crypto\Random\OSRNG\rng_base.py -> build\lib.win-amd64-3.6\Crypto\Random\OSRNG
    copying lib\Crypto\Random\OSRNG\__init__.py -> build\lib.win-amd64-3.6\Crypto\Random\OSRNG
    creating build\lib.win-amd64-3.6\Crypto\SelfTest
    copying lib\Crypto\SelfTest\st_common.py -> build\lib.win-amd64-3.6\Crypto\SelfTest
    copying lib\Crypto\SelfTest\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\common.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_AES.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_ARC2.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_ARC4.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_Blowfish.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_CAST.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_DES.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_DES3.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_pkcs1_15.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_pkcs1_oaep.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\test_XOR.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    copying lib\Crypto\SelfTest\Cipher\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Cipher
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\common.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_HMAC.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_MD2.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_MD4.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_MD5.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_RIPEMD.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_SHA.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_SHA224.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_SHA256.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_SHA384.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\test_SHA512.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    copying lib\Crypto\SelfTest\Hash\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Hash
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Protocol
    copying lib\Crypto\SelfTest\Protocol\test_AllOrNothing.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Protocol
    copying lib\Crypto\SelfTest\Protocol\test_chaffing.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Protocol
    copying lib\Crypto\SelfTest\Protocol\test_KDF.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Protocol
    copying lib\Crypto\SelfTest\Protocol\test_rfc1751.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Protocol
    copying lib\Crypto\SelfTest\Protocol\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Protocol
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\PublicKey
    copying lib\Crypto\SelfTest\PublicKey\test_DSA.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\PublicKey
    copying lib\Crypto\SelfTest\PublicKey\test_ElGamal.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\PublicKey
    copying lib\Crypto\SelfTest\PublicKey\test_importKey.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\PublicKey
    copying lib\Crypto\SelfTest\PublicKey\test_RSA.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\PublicKey
    copying lib\Crypto\SelfTest\PublicKey\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\PublicKey
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Random
    copying lib\Crypto\SelfTest\Random\test_random.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random
    copying lib\Crypto\SelfTest\Random\test_rpoolcompat.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random
    copying lib\Crypto\SelfTest\Random\test__UserFriendlyRNG.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random
    copying lib\Crypto\SelfTest\Random\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Random\Fortuna
    copying lib\Crypto\SelfTest\Random\Fortuna\test_FortunaAccumulator.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\Fortuna
    copying lib\Crypto\SelfTest\Random\Fortuna\test_FortunaGenerator.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\Fortuna
    copying lib\Crypto\SelfTest\Random\Fortuna\test_SHAd256.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\Fortuna
    copying lib\Crypto\SelfTest\Random\Fortuna\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\Fortuna
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Random\OSRNG
    copying lib\Crypto\SelfTest\Random\OSRNG\test_fallback.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\OSRNG
    copying lib\Crypto\SelfTest\Random\OSRNG\test_generic.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\OSRNG
    copying lib\Crypto\SelfTest\Random\OSRNG\test_nt.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\OSRNG
    copying lib\Crypto\SelfTest\Random\OSRNG\test_posix.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\OSRNG
    copying lib\Crypto\SelfTest\Random\OSRNG\test_winrandom.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\OSRNG
    copying lib\Crypto\SelfTest\Random\OSRNG\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Random\OSRNG
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Util
    copying lib\Crypto\SelfTest\Util\test_asn1.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Util
    copying lib\Crypto\SelfTest\Util\test_Counter.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Util
    copying lib\Crypto\SelfTest\Util\test_number.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Util
    copying lib\Crypto\SelfTest\Util\test_winrandom.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Util
    copying lib\Crypto\SelfTest\Util\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Util
    creating build\lib.win-amd64-3.6\Crypto\SelfTest\Signature
    copying lib\Crypto\SelfTest\Signature\test_pkcs1_15.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Signature
    copying lib\Crypto\SelfTest\Signature\test_pkcs1_pss.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Signature
    copying lib\Crypto\SelfTest\Signature\__init__.py -> build\lib.win-amd64-3.6\Crypto\SelfTest\Signature
    creating build\lib.win-amd64-3.6\Crypto\Protocol
    copying lib\Crypto\Protocol\AllOrNothing.py -> build\lib.win-amd64-3.6\Crypto\Protocol
    copying lib\Crypto\Protocol\Chaffing.py -> build\lib.win-amd64-3.6\Crypto\Protocol
    copying lib\Crypto\Protocol\KDF.py -> build\lib.win-amd64-3.6\Crypto\Protocol
    copying lib\Crypto\Protocol\__init__.py -> build\lib.win-amd64-3.6\Crypto\Protocol
    creating build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\DSA.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\ElGamal.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\pubkey.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\RSA.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\_DSA.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\_RSA.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\_slowmath.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    copying lib\Crypto\PublicKey\__init__.py -> build\lib.win-amd64-3.6\Crypto\PublicKey
    creating build\lib.win-amd64-3.6\Crypto\Signature
    copying lib\Crypto\Signature\PKCS1_PSS.py -> build\lib.win-amd64-3.6\Crypto\Signature
    copying lib\Crypto\Signature\PKCS1_v1_5.py -> build\lib.win-amd64-3.6\Crypto\Signature
    copying lib\Crypto\Signature\__init__.py -> build\lib.win-amd64-3.6\Crypto\Signature
    Skipping optional fixer: buffer
    Skipping optional fixer: idioms
    Skipping optional fixer: set_literal
    Skipping optional fixer: ws_comma
    running build_ext
    warning: GMP or MPIR library not found; Not building Crypto.PublicKey._fastmath.
    building 'Crypto.Random.OSRNG.winrandom' extension
    error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\user\appdata\local\programs\python\python36\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\User\\AppData\\Local\\Temp\\pip-install-yh93wdr0\\pycrypto_ff3c5548ebc643dfa87630bf7a78a4b6\\setup.py'"'"'; __file__='"'"'C:\\Users\\User\\AppData\\Local\\Temp\\pip-install-yh93wdr0\\pycrypto_ff3c5548ebc643dfa87630bf7a78a4b6\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\User\AppData\Local\Temp\pip-record-3s3fcqco\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\user\appdata\local\programs\python\python36\Include\pycrypto' Check the logs for full command output.

Error: Can’t find Python executable “python”, you can set the PYTHON env variable. [Install the dependencies after pulling down the code]

Drop down the code and install the dependency. Finally, error: can’t find Python executable “Python”, you can set the python env variable

report errors:

Solution:

Input: npm install --global --production windows-build-tools

but report another error: Please restart this script from an administrative PowerShell!
The build tools cannot be installed without administrative rights.
To fix, right-click on PowerShell and run “as Administrator”

Solution:
Run cmd as administrator and type the above command again

Done!

Spider Error: Scratch processing timeout [How to Solve]

Previously, the timeout exception was handled in download middleware, but it was always very laborious

Today, I checked the document and found that it can be processed in the errback callback

from scrapy.spidermiddlewares.httperror import HttpError
from twisted.internet.error import DNSLookupError
from twisted.internet.error import TimeoutError, TCPTimedOutError


yield scrapy.Request(url=full_url, errback=self.error_httpbin, dont_filter=True, callback=self.parse_list, meta={"hd": header})


def error_httpbin(self, failure):
        # failure.request is the Request object, if you need to retry, directly yield can
        # if failure.check(HttpError):
        # these exceptions come from HttpError spider middleware
        # you can get the non-200 response
        # response = failure.value.response
        # self.logger.error('HttpError on %s', response.url)

        if failure.check(DNSLookupError):
            print("DNSLookupError------->")
            # this is the original request
            request = failure.request
            yield request
            # self.logger.error('DNSLookupError on %s', request.url)
        elif failure.check(TimeoutError, TCPTimedOutError):
            print("timeout------->")
            request = failure.request
            yield request
            # self.logger.error('TimeoutError on %s', request.url)

It is hereby recorded that the timeout exception has not been handled in this way before