Explain the function of static keyword and final keyword in Java in detail>>>

Python run error: runtimeerror: cudnn error: cudnn_ STATUS_ INTERNAL_ ERROR

Solution:

Add:

torch.cuda.set_device(0)

Nan solution for training RNN network loss

(1) The cause of gradient explosion can be solved by gradient ruling

GRAD_CLIP = 5
loss.backward()
torch.nn.utils.clip_grad_norm_(model.parameters(), GRAD_CLIP)
optimizer.step()

(2) Testmodel and evaluate

with torch.no_grad():

(3) Lower the learning rate

RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 ‘self’ in call to _ th_ addmm

There are three places in the code that need CUDA () conversion:

Whether the model is placed on CUDA model = model. To (device)

Whether the input data is put on CUDA data = data. To (device)

Whether the new tensor in the model is placed on CUDA P = torch. Tensor ([1]). To (device) 0

In the first article, model = model. To (device) is only instantiated in model__ init__() If instantiated in forward and used directly, the model will not be placed in CUDA

Here is an error code:

import torch
import torch.nn as nn


data = torch.rand(1, 10).cuda()


class TestMoule(nn.Module):
    def __init__(self):
        super(TestMoule, self).__init__()
        # self.linear = torch.nn.Linear(10, 2)

    def forward(self, x):
        # return self.linear(x)
        return torch.nn.Linear(10, 2)(x)


model = TestMoule()
model = model.cuda()

print(model(data))

RuntimeError: CUDA error: an illegal memory access was encountered

One of the above problems is that some functions under the NN module pass in GPU type data, with the following error code:

import torch

data = torch.randn(1, 10).cuda()

layernorm = torch.nn.LayerNorm(10)
# layernorm = torch.nn.LayerNorm(10).cuda()

re_data = layernorm(data)
print(re_data)

RuntimeError: CUDA error: device-side assert triggered

The category target of classification is not one-to-one corresponding to the softmax value of model output

Targets is a value of 1-3, but softmax calculates a value of 0-2, so the above error is prompted

df = pd.read_csv('data/reviews.csv')

def to_sentiment(score):
    score = int(score)
    if score <= 2:
        return 0
    elif score == 3:
        return 1
    else:
        return 2

df['sentiment'] = df.score.apply(to_sentiment)