Python scrapy/ ERROR: Spider must return request, item, or None, got ‘Tag ‘insettlement program

When I write about a crawler today, I encounter a pit and prompt [scratch. Core. Scraper] error: spider must return request, item, or none, got ‘tag’ in & lt; GET https://www.

In fact, the reason is very unexpected. I use item in the code, and at the same time, the scratch uses item to pass data. As a result, this problem is caused

 for item in soup.select(".job-list-item"):
            uu=item.select_one("a").get('href').split("?")[0]
            if uu is not None:
                item['wz']=uu
                yield item

If you change yield to return, no error will be reported, but the data cannot be transferred. You need to change it to the following code

for itema in soup.select(".job-list-item"):
            uu=itema.select_one("a").get('href').split("?")[0]
            if uu is not None:
                item['wz']=uu
                yield item

 

Similar Posts: