i made a code from a tutorial, its practically the same. The guy run it from terminal with a .csv file as output, but when i run it, it appears a lot of options for running the file but doesnt make anything that i want but doesnt throw any error apparently. What am i doing wrong?, i run it like this:

I’m working on a Scrapy project and have a custom spider defined as follows:

I’m working on a Scrapy project and have a custom spider defined as follows:

I’m working on a Scrapy project and have a custom spider defined as follows:

The code scrapes job details from a career website.It returns the jobs from the first page and the same exact jobs for every other page when it tries to scrape the rest of the pages on the website.

The problem is that I can’t debug Scrapy crawlers in VS Code. The problem is that always when I start debugging it breaks on one of my imports. Of course, I played a lot with that import in order to fix it, but it didn’t helped. I also tried with venv and without venv, but it doesn’t helped.

I’m encountering an issue while trying to pass callback keyword arguments (cb_kwargs) in my Scrapy spider. Here’s a simplified version of my code structure:

I’m trying to scrape a website with a scroll-down loading page. The page does not load new elements but just updates the element’s content when scrolling down. So I’m trying to use selenium in the DownloaderMiddleware to scroll, and I want to return the current page_sorce every time I scroll. (as it is different every time)

I’m following a course. It’s a bit outdated so some stuff changed on the website.

I’m tring to scrap a racing website.from the site i have to scrapp all information of every associated riders to the team.LIsting of the Team is a entry point of the scraping process.