Scrapy Spider – XPATH

  python, scrapy, web-scraping, xpath

can someone help me here? I’m not able to get the data from this page, I don’t know if I did something wrong with xpath. Any suggestion?

import scrapy

class CatalogoAplusSpider(scrapy.Spider):
    name = 'catalogo_aplus'
    allowed_domains = ['www.aplus-automotive.com']
    start_urls = ['http://www.aplus-automotive.com/index.php?pg=products&dil=pt&kategori_id=&marka=&model=&str=&grup=&aplusnumber=&oemnumber=&crossnumber=&sayfanum=0']

    def parse(self, response):
        for catalogo_aplus in response.xpath("//div[@class='urun-sonuclari-listesi']/table/tbody/tr/td"):
            yield{
                'cod_aplus': catalogo_aplus(".//div[@class='urun-sonuclari-listesi']/table/tbody/tr/td[2]/a").get(),
                'nome': catalogo_aplus(".//div[@class='urun-sonuclari-listesi']/table/tbody/tr/td[3]/a").get(),
                'fabricante': catalogo_aplus(".//div[@class='urun-sonuclari-listesi']/table/tbody/tr/td[4]/a").get(),
                'modelo': catalogo_aplus(".//div[@class='urun-sonuclari-listesi']/table/tbody/tr/td[5]/a").get(),
                'oem': catalogo_aplus(".//div[@class='urun-sonuclari-listesi']/table/tbody/tr/td[6]/a").get()
            }

When one of the fields has more than one value, how do I choose all of them? In some cases it has more than one OEM, I wanted to capture all available.

Example here

//div[@class='urun-sonuclari-listesi']/table/tbody/tr/td[6]/a

Source: Python Questions

LEAVE A COMMENT