Category : web-scraping

I have been trying to scrape data from investing.com and get Volume and Net Volume data for some bonds from the technical analysis charts (Example: https://ng.investing.com/rates-bonds/us912810rk60-streaming-chart) Technical analysis chart showing volume and net volume I attempted this using python requests module but got stuck when retrieving data from the graph itself. Source: Python..

Read more

This is my web scrap target site. https://www.aliexpress.com/wholesale?catId=0&SearchText=ipad&SortType=default&g=n&page=1 With this code, I can get 60 items. import time from selenium import webdriver from time import sleep options = webdriver.ChromeOptions() options.add_argument(‘headless’) options.add_experimental_option(‘excludeSwitches’, [‘enable-logging’]) options.add_argument(‘–lang=en’) driver = webdriver.Chrome(r’c:chromedriverchromedriver.exe’, options=options) options.add_argument( "user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36") url = ‘https://www.aliexpress.com/wholesale?catId=0&SearchText=ipad&SortType=default&g=n&page=1’ driver.get(url) ..

Read more

I am on this page on Tokyo Olympic Website I would like to get all the elements that has class name beginning with a specific string. For instance I want to get all elements that begin with ‘col-sm-‘. html.r.find(‘.col-sm-6’) gives me all elements that have class name col-sm-6. However I would like to get all ..

Read more

I’m trying to scrape all monetary policy reports on this ECB website here using python’s Selenium package. Below is my code: from selenium import webdriver CHROME_PATH = <INSERT_CHROME_PATH_HERE> url = "https://www.ecb.europa.eu/press/govcdec/mopo/html/index.en.html" xpath = """//*[@id=’snippet*’]/dd/div[2]/span/a | # xpath of monetary policy report links //*[@id=’snippet1′]/dd/div[2]/span/a | //*[@id=’snippet2′]/dd/div[2]/span/a | //*[@id=’snippet3′]/dd/div[2]/span/a | //*[@id=’snippet4′]/dd/div[2]/span/a | //*[@id=’snippet5′]/dd/div[2]/span/a | //*[@id=’snippet6′]/dd/div[2]/span/a | //*[@id=’snippet7′]/dd/div[2]/span/a ..

Read more