Category : selenium-webdriver

Still new(‘ish) to Selenium but I’ve scraped a number of pages and I’m super confused as to why I can’t get this to work without timing out because it cant find the element. I’m trying to just pull and save a large amount of Reddit source pages from Web.Archive.org to run separate script on later. ..

Read more

I am extracting google play store reviews using selenium webdriver. I want to extract the reviews from certain date onwards. For example, from 1-10-2020 to Today(). Currently I am extracting reviews using following code: from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By driver = webdriver.Chrome(path) ..

Read more

I am using selenium and python to do a big project, I have to go to 320000 webpages one by one and scrape details from each one and then sleep for a second and move on. As follows: links = ["https://www.thissite.com/page=1","https://www.thissite.com/page=2","https://www.thissite.com/page=3"] for i in links: browser.get(i) scrapedinfo = browser.find_elements_by_xpath("*//div/productprice").text open("file.csv","w").write(scrapedinfo) time.sleep(1) The Greatest problem with ..

Read more

Is it because i using python REPL? I don’t know what that is. full code and error image My code: from selenium import webdriver from selenium.webdriver.common.keys import Keys import time PATH = "C:Program Files (x86)chromedriver.exe" driver = webdriver.Chrome(PATH) driver.get("https://member.lazada.com.my/user/login?spm=a2o4k.home.header.d5.1e532e7eD9UgkJ&redirect=https%3A%2F%2Fwww.lazada.com.my%2F%3Fspm%3Da2o4k.login_signup.header.dhome.7d0d49fbWgHyQw") print(driver.title) time.sleep(10) driver.find_element_by_id("a2o4k.login_signup.menu.i0.128549fbdpzNIi").click() Error code: Lazada.com.my: Online Shopping Malaysia – Mobiles, Tablets, Home Appliances, TV, Audio ..

Read more

I made a code from bs4 import BeautifulSoup from selenium import webdriver from urllib.request import urlopen from urllib.parse import quote_plus import time url1 = "https://www.instagram.com/explore/tags/" url2 = input("write a hashtag : ") url = url1 + quote_plus(url2) driver = webdriver.Chrome(‘executable_path=r"C:Chromechromedriver.exe"’) driver.get(url) And I got an error message Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line ..

Read more