Category : beautifulsoup

I try to get the data from within the <script> tag but ‘model_data’ return None. When I run the code I get the error: model_data = model_data.group(1) AttributeError: ‘NoneType’ object has no attribute ‘group’ What is wrong here? html_doc = """ <script> var modelData = { "hlsUrl": "null", "account": "4LH7J44IYPAGEZEY6E3UL" } </script> """ soup = ..

Read more

I’m new in python. I have a question can I read a script value with python. import requests from bs4 import BeautifulSoup import json url=’https://www.iltelegrafolivorno.it/cronaca/radi-vandalico-vetrate-dustrutte-cecina-1.6231324′ r = requests.get(url) soup = BeautifulSoup(r.content, ‘html.parser’) print(soup.find_all(‘script’)[9]["cp30"]) Source: Python-3x..

Read more

from bs4 import BeautifulSoup import requests class Coin: """a simple way to represent a crypto coin""" stored_coins = [] #class attribute to track coin names def __init__(self, name): """initialize the attributes of the coin to describe coin""" self.name = name self.stored_coins = stored_coins def collect_info(self): for token in stored_coins: source = requests.get(f"https://coinmarketcap.com/currencies/{token}/") soup = BeautifulSoup(source,’lxml’) ..

Read more

from bs4 import BeautifulSoup as bs import requests import pandas as pd url = "https://www.property24.com/for-sale/woodland-hills-wildlife-estate/bloemfontein/free-state/10467/109825373" data = requests.get(url) soup = bs(data.content,"html.parser") The code below are a test with to get 1 item. property_overview = soup.find(class_="p24_regularListing").find(class_="p24_propertyOverview").find(class_=’p24_propertyOverviewRow’).find(class_=’col-xs-6 p24_propertyOverviewKey’).text property_overview Output : ‘Listing Number’ The code below is what we have to get all the col-xs-6 p24_propertyOverviewKey p24_regularListing_items ..

Read more

when I am scraping content from a web site some links dot have HTTP in the src tag for that I added this code from bs4 import BeautifulSoup html = “”” <div class=”answer-given-body ugc-base”> <p><img alt=”” src=”//d2vlcm61l7u1fs.cloudfront.net/media%2F61d%2F61d6042d-e4dd-41d9-9a5c-0ceb481ddbc9%2FphpKFGb9B.png”/><img alt=”” src=”//d2vlcm61l7u1fs.cloudfront.net/media%2Fd72%2Fd72dfa6c-8e50-475a-86cf-678a04ae4606%2FphpQZYPYo.png”/><img alt=”” src=”//d2vlcm61l7u1fs.cloudfront.net/media%2F4c7%2F4c775a01-8590-4b93-bc20-03d282586f95%2FphpE7XFWI.png”/></p> </div> “”” soup = BeautifulSoup(html, “html.parser”) # Select all the `img` tags for tag ..

Read more

I use Beautifulsoup to get data from a website where the data I need is inside a <script> tag. I get a response looking like this and want to get the content of "name", "thumbnailUrl, account, Id": <script type="text/javascript"> var modelData = { name: ‘somename’, thumbnailUrl: ‘https://website.com/blob/bG9uZ2RhbjovL0ZPVVIvbGRwcm9kLWRlL3ljb3B6YTY4N0pnQ2Nfc3JYcVV3VXc9PQ’, account: ‘5LH7J44IYPAGEZEYA9KIL’, Id: ‘someid’ }; store.initOmlib({"ClusterEndpoints":{"ONE":["http://us.site.me"],"TWO":["http://sg.site.me"],"FOUR":["http://de.site.me"],"FIVE":["http://in.site.me"],"SIX":["http://ja.site.me"],"SEVEN":["http://br.site.me"]},"ClusterEndpointsInternal":{"ONE":["http://usi.site.me"],"TWO":["http://sgi.site.me"],"FOUR":["http://dei.site.me"],"FIVE":["http://ini.site.me"],"SIX":["http://jai.site.me"],"SEVEN":["http://bri.site.me"]},"ClusterKeys":{"FIVE":"Cf0Mw0I2/cZf6alwfMhelEEOb6xq23IhPvC9E4eoaXU=","SIX":"bfYXVkWhs/gv+TCJ3EeeEE3oxiZRDpJO0fecUGdq2Qg=","ONE":"xkkzyNJmZ1DmNPxGwrykZ2O91f10KNXQvspa15nKKGs=","FOUR":"xMRCvh1eki9JEceBcV7Bx49uaQYpX8FdD0eZ+LCGqCc=","TWO":"XaG4I7b7wDOZ+lGHSPwbJ2HLkIFf0UGYAWz9c9LkiQk=","SEVEN":"LuSOGA/u5PL7rP8PG3cr6bqgQy7jXEv65iuHUX9ePQY="},"DefaultCluster":"ONE","IdpEndpoints":["http://idp.site.me"],"IdpKey":"MIOC9PS8KIwXOXSHtplBZLSpIqcifns0jzExtkHXw1g=","ReadOnlyEndpoints":["http://site.gg"],"ReadOnlyKey":"QKxHfdLVgbn+VYpnUiCFLMq/hhUpkpx7occEY3Z0Wnk="}, {"Id":"001026a1c1064a1b9305400814783c2385e2a978f13a","Secret":"0110de13b2187fe3078e13d9f6ad4e7567fdc143e915c9cb4df67ca"}); if ..

Read more

Hello all I’m trying to connect to web site with python beautifulsoup but when I try to connect I have visualise content of redirect page. Have you any suggest? Web page haven’t token. from requests import Session from bs4 import BeautifulSoup as bs import requests import re with requests.Session() as sess: login_page = sess.get("https://app5-eu.linkpulse.com/lp/login") data ..

Read more